Serendipitous technologies: a human-human-machine collaboration

By Marina DiMaio, Digital Potentials Advisory Coordinator

Sometimes the projects that we do at the AGGV do not always ‘fit’ within the standard white-cubed gallery spaces you will find in our building on Moss Street. Sometimes our curatorial projects take place in remote communities, deep in the basement archives, in collaboration with other arts institutions, or in this case, within a kind of algorithmic museum!

Last summer, as the Gallery formed the Digital Potentials Advisory (DPA), a cyber support community of digital scholars, the group quickly became anxious to start experimenting. This resulted in a spin-off project with DPA member and technologist Noya Kohavi. As the DPA turned to the potential of the AGGV’s vast digital archive of over 20,000 objects, Noya created LINEAGE — an AI visual discovery engine and proof-of-concept project that is now accessible to the public on our website.

Working with Carol Sawyer, Brendan Tang, Howie Tsui and Lorilee Wastasecoot — artists and curators who brought pre-existing interests and specializations in collections research to the project — Noya, fellow DPA member Emily Luce, and Michelle Jacques led a human-human-machine collaboration centered on queer computing as a methodology for computer vision.

The process was one of exploring new digital trails, of writing new code and centering a multiplicity of perspectives that constantly complicated the hierarchies inherent in the AGGV’s existing metadata. The group began by going back to the digital drawing board in an attempt to generate more fluid connections and frameworks for the AGGV’s collection — to create space for art objects that could have more than binary classifications.

Image credit: Howie Tsui, Speculative Webs, 2021. From a series of gifs produced as a creative response to the LINEAGE Project.  

While this exploratory project was naturally embraced by the artists and curators, the group also encountered the challenges and limitations offered by supervised vs. unsupervised machine learning (i.e. labeled vs. unlabeled datasets — unsupervised models work independently to learn the fundamental frameworks of unlabeled data). Noya ultimately decided to employ an unsupervised model to the LINEAGE search and discovery engine, because of the diverse range of categorical and thematic interests defined by the artists for these new fluid frameworks and non-binary classifications. So, while utilizing an unsupervised approach allowed us to center the group’s varied perspectives and contend with the aggregate scope of the AGGV’s collection, it also meant that we were left with groupings that are sometimes unclear, or a system filled with fascinating ambiguities that ideally would continue to train and improve over time.

Image credit: Howie Tsui, Speculative Webs, 2021. From a series of gifs produced as a creative response to the LINEAGE Project.  

As someone who has interacted with the AGGV’s collection from a variety of different perspectives, through various roles I’ve held at the gallery, I find the perplexing enigma of this new visual discovery engine exciting! In an almost poetic way, it forces us to pause and momentarily overlook the complicated vocabularies of art history that can sometimes dominate and predetermine the digital trails that we might meander. Sifting through this photomontage of the gallery’s colossal collection, I am able to wander around, and bring my intuition to the forefront — to discover disparate groupings of objects that I never would have found or connected if I was simply searching for them using a text-based search on eMuseum. It allows me to engage in a more expansive way of seeing. It puts the viewer in the position of the interpreter. And although the LINEAGE search engine is far from perfect, this necessary test phase points to a kind of deep digital learning — one that is not just about accuracy and mechanical perfection, but also about creating more fluid, just, accessible and serendipitous technologies. 

Image credit: Screenshot of a LINEAGE search result. LINEAGE discovery engine created by Noya Kohavi. Website designed by Andrew LeClair.

Noya Kohavi is a queer feminist technologist, consultant, and NEW INC member whose work utilizes computer vision and computational linguistics to generate new narratives and insights, focusing on data ethics and a queer approach to computing.

Emily Luce is an independent designer, artist, researcher, expert collaborator, and key instigator for the AGGV’s larger Digital Potentials Advisory project/team.

Carol Sawyer is an artist working primarily with photography, installation, video, and improvised music. Her practice explores connections between photography and fiction, performance, memory, and history. Conducting research in online databases is a large part of Sawyer’s creative process. 

Brendan Tang is a visual artist known for his sculptural ceramic work that explores issues of identity and the hybridization of material and non-material culture while simultaneously expressing a love of both futuristic technologies and ancient traditions. 

Howie Tsui works in a variety of media, constructing tense, fictive environments that subvert venerated art forms and narrative genres. His process has involved researching Chinese scroll painting in collections in Asia and Canada, and recently, he has been looking at photo archives/etc to re-imagine vernacular objects to populate his picture spaces.

Lorilee Wastasecoot is a Ininew iskwew (Cree woman) with ancestral roots stemming from Peguis First Nation, York Factory in northern Manitoba and in the Red River. She is currently Curator of Indigenous Art and Engagement at the Legacy Art Galleries at UVica where she works with Indigenous artists and communities to raise up Indigenous culture and knowledge through art.

Feature Image: Screenshot of LINEAGE discovery engine/website homepage. LINEAGE discovery engine created by Noya Kohavi. Website designed by Andrew LeClair.