Dreaming Visual Data

At Impira, we’re finally gaining ground on the visual data problem, by taking cues from the original database technology: our brains.

In a book I recently read on the topic of sleep—Why We Sleep by Dr. Matthew Walker—I got a refresher on the functional difference between REM and NREM sleep in the brain’s encoding of memories. While NREM sleep is critical for the transfer of memories from the short-term depot (the hippocampus) to a long-term secure vault (the cortex), REM sleep is where the real magic of integration occurs. To quote the author (p. 132):

[REM] sleep provides a nighttime theater in which your brain tests out and builds connections between vast stores of information…In ways your waking brain would never attempt, the sleeping brain fuses together disparate sets of knowledge that foster impressive problem-solving capabilities.  

This contrast between NREM and REM struck me as analogous to the contrast between old-school and new-school approaches to information management—most especially in the realms of Digital Asset Management (DAM) and Product Information Management (PIM), which in large part concern the consolidation of visual data (like image and video files) into structured, searchable repositories.

I picked up Walker’s book mainly in hopes of persuading myself to catch a few extra Z’s come bedtime. So I was surprised when the insights embedded within it called to mind something I’ve been learning at the office—which is that in the world of DAM/PIM, anyone can pull off the storage/NREM-side (at least at smaller scales). The integration/REM-side is where things get really interesting.

Effective as they may be in warehousing large accumulations of digital assets and product data, traditional DAM/PIM systems don’t offer a way of organically integrating fresh data. Instead, they rely on users and administrators to process new arrivals and fit them into the existing structure. In the case of DAM and PIM, this primarily involves building out a taxonomy, curating metadata, and de-duplicating files to prevent messiness and overflow within particular taxonomic groupings (taxa).

This reliance on human housekeeping swiftly becomes a liability at increasing scale. As the volume of proprietary data grows, so too does the intricacy of the taxonomy and the density of its constituent taxa, which soon turn into content bottlenecks, impeding the flow of digital assets, products, and experiences rather than streamlining it.

But imagine if there were a REM-like functionality built into DAM/PIM systems, one charged with integrating their disparate data into a coherent, self-adaptive whole, and capable of restructuring that whole as needed given influxes of new data.

At the risk of putting too fine a point on it—imagine if your database could dream.

Imagine if, rather than simply ingesting data which then had to be tagged and sorted manually, your DAM/PIM systems autonomously built connections between those data—personalized connections that would improve over time as the systems grew more familiar with the data stored within them, with their distinctive taxa and nomenclatures.

Impira makes this—”data dreaming”—a reality through a blend of machine learning, computer vision, and API-based Smart Connections. When you upload an image or video file into Impira, our computer vision breaks that file down into its constituent visual elements and automatically enriches it with corresponding keywords to facilitate search. Meanwhile, machine learning models work to link visual assets to relevant information drawn from other data sources (like PIM, ERP, or cloud drives) and to related assets already banked in memory.

Impira also leverages feedback loops to improve its performance over time, tracking user interactions and continuously refining search results based on tiny pieces of feedback (like queries and clicks). Much as our conscious experiences often reverberate into our dreams, users of Impira see their conscious choices reflected in the software’s UI. And where non-dreaming databases require human users to structure new information, Impira integrates new assets and taxa automatically—in its sleep.