According to new research by Mohammad Amin Sadeghi et al, the way we interact with the world might be simpler than we think. Their research focuses on intermediate composites between objects and scenes, what they call “visual phrases” such as “person riding a horse” or “dog on a couch” that help a machine recognize situations faster than seeing them as “person” and “horse” or “couch” and “dog” individually and then putting the pieces together.

What we are fascinated by, naturally, are the implications for the world of data storytelling with visualizations that show data in context.

Visual Phrases

According to Sadeghi “Visual phrases correspond to chunks of meaning bigger than objects and smaller than scenes.” They describe the experiment thus: “One might think of our system as having a phrase table with entities like ‘person’, ‘horse’, and ‘person riding horse’. The ultimate goal is to look at all phrases and find the longest phrase that matches.”

Essentially, Sadeghi and his colleagues were attempting to build a faster decoding algorithm, one which saw a picture in context rather than attempting to distinguish individual parts and then infer a situation.

And they were largely successful. Using their method visual decoders were able to pick out visual phrases with speed and accuracy, whereas trying to pick out the individual elements of those phrases was more difficult and time consuming.

Data Storytelling

Sadeghi was able to prove that seeing the whole picture made the story clear, even for a decoding algorithm. The authors point out that “the reduction in the visual complexity exhibited by visual phrases is often so great that very accurate detectors can be trained with little training data.” Their research showed that seeing the world in context, not as individual chunks which are then somehow combined, is far more efficient.

What the researchers didn’t mention, and perhaps is more interesting, is the connection to how our own minds work.

David Bordwell, cognitive theorist of note, writes “The astonishing progress in programming computers to execute many kinds of reasoning has led to reflections on whether this new machine might not offer an important analogy to human mentation.”

This is exactly the kind of work that Sadeghi undertakes: if decoding algorithms can recognize visual phrases faster than individual parts, can the same be said of us?

We believe the answer is an unqualified yes. Recent research has shown that the human visual system is incredibly fast, and can process entire images and determine whether or not they contain or do not contain an object.

The bottom line is that human beings, like the decoding algorithms they create, see the world in context. Breaking down the larger picture into individual chunks can be useful, but it must first start with the big picture, which our eyes receive and brains create automatically.

This process should lie at the heart of effective data visualization: focusing too completely on any one object blinds us to its context.  We can see a piece of operational data in a spreadsheet, and another piece of the picture in another spreadsheet. But until we can see them together, in context, the picture will never become clear.

That’s why VisualCue’s most basic unit of visualization, called the Tile, takes points of data from disparate spreadsheets, sources and databases reporting on a single person, process or asset and places them together. Think of it like seeing different parts of the picture at the same time. When you see the whole picture, the story becomes clear.

At least ten different points of data gathered in one
At least ten different points of data gathered in one

Put Tiles together and you get a Mosaic, a data visualization that lets you see the data your business generates in context so you can put the bigger story together.

The VisualCue Platform
Seeing all of the data in a single screen puts it in context and the story becomes clear

Using intuitive pictures to portray the whole story lets you not only see the big picture, but its constituent parts at the same time so you have actionable intelligence that immediately conveys how you can improve.

Until next time,

The VisualCrew