Beauty Is in the Eye of the Beholder—but Memorability May Be Universal

Imagine spending a weekend afternoon with friends at an art museum: nodding with crossed arms, desperately searching for something insightful to say. The vast majority of paintings you stroll past are immediately forgotten, but some stick in your mind. As it turns out, the paintings you remember are likely the same ones everyone else does.

There’s a scientific term for that: image memorability. “It’s the idea that, essentially, there are some intrinsic patterns that make some content more memorable than others,” says Camilo Fosco, a PhD student studying computer science at MIT and the CTO of Memorable AI, a startup that uses machine learning to test how engaging content will be for advertisers and creators. In other words, certain pieces of art have that je ne sais quoi—and now a team of scientists is using AI to figure out what it is.

In a study published earlier this month in the Proceedings of the National Academy of Sciences, University of Chicago researchers Trent Davis and Wilma Bainbridge show that the memorability of artwork is not only consistent across people, but predictable by AI. In an online experiment, they pulled about 4,000 paintings from the Art Institute of Chicago’s database, excluding anything the institute labeled “boosted,” or especially famous. Over 3,200 people viewed hundreds of images so that each painting was seen by about 40 people. Then the volunteers were shown the paintings they had seen mixed in with ones they hadn’t and asked whether they remembered them or not. People were really consistent—everyone tended to remember (or forget) the same images.

Using a deep learning neural network called ResMem, designed by data scientist Coen Needell as part of his master’s thesis in Bainbridge’s psychology lab, the research team was able to predict how likely each painting was to be memorable. ResMem roughly mimics how the human visual system passes information from the retina to the cortex, first processing basic information like edges, textures, and patterns, then scaling up to more abstract information, like object meaning. Its memorability scores were very highly correlated with those given by people in the online experiment—even though the AI knew nothing about the cultural context, popularity, or significance of each artwork.

Counterintuitively, these findings suggest that our memory for art has less to do with subjective experiences of beauty and personal meaning, and more to do with the artwork itself—which may have major implications for artists, advertisers, educators, and anyone hoping to make their content stick in your brain. “You might think art is a very subjective thing,” says Bainbridge, “but people are surprisingly consistent in what they remember and forget.”

Although the online experiment was an intriguing start, she continues, “it’s more interesting if we can predict memory out in the real world.” So along with Davis, then an undergraduate double-majoring in neuroscience and visual arts, Bainbridge recruited 19 more people to actually wander through the museum’s American Art wing as though they were exploring with friends. The only requirement was that they see each piece at least once. “Especially as an artist myself, I wanted the results to apply to the real world,” says Davis, who is now the lab’s manager. “We wanted it to be a natural and enjoyable museum experience.”