TLDR

ArtReview24 is a python-based script to playfully point out the increasing blurred line between AI and human communication regarding our understanding of art.

Introduction

Artificial intelligence and machine learning have an immense impact on our everyday lives: Autonomous systems such as ID scans, improved communication systems up to augmented reality seem to be indispensable and offer great potential. Self driving cars, talking assistants and rather inconspicuous bots learn and mimic behavioural patterns, making AI more and more indistinguishable.

Even though recognition software development is at its peak, blind spots have occured, says Prof. Fei-Fei Li, introducing an image recognition software she has been working on.

When shown a statue of a horseman in a town the software recognized “A man riding his horse down a street next to a building”. “We haven’t taught him art 1x1 yet” Prof. Li jokingly mentions, as she continues to compare the software to a young child’s vision.

We quickly set about testing and working with the software to find out it’s way of working and limits. Although accurate keywords and precise descriptions create a human-like appearance, some incongrues, out of place statements break the illusion, revealing the AI working in the background.

This unsuspecting, innocent and child-like perspective made us curious to learn more about AI and deep learning: How does it detect art differently from us? In what context does it put the individually recognized components? Which elements take center stage, which doesn’t it even notice? How does it differ from human observations and reviews and what does it say about our point of view?

How it works

To question and play with human and artificial communication in terms of art we used the software language Python to link several softwares in order to create a bot, commenting on artworks. We feature the MET New York Museum’s online art collection, which is the biggest online archive with more than 400.000 pieces of art, and let the bot randomly choose one image, that is then presented to the image recognition software Clarifai. It is capable of autonomously detecting over 11,000 elements, such as objects, themes and moods and proceeds to analyze the initial picture. The resulting keywords reveal the AI’s major attentions which are then assorted into a list of nouns and adjectives by another linked python script. To avoid iterations we have added a blacklist of obvious, art related phrases such as "art", "decoration", "design", "handwriting", ...

The final part is the embedding of Clarifai’s keywords into art reviews. As the script is unable to communicate itself in proper english and seriously form an opinion on aesthetics we are relying on a list of more than 150 comments on art, which we have recorded from social media and which will be continuously updated. The sentences convey lapidary comments as well as profound statements, from which clauses have been removed to be replaced with Clarifai’s keywords. The results are often not to be distinguished from human-likereviews due to emotional or well-grounded remarks. However grammar mistakes, repetitions and eventually misnomers offer reasons to doubt the art reviewer’s authenticity, breaking the illusion and revealing the deception. 

ArtReview24 does not only question AI’s blind spot but also the participants perception on machine learning and art, but also switches the relation within the communication model: Does it function as another participant with different views? Does it deliver a thought-provoking impulse on our own sense of art? Is it more likely to be in a curational role? Or even an artistic itself?

The graphic

ArtReview24’s results merge to a rough, rudimentary mashup: The randomly chosen artwork is left-aligned, as well as the art comment, for which we chose a bright red as the font colour, leading to a better legibility due to the striking contrast (most of the times). Then again, some compositions seem illegible, for instance when a reddish artwork appears, or the comment overlapping the shown art. The left-justified content also gives the impression of a virtual dialog or chat, where the opponent’s messages appear on the upper left corner. This blurs out the line between a real life interlocutor and an automated bot, just as well as content-related inconsistencies.

The name

As a reference to the many automated, self updating websites offering reviews on vacations, restaurants etc we decided align our bot with the numerous current service websites: Working 24/7 and focussing on rather quantity than quality, the titel predicts an all day service, operating a lot and fastly, bringing art and tech together.