About the AI Explorer
The Harvard Art Museums is using computer vision to categorize, tag, describe, and annotate its collection of art pieces. Since the computer lacks any context or formal training in art history, the machine views and annotates our collection as if walking into an art museum for the first time. The perspective offered by AI leans closer to reflecting the public rather than experts. Currently, the Harvard Art Museums’ search interface relies on descriptions written by art historians. The addition of AI-generated annotations to our database makes the Harvard Art Museums’ art collection more accessible to non-specialists.
The Harvard Art Museums has collected artificially-generated data on artworks from five different AI and computer-vision services: Amazon Rekognition, Clarifai, Imagga, Google Vision, and Microsoft Cognitive Services. For each art piece, these services provide interpretations otherwise known as “annotations” that include generated tags and captions and object, face, and text recognition. When a user searches for a keyword, this site takes the user-inputted keyword and finds artworks that contain a matching machine-generated tag. From there, the user can go to an individual piece to see and compare the annotations from the five AI services.