Human Generated Data

Title

Pen Box with Lovers, Flowers, and Birds
Date

c. 1850-1875
People
Classification

Artists' Tools
Credit Line

Harvard Art Museums/Arthur M. Sackler Museum, Gift of A. Soudavar in memory of his mother Ezzat-Malek Soudavar, 2014.333

Human Generated Data

Title

Pen Box with Lovers, Flowers, and Birds

People
Date

c. 1850-1875

Classification

Artists' Tools

Machine Generated Data

Tags

Amazon
created on 2019-04-09

Rug 89.6
Pencil Box 82.1
Art 72.6
Painting 69.8
Human 65.2
Person 65.2
Person 61.5

Clarifai
created on 2018-02-10

art 98
illustration 97.6
vintage 96.6
desktop 95.8
cutout 94
no person 93.2
design 93.1
one 92.7
retro 92.1
decoration 91.7
card 91.6
paper 91.2
graphic 90.9
painting 90.3
picture frame 90.2
sign 90
symbol 89.7
container 89
blank 88.9
color 88.7

Imagga
created on 2018-02-10

box 66.3
container 54.9
damask 22.8
paper 13.3
old 13.2
close 13.1
clip 13.1
fastener 12.9
object 12.5
arabesque 12.2
education 12.1
design 11.6
grunge 11.1
business 10.9
word 10.4
food 10.2
nobody 10.1
school 9.9
textured 9.6
black 9.6
money 9.4
art 9.3
single 9
gold 9
restraint 9
texture 9
decoration 9
currency 9
color 8.9
brown 8.8
symbol 8.7
text 8.7
alphabet 8.7
yellow 8.6
finance 8.4
chocolate 8.2
wealth 8.1
success 8
space 7.8
play 7.8
letters 7.7
horizontal 7.5
fashion 7.5
wood 7.5
frame 7.5
toy 7.4

Google
created on 2018-02-10

product design 59.9
rectangle 57.6

Microsoft
created on 2018-02-10

envelope 33.1
businesscard 22.3

Face analysis

Amazon

Microsoft

Google

AWS Rekognition

Age 20-38
Gender Male, 99.6%
Confused 7.8%
Angry 5.9%
Surprised 8.2%
Happy 2.7%
Calm 57.2%
Disgusted 13.7%
Sad 4.5%

AWS Rekognition

Age 17-27
Gender Female, 90.6%
Surprised 5.7%
Angry 7.6%
Happy 5.7%
Sad 25.8%
Disgusted 6.1%
Confused 7.1%
Calm 42%

Microsoft Cognitive Services

Age 16
Gender Male

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very likely
Blurred Very unlikely

Feature analysis

Amazon

Rug 89.6%
Person 65.2%

Captions

Microsoft

a room with white walls 23.3%
a person taking a selfie in a room 15.3%
a close up of a mans face in a room 15.2%