Human Generated Data

Title

Melancholia (triptych)

Date

1946

People

Artist: Karl Zerbe, American 1903 - 1972

Classification

Paintings

Credit Line

Harvard Art Museums/Fogg Museum, Gift of the Estate of Herbert D. Schutz in memory of Herbert D. Schutz, Harvard College, Class of 1944 and Betty Prescott Schutz, Radcliffe College, Class of 1946, 2006.72.A-C

Human Generated Data

Title

Melancholia (triptych)

People

Artist: Karl Zerbe, American 1903 - 1972

Date

1946

Classification

Paintings

Machine Generated Data

Tags

Amazon
created on 2019-04-05

Art 97.5
Person 96.9
Human 96.9
Person 94.1
Painting 85.2
Poultry 80.5
Animal 80.5
Chicken 80.5
Bird 80.5
Fowl 80.5
Person 76.8
Person 75.7
Tower 68.7
Clock Tower 68.7
Architecture 68.7
Building 68.7
Person 60.5
Photography 60.2
Photo 60.2

Clarifai
created on 2018-04-19

illustration 99.9
art 99.9
painting 99.7
religion 98.9
print 98.6
people 98.5
god 98.3
saint 98.2
aura 97.9
lithograph 94.9
man 94.8
Renaissance 94.7
book 93.6
Mary 93.1
crown 92.4
woman 92.1
holy 91.2
adult 90.3
kneeling 90
baby 89.9

Imagga
created on 2018-04-19

comic book 100
print media 70.4
religion 46.6
church 44.4
art 40.8
religious 30
faith 28.7
god 24.9
culture 23.9
holy 22.2
window 21.4
old 20.9
prayer 20.3
temple 20.2
architecture 19.5
mosaic 19.2
cathedral 19.2
antique 18.3
saint 18.3
stained 18.3
spiritual 18.2
glass 17.9
bible 16.7
colorful 16.5
pray 16.5
traditional 15.8
gold 15.6
ancient 15.6
catholic 15.6
travel 15.5
spirituality 15.4
detail 15.3
icon 15
vintage 14.9
decoration 14.8
sacred 14.6
worship 14.5
painted 13.4
golden 12.9
color 12.8
symbol 12.8
painting 12.6
history 12.5
design 12.4
artistic 12.2
east 12.1
letter 11.9
sculpture 11.7
belief 11.7
stamp 11.6
door 11.4
scene 11.3
wall 11.1
tourism 10.7
museum 10.7
interior 10.6
post 10.5
oriental 10.4
black 10.2
historic 10.1
artwork 10.1
peace 10.1
masterpiece 9.9
postmark 9.9
postage 9.8
renaissance 9.8
postal 9.8
pattern 9.6
mail 9.6
historical 9.4
paint 9.1
printed 8.9
bangkok 8.9
catholicism 8.8
paintings 8.8
envelope 8.8
statue 8.6
famous 8.4
one 8.2
landmark 8.1
shows 7.9
shrine 7.9
painter 7.8
delivery 7.8
heritage 7.7
decorated 7.7
unique 7.6
decorative 7.5
building 7.1

Google
created on 2018-04-19

Microsoft
created on 2018-04-19

text 99.9
book 95.5
different 86.3
covered 77.5
various 68.6
many 65.1
bunch 61.6
items 56.3
variety 18.3
several 15.1

Face analysis

Amazon

Microsoft

Google

AWS Rekognition

Age 26-43
Gender Male, 54.3%
Disgusted 45%
Calm 46.1%
Happy 45.1%
Surprised 45.1%
Sad 49.5%
Angry 49%
Confused 45.2%

AWS Rekognition

Age 26-43
Gender Female, 56.3%
Surprised 0.3%
Happy 0%
Sad 7.2%
Confused 0.3%
Disgusted 0.2%
Angry 0.5%
Calm 91.6%

AWS Rekognition

Age 26-43
Gender Female, 63.5%
Angry 2.4%
Disgusted 2.1%
Sad 86.6%
Surprised 0.9%
Confused 1.4%
Happy 1.1%
Calm 5.5%

AWS Rekognition

Age 60-80
Gender Male, 96.8%
Sad 4.9%
Confused 2%
Surprised 0.8%
Angry 1.7%
Disgusted 0.2%
Happy 0.3%
Calm 90.1%

AWS Rekognition

Age 29-45
Gender Female, 54.5%
Angry 45.1%
Surprised 45%
Calm 45.2%
Sad 54.7%
Happy 45%
Disgusted 45%
Confused 45%

AWS Rekognition

Age 4-7
Gender Female, 50.3%
Sad 50.1%
Happy 49.6%
Calm 49.6%
Surprised 49.6%
Disgusted 49.5%
Confused 49.5%
Angry 49.7%

AWS Rekognition

Age 35-52
Gender Female, 54.6%
Calm 45.4%
Angry 45.2%
Confused 45.1%
Sad 54.2%
Disgusted 45%
Surprised 45%
Happy 45.1%

Microsoft Cognitive Services

Age 32
Gender Male

Microsoft Cognitive Services

Age 40
Gender Male

Microsoft Cognitive Services

Age 34
Gender Male

Microsoft Cognitive Services

Age 37
Gender Male

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very likely
Blurred Very unlikely

Feature analysis

Amazon

Person 96.9%
Painting 85.2%
Chicken 80.5%
Clock Tower 68.7%

Captions

Microsoft

a bunch of items that are on display 57.2%
a bunch of items that are on top of a book 35.4%
a bunch of items that are on a book 33.1%