Human Generated Data

Title

Mocking of Christ

Date

17th century

People

Artist: Unidentified Artist,

Classification

Paintings

Credit Line

Harvard Art Museums/Fogg Museum, Gift of Otto Naumann and Heidi D. Shafranek in honor of George S. Abrams, Harvard College ’54, Harvard Law ’57, 2021.413

Human Generated Data

Title

Mocking of Christ

People

Artist: Unidentified Artist,

Date

17th century

Classification

Paintings

Machine Generated Data

Tags

Amazon
created on 2022-08-15

Human 99.9
Person 99.9
Painting 99.3
Art 99.3
Person 95.8
Person 95.8
Person 94.4
Person 93.2
Person 89.2
Person 83.9
Person 83.1
Leisure Activities 82.8
Musical Instrument 74.4
Musician 74.4
Person 73.3
Person 62.6
Person 60.3
Person 46.6

Imagga
created on 2022-08-15

dark 27.6
person 26.3
black 23.6
people 21.2
night 18.6
body 18.4
sexy 17.7
light 16.8
musical instrument 16.6
fashion 15.8
adult 15.4
attractive 15.4
hot 15.1
disco 14.9
man 14.8
guitar 14.6
dancer 14.4
style 14.1
model 14
lady 13.8
posing 13.3
electric guitar 12.9
male 12.8
sensuality 12.7
one 12.7
art 12.7
dance 12.6
stringed instrument 12.6
fire 12.2
women 11.9
sensual 11.8
music 11.8
portrait 11.6
pretty 11.2
performer 10.9
human 10.5
erotic 10.4
passion 10.3
elegant 10.3
love 10.3
room 10
silhouette 9.9
fantasy 9.9
dancing 9.6
sax 9.6
seductive 9.6
party 9.5
club 9.4
makeup 9.2
cabaret 9.1
dress 9
religion 9
clothing 8.8
nightclub 8.8
evil 8.8
hair 8.7
energy 8.4
smoke 8.4
looking 8
darkness 7.8
face 7.8
motion 7.7
studio 7.6
elegance 7.6
wind instrument 7.5
stylish 7.2
device 7.2
romance 7.1
spot 7.1
modern 7

Google
created on 2022-08-15

Drum 86.5
Musical instrument 85.3
Art 84.7
Painting 84.3
Snapshot 74.3
Event 71.9
Artist 71.7
Visual arts 70.7
Suit 70.1
Stock photography 66.9
Vintage clothing 66.2
Illustration 64.5
History 61.1
Music 59.6
Musician 59.6
Room 58.8
Artwork 57.9
Drawing 57.4
Modern art 56.3
Collection 56.1

Microsoft
created on 2022-08-15

text 97.9
painting 97.8
person 92.5
posing 89.3
drawing 88.6
clothing 78.2
standing 77
group 63.8
old 52.4

Face analysis

Amazon

Microsoft

Google

AWS Rekognition

Age 39-47
Gender Male, 88%
Sad 99.9%
Fear 12.8%
Surprised 7%
Calm 3.4%
Angry 1.9%
Disgusted 1.3%
Confused 1.2%
Happy 0.4%

AWS Rekognition

Age 18-24
Gender Female, 99.9%
Calm 64.8%
Fear 23.7%
Surprised 8.6%
Happy 3.2%
Sad 2.8%
Angry 0.8%
Disgusted 0.8%
Confused 0.5%

AWS Rekognition

Age 30-40
Gender Male, 77.5%
Calm 82.8%
Fear 13%
Surprised 6.3%
Sad 2.6%
Disgusted 1.7%
Angry 0.5%
Happy 0.3%
Confused 0.2%

AWS Rekognition

Age 22-30
Gender Female, 99.7%
Calm 54.5%
Fear 30%
Sad 11%
Surprised 7.3%
Angry 1.1%
Disgusted 0.7%
Happy 0.6%
Confused 0.4%

AWS Rekognition

Age 38-46
Gender Male, 99.7%
Calm 99.9%
Surprised 6.3%
Fear 5.9%
Sad 2.2%
Happy 0%
Angry 0%
Disgusted 0%
Confused 0%

AWS Rekognition

Age 30-40
Gender Male, 94.4%
Calm 84%
Sad 12.6%
Surprised 6.5%
Fear 6%
Confused 0.7%
Angry 0.6%
Happy 0.4%
Disgusted 0.3%

AWS Rekognition

Age 24-34
Gender Male, 99.7%
Calm 82.6%
Fear 10.5%
Surprised 6.6%
Sad 4.6%
Confused 0.8%
Angry 0.4%
Disgusted 0.3%
Happy 0.3%

Microsoft Cognitive Services

Age 44
Gender Male

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Feature analysis

Amazon

Person 99.9%
Painting 99.3%

Captions

Microsoft

a group of people posing for a photo 91.6%
a group of people posing for the camera 91.5%
a group of people posing for a picture 91.4%

Text analysis

Google

SE
SSREP
SSREP SE