Human Generated Data

Title

Untitled (New York City Reformatory, New Hampton, New York)

Date

May 1934-June 1934

People

Artist: Ben Shahn, American 1898 - 1969

Classification

Photographs

Credit Line

Harvard Art Museums/Fogg Museum, Gift of Bernarda Bryson Shahn, by exchange, P2000.42

Copyright

© President and Fellows of Harvard College

Human Generated Data

Title

Untitled (New York City Reformatory, New Hampton, New York)

People

Artist: Ben Shahn, American 1898 - 1969

Date

May 1934-June 1934

Classification

Photographs

Machine Generated Data

Tags

Amazon
created on 2019-04-07

Person 99.3
Human 99.3
Person 99.3
Person 99.2
Person 99.1
Person 98.9
Face 98.8
Person 98.4
Person 97.5
People 96.6
Person 96.3
Person 96.2
Person 95.5
Person 95.5
Person 94.5
Smile 93.3
Person 92.5
Person 92.3
Hat 76.9
Apparel 76.9
Clothing 76.9
Photography 63.2
Portrait 63.2
Photo 63.2
Head 61.4
Man 59.8
Team 59.3

Clarifai
created on 2018-03-23

people 99.9
group 98.8
many 98.7
adult 97.7
man 97.3
group together 94.6
portrait 93.5
wear 90.2
leader 89.8
military 89.5
administration 88.3
war 85.9
outfit 85.3
woman 84.2
several 80.2
music 79.4
crowd 74.5
uniform 74.5
soldier 73.8
facial hair 73.7

Imagga
created on 2018-03-23

kin 94
statue 36.3
sculpture 30.2
ancient 25.1
old 23.7
architecture 21.9
religion 21.5
history 21.5
stone 21.3
art 19.1
monument 17.7
culture 16.2
religious 15.9
historic 15.6
man 15.5
building 15.2
historical 15
travel 14.8
world 13.9
male 13.5
god 13.4
antique 13
tourism 12.4
marble 11.6
vintage 11.6
famous 11.2
people 11.2
temple 11.1
grandfather 11.1
city 10.8
soldier 10.8
holy 10.6
church 10.2
cash 10.1
landmark 9.9
roman 9.7
portrait 9.7
heritage 9.7
money 9.4
dollar 9.3
currency 9
army 8.8
person 8.5
figure 8.3
catholic 8.2
one 8.2
aged 8.1
symbol 8.1
bank 8.1
carving 8
decoration 8
love 7.9
face 7.8
warrior 7.8
museum 7.8
us 7.7
spirituality 7.7
spiritual 7.7
sky 7.7
bill 7.6
banking 7.4
business 7.3
uniform 7.3

Google
created on 2018-03-23

Microsoft
created on 2018-03-23

person 99.6
outdoor 98.2
group 77.8
player 63.8
posing 60.2
team 28.7
crowd 2.6

Face analysis

Amazon

Microsoft

Google

AWS Rekognition

Age 57-77
Gender Male, 97.5%
Confused 0.6%
Happy 95.4%
Sad 2.7%
Angry 0.3%
Disgusted 0.4%
Surprised 0.3%
Calm 0.2%

AWS Rekognition

Age 26-43
Gender Male, 98.8%
Sad 0.2%
Calm 0.1%
Confused 0.4%
Disgusted 0.1%
Surprised 0.5%
Happy 98.4%
Angry 0.3%

AWS Rekognition

Age 26-43
Gender Male, 99.4%
Happy 41.7%
Disgusted 4.4%
Calm 11.7%
Sad 5.8%
Confused 21.7%
Angry 8.5%
Surprised 6.1%

AWS Rekognition

Age 15-25
Gender Male, 95.3%
Confused 8.6%
Surprised 2.4%
Happy 3.3%
Angry 11.1%
Disgusted 4.3%
Sad 29.3%
Calm 41%

AWS Rekognition

Age 26-43
Gender Female, 64.1%
Confused 0.2%
Happy 99.5%
Sad 0.1%
Angry 0.1%
Disgusted 0%
Surprised 0.1%
Calm 0%

AWS Rekognition

Age 26-43
Gender Male, 98.6%
Surprised 2.1%
Angry 8%
Sad 22.3%
Calm 22.2%
Disgusted 16%
Confused 26.8%
Happy 2.6%

AWS Rekognition

Age 26-43
Gender Male, 99.7%
Calm 3.6%
Angry 3.5%
Disgusted 1.8%
Happy 77.2%
Confused 7.6%
Surprised 2.6%
Sad 3.8%

AWS Rekognition

Age 38-59
Gender Male, 97.1%
Surprised 1.3%
Confused 3%
Disgusted 0.7%
Calm 76.6%
Angry 4.5%
Sad 11%
Happy 2.8%

AWS Rekognition

Age 35-52
Gender Male, 94.2%
Sad 7.9%
Surprised 5.6%
Confused 6.6%
Happy 10.1%
Calm 59.1%
Disgusted 4.1%
Angry 6.6%

AWS Rekognition

Age 26-43
Gender Male, 98.3%
Happy 2.8%
Angry 4.2%
Disgusted 0.9%
Calm 75.7%
Sad 4.5%
Surprised 3.2%
Confused 8.6%

AWS Rekognition

Age 26-43
Gender Male, 98.2%
Sad 11.2%
Disgusted 9%
Surprised 5.5%
Calm 18%
Angry 21.2%
Happy 13.7%
Confused 21.4%

AWS Rekognition

Age 11-18
Gender Female, 83.4%
Surprised 0.3%
Confused 0.4%
Disgusted 0.6%
Calm 0.7%
Angry 0.2%
Sad 0.2%
Happy 97.6%

AWS Rekognition

Age 35-52
Gender Female, 92.8%
Calm 43.8%
Disgusted 28.4%
Angry 9.7%
Happy 5.4%
Confused 2.2%
Surprised 7.4%
Sad 3.1%

AWS Rekognition

Age 20-38
Gender Male, 83.6%
Calm 31.9%
Angry 11.5%
Disgusted 4.8%
Happy 28.7%
Confused 4.3%
Surprised 5.8%
Sad 13.2%

AWS Rekognition

Age 20-38
Gender Male, 91.4%
Happy 2.7%
Confused 2.8%
Sad 9.8%
Angry 4.6%
Calm 78.6%
Disgusted 0.6%
Surprised 0.9%

Microsoft Cognitive Services

Age 40
Gender Male

Microsoft Cognitive Services

Age 48
Gender Male

Microsoft Cognitive Services

Age 20
Gender Male

Microsoft Cognitive Services

Age 28
Gender Male

Microsoft Cognitive Services

Age 26
Gender Male

Microsoft Cognitive Services

Age 36
Gender Male

Microsoft Cognitive Services

Age 39
Gender Male

Microsoft Cognitive Services

Age 58
Gender Male

Microsoft Cognitive Services

Age 46
Gender Male

Microsoft Cognitive Services

Age 32
Gender Male

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very likely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very likely
Headwear Very likely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very likely
Headwear Unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very likely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very likely
Headwear Very likely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very likely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Possible
Headwear Possible
Blurred Very unlikely

Feature analysis

Amazon

Person 99.3%
Hat 76.9%

Captions

Microsoft

a group of people posing for a photo 96.9%
a group of people posing for the camera 96.8%
a group of people posing for a picture 96.7%