Human Generated Data

Title

Untitled (copy photograph of Ben Shahn's Rikers Island Penitentiary mural study: prisoners descending stairs [2])

Date

1934-1935

People

Artist: Walker Evans, American 1903 - 1975

Classification

Photographs

Credit Line

Harvard Art Museums/Fogg Museum, Gift of Bernarda Bryson Shahn, P2000.55

Human Generated Data

Title

Untitled (copy photograph of Ben Shahn's Rikers Island Penitentiary mural study: prisoners descending stairs [2])

People

Artist: Walker Evans, American 1903 - 1975

Date

1934-1935

Classification

Photographs

Machine Generated Data

Tags

Amazon
created on 2021-12-14

Banister 99.4
Handrail 99.4
Human 99.3
Person 99.3
Person 99.2
Person 96.2
Person 94.6
Person 94.6
Person 93.9
Person 92.9
Person 87.3
Person 85.9
Person 82.4
Railing 77.7
People 62.8
Prison 58
Porch 57.6

Imagga
created on 2021-12-14

architecture 53.3
building 40.5
city 32.4
old 30.7
step 28.3
structure 25.3
support 24.2
church 24
window 21
ancient 20.8
stone 20.2
urban 20.1
travel 19.7
tourism 19
history 18.8
bridge 18.7
device 17.8
brick 17.8
barrier 17.5
arch 16.8
construction 16.3
landmark 16.3
cathedral 16
famous 15.8
tower 15.2
exterior 14.7
wall 14.6
fence 13.8
historic 13.8
picket fence 13.6
roof 13.5
column 13.1
monument 13.1
house 13
balcony 12.8
facade 12.6
historical 12.2
street 12
tourist 11.9
religion 11.7
obstruction 11
statue 10.5
buildings 10.4
town 10.2
detail 9.7
palace 9.4
vacation 9
sky 8.9
style 8.9
dome 8.7
door 8.6
religious 8.4
art 7.9
columns 7.8
marble 7.7
modern 7.7
architectural 7.7
windows 7.7
united 7.6
cityscape 7.6
traditional 7.5
place 7.4
university 7.2
stairs 7.2
prison 7.1
glass 7

Google
created on 2021-12-14

Microsoft
created on 2021-12-14

text 98.2
drawing 94.7
sketch 85.3
cartoon 61.1

Face analysis

Amazon

Microsoft

Google

AWS Rekognition

Age 20-32
Gender Male, 95.5%
Angry 46.4%
Calm 24.7%
Sad 13.4%
Disgusted 10.9%
Surprised 1.8%
Confused 1.1%
Happy 0.9%
Fear 0.8%

AWS Rekognition

Age 49-67
Gender Male, 88.1%
Sad 59.8%
Fear 36.5%
Surprised 1.2%
Confused 1.1%
Calm 1%
Disgusted 0.1%
Angry 0.1%
Happy 0.1%

AWS Rekognition

Age 32-48
Gender Male, 83.8%
Calm 77.1%
Sad 16.8%
Angry 1.8%
Confused 1.3%
Happy 1.1%
Surprised 0.9%
Fear 0.7%
Disgusted 0.3%

AWS Rekognition

Age 16-28
Gender Male, 99.5%
Calm 87.4%
Angry 3.5%
Confused 2.9%
Disgusted 2.3%
Happy 1.5%
Sad 1%
Surprised 1%
Fear 0.4%

AWS Rekognition

Age 49-67
Gender Male, 92.4%
Surprised 42.7%
Calm 24.4%
Fear 12.2%
Angry 7.1%
Sad 6.7%
Confused 4.7%
Disgusted 1.5%
Happy 0.7%

AWS Rekognition

Age 32-48
Gender Male, 99.3%
Calm 99.1%
Happy 0.4%
Angry 0.2%
Sad 0.1%
Surprised 0.1%
Disgusted 0.1%
Confused 0%
Fear 0%

AWS Rekognition

Age 24-38
Gender Male, 97.3%
Sad 36.6%
Calm 33%
Fear 15.7%
Angry 5.2%
Confused 4.4%
Surprised 2.5%
Disgusted 1.9%
Happy 0.8%

AWS Rekognition

Age 32-48
Gender Male, 96%
Calm 54.3%
Sad 20.7%
Disgusted 9.5%
Angry 8.8%
Confused 2.3%
Fear 1.5%
Happy 1.4%
Surprised 1.4%

AWS Rekognition

Age 19-31
Gender Male, 95.9%
Calm 87.9%
Sad 10.6%
Angry 0.7%
Confused 0.5%
Happy 0.1%
Surprised 0.1%
Disgusted 0%
Fear 0%

AWS Rekognition

Age 38-56
Gender Male, 97.2%
Calm 79.6%
Sad 15.6%
Angry 2.8%
Fear 1.1%
Confused 0.5%
Disgusted 0.2%
Happy 0.1%
Surprised 0.1%

AWS Rekognition

Age 26-42
Gender Female, 73.4%
Sad 93.4%
Calm 3.6%
Angry 1.2%
Fear 0.9%
Happy 0.3%
Confused 0.2%
Surprised 0.2%
Disgusted 0.1%

AWS Rekognition

Age 49-67
Gender Male, 89.6%
Sad 62%
Calm 33.3%
Angry 2.9%
Fear 0.8%
Confused 0.4%
Happy 0.2%
Surprised 0.2%
Disgusted 0.1%

AWS Rekognition

Age 52-70
Gender Male, 75.1%
Fear 57.6%
Confused 15.2%
Sad 14.7%
Surprised 5.4%
Calm 3.8%
Angry 1.6%
Disgusted 1%
Happy 0.5%

AWS Rekognition

Age 20-32
Gender Female, 56.8%
Sad 89.2%
Calm 6.3%
Angry 1.7%
Fear 0.8%
Confused 0.7%
Happy 0.6%
Disgusted 0.4%
Surprised 0.3%

AWS Rekognition

Age 39-57
Gender Male, 55.5%
Calm 58.4%
Sad 25.8%
Angry 9.3%
Fear 2.4%
Happy 1.9%
Surprised 1.5%
Confused 0.5%
Disgusted 0.2%

AWS Rekognition

Age 24-38
Gender Male, 88.8%
Fear 32.2%
Angry 22.5%
Calm 22%
Disgusted 14.8%
Surprised 5.9%
Happy 1.6%
Confused 0.6%
Sad 0.3%

Microsoft Cognitive Services

Age 69
Gender Male

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Possible
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Possible
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Possible
Blurred Very unlikely

Feature analysis

Amazon

Person 99.3%

Captions

Microsoft

a group of people posing for a photo 80.8%
a group of people pose for a photo 80.7%
a group of people standing in a room 80.6%