Human Generated Data

Title

Untitled (Civil Works Administration demonstration, United States Customs' House, New York City)
Date

December 1933-March 1934
People

Artist: Ben Shahn, American 1898 - 1969
Classification

Photographs
Credit Line

Harvard Art Museums/Fogg Museum, Gift of Bernarda Bryson Shahn, P1970.4247

Human Generated Data

Title

Untitled (Civil Works Administration demonstration, United States Customs' House, New York City)

People

Artist: Ben Shahn, American 1898 - 1969

Date

December 1933-March 1934

Classification

Photographs

Credit Line

Harvard Art Museums/Fogg Museum, Gift of Bernarda Bryson Shahn, P1970.4247

Machine Generated Data

Tags

Amazon
created on 2022-01-08

Interior Design 99.9
Indoors 99.9
Audience 99.8
Crowd 99.8
Human 99.8
Person 98.5
Room 96.2
Sailor Suit 88.6
Person 88.4
Monitor 86.2
Electronics 86.2
Screen 86.2
Display 86.2
Person 84.6
Speech 78.4
Person 71
Clothing 70.7
Apparel 70.7
Lecture 67.8
People 65.3
Theater 58.4
Hat 53.4
Person 42.2

Clarifai
created on 2023-10-25

people 99.8
group 98.7
many 97.7
adult 96.3
war 95.1
veil 94
man 94
child 92.2
group together 91.6
wear 90.8
lid 90.5
woman 90.2
art 90
military 89.8
soldier 88.4
portrait 87.9
retro 86.6
vintage 86.5
boy 85.3
nostalgia 83.3

Imagga
created on 2022-01-08

chair 55.3
classroom 45
seat 42.8
room 38.2
theater 31.9
cinema 31.4
keyboard 27.5
building 26.2
folding chair 24.4
computer 23.7
support 18.8
structure 18.2
row 16.8
furniture 15.2
business 15.2
technology 14.8
empty 13.7
laptop 13.2
device 12.4
industry 11.1
key 11
close 10.8
keys 10.7
water 10.7
group 10.5
landscape 10.4
monitor 10.1
equipment 10
restaurant 9.8
interior 9.7
pattern 9.6
hall 9.6
work 9.4
modern 9.1
color 8.3
letter 8.2
lake 8.2
closeup 8.1
button 7.9
objects 7.8
sea 7.8
audience 7.8
table 7.8
keypad 7.8
summer 7.7
communication 7.6
many 7.4
office 7.3
design 7.3
travel 7

Google
created on 2022-01-08

Hat 88
Fedora 82.3
Headgear 81.1
Art 80.7
Font 78.4
Sun hat 74.5
Crowd 72.1
Vintage clothing 71.7
Event 69.9
Monochrome 66
Room 65.5
History 64.1
Crew 60.8
Suit 59.9
Visual arts 59.6
Collection 59.5
Paper product 58.9
Monochrome photography 57.9
Working animal 57
Cowboy hat 56.1

Microsoft
created on 2022-01-08

indoor 91.8
person 86.4
clothing 82.2
man 75.4
hat 60.6
crowd 0.6

Color Analysis

Face analysis

Amazon

Google

AWS Rekognition

Age 11-19
Gender Male, 99.7%
Calm 99.6%
Surprised 0.2%
Confused 0%
Happy 0%
Sad 0%
Disgusted 0%
Angry 0%
Fear 0%

AWS Rekognition

Age 26-36
Gender Male, 96%
Calm 60.4%
Sad 35.9%
Fear 1%
Angry 1%
Happy 0.6%
Surprised 0.5%
Confused 0.4%
Disgusted 0.3%

AWS Rekognition

Age 23-31
Gender Male, 96.2%
Calm 95%
Angry 2.9%
Surprised 0.6%
Happy 0.4%
Sad 0.4%
Fear 0.3%
Confused 0.2%
Disgusted 0.2%

AWS Rekognition

Age 24-34
Gender Male, 98.4%
Calm 97.5%
Sad 2%
Confused 0.2%
Angry 0.1%
Surprised 0.1%
Fear 0%
Disgusted 0%
Happy 0%

AWS Rekognition

Age 36-44
Gender Male, 99.9%
Disgusted 79.7%
Calm 16.8%
Angry 1%
Confused 0.8%
Sad 0.8%
Surprised 0.4%
Happy 0.2%
Fear 0.2%

AWS Rekognition

Age 51-59
Gender Female, 98.8%
Calm 87.4%
Angry 5.9%
Sad 3.5%
Surprised 1.8%
Fear 0.6%
Confused 0.3%
Happy 0.3%
Disgusted 0.2%

AWS Rekognition

Age 31-41
Gender Male, 99.5%
Calm 68.4%
Sad 28.4%
Surprised 1%
Confused 0.9%
Angry 0.5%
Disgusted 0.4%
Fear 0.3%
Happy 0.1%

AWS Rekognition

Age 36-44
Gender Male, 99.7%
Calm 40.8%
Sad 36.7%
Angry 7.8%
Confused 5.8%
Happy 3.4%
Surprised 2.7%
Fear 1.6%
Disgusted 1.2%

AWS Rekognition

Age 22-30
Gender Female, 70.8%
Calm 76.9%
Angry 9.9%
Sad 3.4%
Surprised 2.8%
Confused 2.6%
Happy 2%
Disgusted 1.2%
Fear 1.2%

AWS Rekognition

Age 22-30
Gender Male, 93.5%
Sad 35.7%
Calm 32.2%
Angry 10.9%
Surprised 6.3%
Disgusted 5.2%
Fear 3.9%
Confused 3.5%
Happy 2.3%

AWS Rekognition

Age 19-27
Gender Male, 98%
Calm 99.4%
Angry 0.2%
Disgusted 0.1%
Surprised 0.1%
Sad 0.1%
Fear 0.1%
Confused 0.1%
Happy 0%

AWS Rekognition

Age 25-35
Gender Male, 99.8%
Calm 99.3%
Confused 0.2%
Happy 0.1%
Sad 0.1%
Angry 0.1%
Surprised 0.1%
Disgusted 0%
Fear 0%

AWS Rekognition

Age 21-29
Gender Female, 98.5%
Angry 60.3%
Calm 23.2%
Fear 9.8%
Sad 2.3%
Surprised 1.9%
Confused 1.2%
Happy 1%
Disgusted 0.4%

AWS Rekognition

Age 26-36
Gender Male, 99.9%
Calm 76.9%
Surprised 9.8%
Happy 3.4%
Angry 3.1%
Sad 2.2%
Confused 1.8%
Fear 1.7%
Disgusted 1%

AWS Rekognition

Age 25-35
Gender Male, 73.8%
Calm 94.8%
Angry 2.3%
Sad 1.9%
Surprised 0.3%
Disgusted 0.2%
Fear 0.2%
Happy 0.2%
Confused 0.1%

AWS Rekognition

Age 25-35
Gender Male, 87%
Angry 37.1%
Calm 22.3%
Confused 12.1%
Surprised 11.6%
Disgusted 7.2%
Fear 5.7%
Sad 2.8%
Happy 1.2%

AWS Rekognition

Age 22-30
Gender Male, 98.7%
Surprised 68.8%
Calm 22.7%
Happy 3.3%
Confused 1.9%
Angry 1.1%
Fear 0.9%
Disgusted 0.7%
Sad 0.5%

AWS Rekognition

Age 34-42
Gender Female, 96.6%
Angry 92.7%
Calm 4.6%
Sad 1.2%
Happy 0.5%
Fear 0.3%
Disgusted 0.2%
Confused 0.2%
Surprised 0.2%

AWS Rekognition

Age 26-36
Gender Male, 79%
Calm 77.4%
Angry 14.3%
Sad 5.3%
Surprised 0.8%
Fear 0.6%
Confused 0.6%
Disgusted 0.5%
Happy 0.5%

AWS Rekognition

Age 43-51
Gender Male, 94.3%
Calm 61.8%
Sad 8.9%
Confused 8.6%
Angry 7.9%
Happy 6%
Surprised 3.2%
Fear 1.9%
Disgusted 1.7%

AWS Rekognition

Age 18-26
Gender Female, 89.9%
Angry 58.5%
Calm 37.8%
Confused 1.5%
Sad 0.8%
Happy 0.4%
Disgusted 0.4%
Surprised 0.3%
Fear 0.2%

AWS Rekognition

Age 36-44
Gender Male, 99.9%
Calm 56%
Angry 14%
Happy 12.9%
Surprised 10.9%
Sad 2.5%
Disgusted 1.7%
Confused 1.4%
Fear 0.7%

AWS Rekognition

Age 24-34
Gender Male, 76.1%
Calm 54.1%
Fear 12.3%
Disgusted 7%
Happy 6.8%
Sad 5.9%
Confused 5.8%
Surprised 4.5%
Angry 3.7%

AWS Rekognition

Age 16-24
Gender Male, 98.6%
Calm 98.7%
Angry 0.8%
Confused 0.1%
Sad 0.1%
Surprised 0.1%
Disgusted 0.1%
Happy 0%
Fear 0%

AWS Rekognition

Age 21-29
Gender Male, 98.8%
Calm 97.9%
Confused 0.5%
Angry 0.4%
Sad 0.4%
Happy 0.3%
Disgusted 0.2%
Surprised 0.2%
Fear 0.1%

AWS Rekognition

Age 21-29
Gender Male, 98.2%
Calm 35.8%
Angry 33.7%
Sad 9.6%
Happy 7.7%
Confused 6.8%
Surprised 2.7%
Disgusted 2.6%
Fear 1.2%

AWS Rekognition

Age 36-44
Gender Female, 97.9%
Calm 46.7%
Angry 31.7%
Sad 10.9%
Surprised 3.9%
Happy 3.1%
Fear 1.8%
Confused 1.1%
Disgusted 0.7%

AWS Rekognition

Age 26-36
Gender Female, 88.5%
Calm 40.1%
Sad 33.3%
Surprised 11.9%
Fear 7.1%
Angry 2.9%
Confused 2.4%
Disgusted 1.6%
Happy 0.7%

AWS Rekognition

Age 24-34
Gender Male, 100%
Calm 94.6%
Sad 3.5%
Angry 0.6%
Surprised 0.5%
Confused 0.3%
Disgusted 0.3%
Fear 0.1%
Happy 0.1%

AWS Rekognition

Age 33-41
Gender Male, 100%
Calm 75%
Angry 15.8%
Sad 7.2%
Confused 0.7%
Happy 0.7%
Disgusted 0.3%
Surprised 0.1%
Fear 0.1%

AWS Rekognition

Age 30-40
Gender Male, 69.9%
Calm 86.6%
Fear 3.9%
Angry 2.5%
Sad 2%
Surprised 2%
Happy 1.4%
Confused 0.9%
Disgusted 0.6%

AWS Rekognition

Age 19-27
Gender Male, 99.8%
Calm 82.3%
Surprised 6.2%
Confused 3.4%
Sad 3.3%
Happy 2.2%
Disgusted 1.1%
Fear 0.8%
Angry 0.8%

AWS Rekognition

Age 41-49
Gender Female, 91.3%
Calm 98.5%
Angry 0.7%
Sad 0.3%
Happy 0.2%
Confused 0.1%
Surprised 0.1%
Disgusted 0.1%
Fear 0%

AWS Rekognition

Age 19-27
Gender Female, 52%
Calm 43.2%
Angry 30.7%
Surprised 10.8%
Sad 4.4%
Happy 3.9%
Fear 2.8%
Disgusted 2.8%
Confused 1.4%

AWS Rekognition

Age 13-21
Gender Female, 99.1%
Angry 86.8%
Sad 8.9%
Calm 3.1%
Surprised 0.3%
Fear 0.3%
Disgusted 0.2%
Happy 0.2%
Confused 0.2%

AWS Rekognition

Age 6-16
Gender Female, 86.2%
Sad 98%
Calm 1.1%
Fear 0.3%
Angry 0.2%
Confused 0.1%
Disgusted 0.1%
Surprised 0.1%
Happy 0.1%

AWS Rekognition

Age 23-33
Gender Male, 99.6%
Calm 56.2%
Happy 40.6%
Surprised 0.9%
Sad 0.8%
Angry 0.8%
Confused 0.3%
Disgusted 0.2%
Fear 0.1%

AWS Rekognition

Age 24-34
Gender Male, 99.8%
Calm 49.5%
Sad 30.8%
Surprised 5.3%
Happy 5.3%
Angry 2.7%
Disgusted 2.4%
Fear 2.3%
Confused 1.6%

AWS Rekognition

Age 24-34
Gender Male, 99.3%
Angry 67.1%
Fear 7.7%
Happy 7.5%
Calm 7.2%
Surprised 5.2%
Sad 2.3%
Confused 1.6%
Disgusted 1.5%

AWS Rekognition

Age 25-35
Gender Male, 90.3%
Calm 84%
Happy 5.4%
Angry 3.7%
Sad 2.5%
Surprised 1.7%
Confused 1.2%
Disgusted 1.2%
Fear 0.3%

AWS Rekognition

Age 26-36
Gender Female, 79.6%
Calm 47.7%
Sad 26.1%
Angry 9%
Happy 8.4%
Disgusted 3.5%
Confused 2.7%
Fear 1.3%
Surprised 1.2%

AWS Rekognition

Age 23-31
Gender Male, 84.9%
Calm 85.2%
Fear 6.4%
Angry 4.5%
Sad 1.9%
Surprised 0.7%
Disgusted 0.7%
Happy 0.3%
Confused 0.3%

AWS Rekognition

Age 34-42
Gender Female, 84.5%
Calm 60.5%
Happy 13.5%
Sad 10.1%
Angry 7.3%
Disgusted 2.5%
Surprised 2.3%
Confused 2.2%
Fear 1.7%

AWS Rekognition

Age 18-26
Gender Male, 99.7%
Calm 66.4%
Sad 23.7%
Confused 3.4%
Surprised 1.8%
Happy 1.7%
Angry 1.5%
Disgusted 1%
Fear 0.6%

AWS Rekognition

Age 24-34
Gender Female, 99.4%
Calm 77.3%
Angry 16.1%
Surprised 2.1%
Disgusted 1.4%
Sad 1.1%
Happy 1.1%
Fear 0.6%
Confused 0.3%

AWS Rekognition

Age 19-27
Gender Female, 100%
Calm 57.3%
Angry 35.3%
Surprised 2.9%
Sad 2.3%
Fear 0.6%
Happy 0.6%
Disgusted 0.5%
Confused 0.4%

AWS Rekognition

Age 14-22
Gender Female, 92.6%
Calm 62.8%
Sad 13.9%
Fear 9.2%
Angry 6.8%
Confused 2.2%
Disgusted 2.1%
Happy 2%
Surprised 1%

AWS Rekognition

Age 28-38
Gender Male, 90.9%
Calm 56.5%
Angry 14.9%
Surprised 8.3%
Happy 7.9%
Sad 4.1%
Disgusted 3.3%
Confused 2.8%
Fear 2.2%

AWS Rekognition

Age 29-39
Gender Male, 70.1%
Calm 98%
Angry 0.7%
Sad 0.4%
Happy 0.3%
Confused 0.2%
Surprised 0.2%
Disgusted 0.1%
Fear 0.1%

AWS Rekognition

Age 24-34
Gender Male, 80.8%
Happy 61.8%
Calm 22%
Angry 7.4%
Surprised 3.2%
Sad 1.6%
Disgusted 1.6%
Fear 1.4%
Confused 1%

AWS Rekognition

Age 27-37
Gender Male, 80.6%
Calm 41%
Surprised 17.9%
Happy 13%
Disgusted 10.4%
Fear 7.6%
Sad 3.9%
Confused 3.2%
Angry 3.1%

AWS Rekognition

Age 7-17
Gender Male, 67.2%
Calm 96.7%
Sad 1%
Surprised 0.7%
Angry 0.4%
Disgusted 0.4%
Happy 0.3%
Confused 0.3%
Fear 0.2%

AWS Rekognition

Age 18-26
Gender Male, 99.8%
Calm 95.7%
Angry 2.9%
Surprised 0.4%
Sad 0.3%
Disgusted 0.3%
Confused 0.2%
Happy 0.1%
Fear 0.1%

AWS Rekognition

Age 30-40
Gender Male, 57%
Calm 85.7%
Angry 9.2%
Sad 2%
Surprised 0.9%
Disgusted 0.8%
Confused 0.7%
Happy 0.5%
Fear 0.2%

AWS Rekognition

Age 16-22
Gender Male, 100%
Calm 60.8%
Fear 10.5%
Angry 8.4%
Sad 7.9%
Surprised 7.6%
Happy 2%
Confused 1.5%
Disgusted 1.3%

AWS Rekognition

Age 21-29
Gender Female, 99.9%
Calm 42.1%
Angry 37%
Sad 8.4%
Surprised 4.2%
Happy 2.6%
Confused 2.5%
Disgusted 2.3%
Fear 0.9%

AWS Rekognition

Age 29-39
Gender Female, 79.7%
Calm 36.6%
Fear 34.8%
Happy 11.8%
Surprised 9.7%
Angry 3.3%
Sad 1.7%
Disgusted 1.2%
Confused 0.9%

AWS Rekognition

Age 18-26
Gender Male, 98.6%
Calm 84.8%
Sad 7.5%
Surprised 2.7%
Fear 2.3%
Angry 1.1%
Confused 0.8%
Happy 0.4%
Disgusted 0.3%

AWS Rekognition

Age 25-35
Gender Male, 60.2%
Confused 54.8%
Sad 20.9%
Calm 12.7%
Surprised 7%
Fear 1.7%
Angry 1.5%
Disgusted 0.8%
Happy 0.6%

AWS Rekognition

Age 18-24
Gender Male, 87.5%
Calm 92.8%
Disgusted 1.5%
Fear 1.5%
Confused 1.5%
Sad 1%
Angry 0.7%
Surprised 0.5%
Happy 0.5%

AWS Rekognition

Age 22-30
Gender Male, 78.3%
Calm 64.4%
Angry 11.8%
Disgusted 8.5%
Sad 6.3%
Surprised 3.9%
Fear 2.2%
Happy 1.6%
Confused 1.3%

AWS Rekognition

Age 39-47
Gender Female, 94%
Angry 98.9%
Calm 0.8%
Sad 0.2%
Surprised 0%
Fear 0%
Confused 0%
Happy 0%
Disgusted 0%

AWS Rekognition

Age 12-20
Gender Male, 97.9%
Calm 98.2%
Confused 0.5%
Angry 0.5%
Sad 0.3%
Happy 0.2%
Fear 0.1%
Surprised 0.1%
Disgusted 0.1%

AWS Rekognition

Age 34-42
Gender Male, 99.5%
Calm 88.7%
Angry 7%
Sad 1.9%
Surprised 1%
Confused 0.6%
Happy 0.5%
Disgusted 0.3%
Fear 0.2%

AWS Rekognition

Age 45-51
Gender Male, 96.8%
Calm 99.6%
Sad 0.3%
Fear 0.1%
Happy 0%
Angry 0%
Confused 0%
Surprised 0%
Disgusted 0%

AWS Rekognition

Age 25-35
Gender Male, 99.7%
Calm 77.8%
Angry 13.4%
Sad 2.6%
Happy 1.7%
Surprised 1.3%
Confused 1.2%
Disgusted 1.1%
Fear 0.9%

AWS Rekognition

Age 34-42
Gender Male, 57.1%
Calm 90.3%
Angry 7.5%
Sad 1.2%
Happy 0.3%
Surprised 0.3%
Confused 0.2%
Disgusted 0.1%
Fear 0.1%

AWS Rekognition

Age 29-39
Gender Male, 55%
Angry 30.6%
Calm 30.6%
Sad 15.4%
Happy 7.6%
Confused 6%
Surprised 3.9%
Disgusted 3.5%
Fear 2.3%

AWS Rekognition

Age 4-10
Gender Male, 55.4%
Calm 57.9%
Sad 15.8%
Disgusted 8.2%
Angry 7.5%
Fear 6%
Surprised 2.1%
Happy 1.5%
Confused 0.9%

AWS Rekognition

Age 23-31
Gender Female, 75.7%
Calm 41.5%
Angry 18%
Sad 13.6%
Happy 9.8%
Confused 9.2%
Fear 3.4%
Surprised 2.7%
Disgusted 1.9%

AWS Rekognition

Age 19-27
Gender Male, 93.9%
Angry 51.3%
Sad 29.1%
Calm 15.5%
Surprised 1.2%
Fear 1.1%
Disgusted 0.8%
Confused 0.7%
Happy 0.4%

AWS Rekognition

Age 34-42
Gender Male, 98.7%
Calm 97.1%
Sad 1.3%
Fear 0.6%
Surprised 0.4%
Angry 0.3%
Confused 0.2%
Disgusted 0.1%
Happy 0.1%

AWS Rekognition

Age 27-37
Gender Male, 63.5%
Calm 90.2%
Angry 5.1%
Sad 2.4%
Surprised 1.1%
Confused 0.4%
Disgusted 0.3%
Fear 0.3%
Happy 0.2%

AWS Rekognition

Age 37-45
Gender Female, 70.8%
Calm 95.1%
Angry 1.3%
Sad 1.1%
Surprised 0.8%
Confused 0.6%
Happy 0.5%
Disgusted 0.4%
Fear 0.2%

AWS Rekognition

Age 25-35
Gender Male, 75.6%
Calm 71.4%
Sad 7.3%
Surprised 6.5%
Happy 4.7%
Angry 4.6%
Disgusted 2.2%
Fear 2%
Confused 1.4%

AWS Rekognition

Age 22-30
Gender Female, 85.3%
Calm 99%
Sad 0.4%
Angry 0.4%
Happy 0.1%
Confused 0.1%
Surprised 0%
Disgusted 0%
Fear 0%

AWS Rekognition

Age 12-20
Gender Female, 92.8%
Sad 51.6%
Calm 35.6%
Fear 3.8%
Angry 3.1%
Happy 1.8%
Disgusted 1.8%
Surprised 1.3%
Confused 0.9%

AWS Rekognition

Age 26-36
Gender Female, 67.3%
Calm 96.3%
Sad 1.4%
Surprised 0.6%
Angry 0.5%
Happy 0.5%
Confused 0.2%
Disgusted 0.2%
Fear 0.2%

AWS Rekognition

Age 24-34
Gender Female, 94%
Happy 56%
Surprised 15.5%
Calm 12.7%
Sad 9.2%
Disgusted 2.4%
Confused 1.4%
Angry 1.3%
Fear 1.3%

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Possible
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very likely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Possible
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Likely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Likely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Unlikely
Blurred Very unlikely

Feature analysis

Amazon

Person 98.5%
Monitor 86.2%
Hat 53.4%

Categories

Imagga

food drinks 77.6%
interior objects 15.4%
paintings art 3.6%
events parties 2%

Captions

Microsoft
created on 2022-01-08

an old photo of a person 63%
an old photo of a restaurant 62.9%
old photo of a person 60.4%

Text analysis

Amazon

в