Human Generated Data

Title

Street scene, New Orleans, Louisiana

Date

1935

People

Artist: Ben Shahn, American 1898 - 1969

Classification

Photographs

Credit Line

Harvard Art Museums/Fogg Museum, Transfer from the Carpenter Center for the Visual Arts, 2.2002.3078

Human Generated Data

Title

Street scene, New Orleans, Louisiana

People

Artist: Ben Shahn, American 1898 - 1969

Date

1935

Classification

Photographs

Machine Generated Data

Tags

Amazon
created on 2021-12-15

Person 99.8
Human 99.8
Person 99.6
Person 99.5
Person 99.5
Person 99.4
Person 99.1
Person 98.3
Person 98.1
Person 97.2
Person 91.5
Person 90.4
Doctor 86.2
Person 81.8
People 77.4
Pet 76
Mammal 76
Animal 76
Dog 76
Canine 76
Person 73.1
Wildlife 61.3
Person 61
Person 50.3

Imagga
created on 2021-12-15

people 34.6
man 32.2
person 31.7
kin 27.8
male 26.3
nurse 25.8
group 22.5
business 21.8
brass 20.4
businessman 20.3
men 19.7
wind instrument 19.4
adult 18.7
room 17
musical instrument 16.1
child 16
family 16
team 15.2
professional 15.1
happy 15
women 14.2
happiness 14.1
silhouette 14.1
job 13.3
indoors 12.3
lifestyle 12.3
meeting 12.2
couple 12.2
smile 10.7
together 10.5
office 10.4
portrait 10.3
teamwork 10.2
two 10.2
girls 10
suit 9.9
black 9.6
businesspeople 9.5
teacher 9.4
life 9.2
businesswoman 9.1
human 9
cheerful 8.9
mother 8.9
new 8.9
patient 8.8
colleagues 8.7
boy 8.7
smiling 8.7
love 8.7
play 8.6
corporate 8.6
trainer 8.5
modern 8.4
manager 8.4
executive 8.3
occupation 8.2
success 8
kid 8
working 7.9
sitting 7.7
summer 7.7
youth 7.7
fashion 7.5
joy 7.5
fun 7.5
holding 7.4
work 7.1
worker 7.1
face 7.1
to 7.1
interior 7.1

Google
created on 2021-12-15

Microsoft
created on 2021-12-15

person 99.9
clothing 98.5
text 96.6
group 87.7
standing 80.2
woman 76.2
posing 75.5
white 67.7
man 67.4
people 64.8
smile 55.8
old 45.5
clothes 30.7
dressed 28.1

Face analysis

Amazon

Microsoft

Google

AWS Rekognition

Age 12-22
Gender Female, 53.8%
Sad 65.6%
Calm 17.9%
Fear 8.1%
Confused 3.2%
Angry 2.7%
Surprised 1.9%
Disgusted 0.3%
Happy 0.2%

AWS Rekognition

Age 5-15
Gender Female, 91.7%
Sad 90.2%
Calm 6%
Angry 2.6%
Fear 0.6%
Confused 0.2%
Surprised 0.2%
Disgusted 0.1%
Happy 0.1%

AWS Rekognition

Age 29-45
Gender Female, 99.4%
Sad 45.4%
Calm 32.7%
Happy 14%
Disgusted 4.3%
Confused 1.5%
Angry 1.2%
Fear 0.6%
Surprised 0.3%

AWS Rekognition

Age 16-28
Gender Female, 84.5%
Calm 90.3%
Sad 3.4%
Happy 2.9%
Angry 2%
Confused 0.6%
Surprised 0.3%
Fear 0.3%
Disgusted 0.2%

AWS Rekognition

Age 5-15
Gender Male, 95.6%
Sad 81.1%
Calm 17.2%
Confused 1.3%
Angry 0.2%
Surprised 0.1%
Fear 0.1%
Happy 0%
Disgusted 0%

AWS Rekognition

Age 2-8
Gender Male, 86.9%
Sad 85.4%
Calm 13.5%
Angry 0.5%
Confused 0.2%
Happy 0.2%
Fear 0.2%
Surprised 0.1%
Disgusted 0%

AWS Rekognition

Age 3-11
Gender Male, 75.4%
Calm 92.5%
Sad 2.9%
Confused 2.3%
Angry 0.8%
Fear 0.5%
Surprised 0.5%
Happy 0.2%
Disgusted 0.2%

AWS Rekognition

Age 2-8
Gender Male, 80.9%
Calm 53.6%
Angry 22.9%
Sad 18.1%
Surprised 1.6%
Fear 1.6%
Confused 1.4%
Disgusted 0.5%
Happy 0.4%

AWS Rekognition

Age 21-33
Gender Male, 95.2%
Calm 99.2%
Sad 0.3%
Happy 0.3%
Surprised 0.1%
Angry 0%
Confused 0%
Disgusted 0%
Fear 0%

AWS Rekognition

Age 27-43
Gender Male, 98%
Calm 96.7%
Sad 1.8%
Angry 0.8%
Surprised 0.5%
Confused 0.1%
Fear 0.1%
Happy 0%
Disgusted 0%

AWS Rekognition

Age 8-18
Gender Male, 97.8%
Calm 84.7%
Surprised 4%
Angry 3.3%
Happy 2.9%
Confused 2.5%
Disgusted 1%
Sad 0.9%
Fear 0.7%

AWS Rekognition

Age 20-32
Gender Male, 96.1%
Calm 98.4%
Angry 0.8%
Sad 0.6%
Surprised 0.1%
Fear 0.1%
Confused 0%
Happy 0%
Disgusted 0%

AWS Rekognition

Age 11-21
Gender Male, 82.8%
Calm 97.6%
Angry 1%
Sad 0.8%
Surprised 0.2%
Happy 0.1%
Disgusted 0.1%
Confused 0.1%
Fear 0.1%

AWS Rekognition

Age 13-25
Gender Female, 51.5%
Calm 94.3%
Sad 2.6%
Angry 1.4%
Happy 0.7%
Confused 0.4%
Surprised 0.3%
Disgusted 0.2%
Fear 0.2%

AWS Rekognition

Age 22-34
Gender Female, 67.8%
Calm 94%
Sad 3.9%
Fear 0.8%
Surprised 0.6%
Happy 0.2%
Angry 0.2%
Confused 0.2%
Disgusted 0.1%

AWS Rekognition

Age 6-16
Gender Male, 94%
Calm 89.7%
Sad 5.3%
Confused 1.6%
Fear 0.9%
Happy 0.7%
Angry 0.7%
Surprised 0.7%
Disgusted 0.3%

AWS Rekognition

Age 22-34
Gender Male, 99.8%
Calm 42.6%
Angry 26.9%
Confused 17%
Disgusted 4.1%
Sad 3%
Surprised 2.9%
Fear 2.6%
Happy 0.8%

AWS Rekognition

Age 29-45
Gender Male, 57.8%
Sad 90.3%
Calm 8%
Confused 0.7%
Angry 0.5%
Fear 0.2%
Surprised 0.1%
Happy 0%
Disgusted 0%

AWS Rekognition

Age 19-31
Gender Female, 86.7%
Calm 92.5%
Surprised 4.6%
Sad 0.7%
Fear 0.7%
Angry 0.6%
Happy 0.5%
Confused 0.4%
Disgusted 0.2%

AWS Rekognition

Age 19-31
Gender Male, 74.6%
Calm 79.6%
Happy 17%
Sad 1.9%
Angry 0.7%
Confused 0.2%
Surprised 0.2%
Fear 0.2%
Disgusted 0.1%

AWS Rekognition

Age 25-39
Gender Male, 90.3%
Calm 87.8%
Sad 8.3%
Confused 1.4%
Happy 0.9%
Surprised 0.6%
Angry 0.5%
Disgusted 0.3%
Fear 0.1%

Microsoft Cognitive Services

Age 9
Gender Male

Microsoft Cognitive Services

Age 26
Gender Male

Microsoft Cognitive Services

Age 36
Gender Male

Microsoft Cognitive Services

Age 25
Gender Female

Microsoft Cognitive Services

Age 29
Gender Male

Microsoft Cognitive Services

Age 10
Gender Male

Microsoft Cognitive Services

Age 23
Gender Male

Microsoft Cognitive Services

Age 30
Gender Male

Microsoft Cognitive Services

Age 27
Gender Male

Microsoft Cognitive Services

Age 29
Gender Male

Microsoft Cognitive Services

Age 23
Gender Male

Microsoft Cognitive Services

Age 7
Gender Male

Microsoft Cognitive Services

Age 9
Gender Male

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very likely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Possible
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Possible
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Unlikely
Blurred Very unlikely

Feature analysis

Amazon

Person 99.8%
Dog 76%

Captions

Microsoft

Ernie Morrison et al. standing in front of a crowd posing for the camera 95.8%
Ernie Morrison et al. posing for a photo 95.7%
Ernie Morrison et al. posing for a photo in front of a crowd 95.6%