Human Generated Data

Title

Untitled (Red House, West Virginia)

Date

October 1935

People

Artist: Ben Shahn, American 1898 - 1969

Classification

Photographs

Credit Line

Harvard Art Museums/Fogg Museum, Gift of Bernarda Bryson Shahn, P1970.1194

Copyright

© President and Fellows of Harvard College

Human Generated Data

Title

Untitled (Red House, West Virginia)

People

Artist: Ben Shahn, American 1898 - 1969

Date

October 1935

Classification

Photographs

Machine Generated Data

Tags

Amazon
created on 2019-04-07

Person 99.5
Human 99.5
Person 99.2
Person 99.1
Person 99
Person 98.6
Person 98.4
Person 98.1
People 95.3
Person 92.1
Person 90.2
Person 89.5
Family 84.9
Clothing 82.6
Apparel 82.6
Person 69.8
Face 67.6
Female 62.8
Crowd 57.9

Clarifai
created on 2018-03-23

people 100
group 99.7
group together 97.6
portrait 97.6
child 97.3
adult 97
many 95.8
music 95.6
woman 95.4
actress 94.8
administration 93
wear 92.8
musician 91.9
offspring 91.3
facial expression 90.9
son 90.6
man 90.6
five 90.2
sibling 90.1
singer 90

Imagga
created on 2018-03-23

kin 96.2
people 37.9
adult 37
portrait 31.1
man 28.9
person 28.7
couple 27.9
attractive 27.3
male 26.3
love 25.2
fashion 24.1
sexy 22.5
black 21.8
model 21
happy 20.7
hair 20.6
face 20.6
together 19.3
lady 18.7
dark 18.4
passion 17.9
studio 17.5
pretty 16.8
youth 16.2
sensuality 15.4
happiness 14.9
style 14.8
sensual 14.5
make 14.5
dress 14.5
cute 14.3
smile 14.3
human 14.2
posing 14.2
group 13.7
body 13.6
boyfriend 13.5
brunette 13.1
lifestyle 13
two 12.7
girlfriend 12.5
romantic 12.5
erotic 12.3
relationship 12.2
fun 12
women 11.9
husband 11.8
elegance 11.8
suit 11.7
smiling 11.6
family 11.6
performer 11.5
loving 11.4
boy 11.3
skin 11.1
romance 10.7
wife 10.4
elegant 10.3
expression 10.2
emotion 10.1
indoor 10
girls 10
hand 9.9
handsome 9.8
night 9.8
hugging 9.7
vogue 9.7
room 9.5
party 9.5
friends 9.4
makeup 9.1
leisure 9.1
passionate 8.8
emotional 8.7
desire 8.7
married 8.6
jeans 8.6
men 8.6
child 8.3
holding 8.3
clothing 8.2
cheerful 8.1
looking 8
hug 7.7
underwear 7.7
modern 7.7
show 7.6
mother 7.5
one 7.5
vintage 7.4
holiday 7.2
art 7.1
lovely 7.1
bow tie 7

Google
created on 2018-03-23

Microsoft
created on 2018-03-23

person 100
outdoor 97.2
posing 91.6
group 89.1
standing 84.5
white 78.8
people 72.2
old 51.9
crowd 2

Face analysis

Amazon

Microsoft

Google

AWS Rekognition

Age 10-15
Gender Female, 99.7%
Surprised 0.6%
Happy 95.7%
Disgusted 0.8%
Calm 0.7%
Confused 0.4%
Angry 0.8%
Sad 0.9%

AWS Rekognition

Age 10-15
Gender Female, 82.1%
Confused 2.2%
Calm 6.2%
Angry 1.9%
Sad 86.3%
Surprised 1.5%
Disgusted 1.5%
Happy 0.6%

AWS Rekognition

Age 10-15
Gender Male, 55.7%
Disgusted 5.3%
Confused 6%
Angry 54.3%
Surprised 4.4%
Calm 14.2%
Happy 6.3%
Sad 9.4%

AWS Rekognition

Age 10-15
Gender Female, 99.1%
Disgusted 0%
Confused 0.1%
Angry 0.1%
Surprised 0.2%
Calm 0.1%
Happy 99.5%
Sad 0%

AWS Rekognition

Age 6-13
Gender Female, 93.9%
Happy 3.1%
Confused 2.3%
Angry 1.4%
Disgusted 1.2%
Sad 17.4%
Calm 72.9%
Surprised 1.7%

AWS Rekognition

Age 10-15
Gender Female, 98.5%
Happy 3.8%
Confused 2.6%
Angry 0.9%
Disgusted 0.5%
Sad 2.9%
Calm 88.3%
Surprised 0.9%

AWS Rekognition

Age 15-25
Gender Female, 99.1%
Calm 98.7%
Disgusted 0.1%
Sad 0.7%
Angry 0.1%
Confused 0.1%
Surprised 0.1%
Happy 0.1%

AWS Rekognition

Age 15-25
Gender Male, 53%
Sad 83.9%
Surprised 1.3%
Calm 6.2%
Happy 1.7%
Angry 3%
Disgusted 1.4%
Confused 2.4%

AWS Rekognition

Age 26-43
Gender Female, 91.4%
Calm 0.5%
Disgusted 0.2%
Angry 0.4%
Happy 0.4%
Confused 0.4%
Sad 98%
Surprised 0.2%

AWS Rekognition

Age 10-15
Gender Female, 96.1%
Calm 29.2%
Confused 1.6%
Surprised 1.9%
Disgusted 2.4%
Happy 2.8%
Angry 2.6%
Sad 59.4%

AWS Rekognition

Age 26-44
Gender Male, 60.2%
Happy 0.9%
Surprised 0.5%
Disgusted 0.4%
Confused 1.3%
Angry 1%
Calm 12.5%
Sad 83.4%

AWS Rekognition

Age 20-38
Gender Female, 56.1%
Sad 62.6%
Confused 3.6%
Happy 7.1%
Disgusted 5%
Angry 7%
Surprised 6.1%
Calm 8.5%

Microsoft Cognitive Services

Age 17
Gender Female

Microsoft Cognitive Services

Age 13
Gender Female

Microsoft Cognitive Services

Age 12
Gender Female

Microsoft Cognitive Services

Age 17
Gender Female

Microsoft Cognitive Services

Age 13
Gender Male

Microsoft Cognitive Services

Age 28
Gender Female

Microsoft Cognitive Services

Age 27
Gender Female

Microsoft Cognitive Services

Age 28
Gender Female

Microsoft Cognitive Services

Age 32
Gender Female

Microsoft Cognitive Services

Age 44
Gender Female

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Possible
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very likely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very likely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Feature analysis

Amazon

Person 99.5%

Captions

Microsoft

a group of people posing for a photo 99%
a group of people posing for the camera 98.9%
a black and white photo of a group of people posing for the camera 97.9%