Human Generated Data

Title

Untitled (Fourteenth Street, New York City)
Date

1932-1935
People

Artist: Ben Shahn, American 1898 - 1969
Classification

Photographs
Credit Line

Harvard Art Museums/Fogg Museum, Gift of Bernarda Bryson Shahn, P1970.2770
Copyright

© President and Fellows of Harvard College

Human Generated Data

Title

Untitled (Fourteenth Street, New York City)

People

Artist: Ben Shahn, American 1898 - 1969

Date

1932-1935

Classification

Photographs

Machine Generated Data

Tags

Amazon
created on 2019-04-07

Apparel 100
Clothing 100
Human 99.8
Person 99.8
Person 99.6
Person 99.4
Hat 98.6
Cap 86.6
Beanie 75.2
Finger 72.4
Hat 69.4
Sun Hat 67.4
Hat 65.7
Helmet 62.7
Face 56.4

Clarifai
created on 2018-03-23

people 99.9
adult 98.1
child 97.2
portrait 97
wear 96.3
group 95.9
lid 95.5
veil 94.7
woman 94.4
man 93.4
two 91.3
administration 90.3
boy 90.1
street 88.6
music 87.6
facial expression 86.2
son 86
war 84.8
one 84.1
three 82

Imagga
created on 2018-03-23

people 34.6
man 33.6
portrait 33
male 32.3
hat 30.7
couple 29.6
happy 26.9
adult 25.9
person 25.2
together 24.5
love 23.7
brother 22.9
face 22.7
smiling 21.7
smile 20.7
attractive 20.3
women 19.8
child 19.6
pretty 19.6
cute 19.4
family 18.7
two 18.6
looking 18.4
dad 17.8
black 17.5
father 16.9
happiness 16.5
fashion 15.8
parent 15.5
handsome 15.2
cowboy hat 15
fun 15
mother 14.8
sitting 14.6
men 14.6
girls 14.6
sexy 14.5
lifestyle 13.7
expression 12.8
sibling 12.7
style 12.6
dark 12.5
romance 12.5
model 12.4
headdress 12.3
guy 12
casual 11.9
clothing 11.6
girlfriend 11.5
world 11.3
relationship 11.2
20s 11
cheerful 10.6
brunette 10.5
boy 10.4
emotion 10.1
cipher 10
romantic 9.8
lady 9.7
indoors 9.7
married 9.6
enjoying 9.5
youth 9.4
leisure 9.1
hand 9.1
husband 9.1
son 8.9
kid 8.9
hair 8.7
affection 8.7
boyfriend 8.7
loving 8.6
wife 8.5
senior 8.4
old 8.4
holding 8.3
children 8.2
posing 8
home 8
hugging 7.8
friends 7.5
human 7.5
city 7.5
relaxing 7.3
sensual 7.3
dress 7.2
body 7.2
lovely 7.1

Google
created on 2018-03-23

photograph 95.6
black 95.2
black and white 90.5
photography 83.3
snapshot 81.8
monochrome photography 81.6
headgear 70.5
monochrome 67.7
fun 63.9
girl 60
gang 50.5

Microsoft
created on 2018-03-23

person 100

Face analysis

Amazon

Microsoft

Google

AWS Rekognition

Age 45-66
Gender Female, 95.9%
Confused 2.9%
Surprised 3.1%
Happy 74.1%
Angry 3%
Disgusted 6.7%
Sad 7.2%
Calm 3.1%

AWS Rekognition

Age 10-15
Gender Male, 85.1%
Disgusted 0.6%
Calm 96.1%
Confused 0.5%
Surprised 1.1%
Sad 0.8%
Happy 0.4%
Angry 0.4%

AWS Rekognition

Age 23-38
Gender Male, 88.5%
Angry 8.3%
Sad 47.3%
Happy 1.9%
Surprised 3.5%
Confused 6.3%
Disgusted 2.8%
Calm 29.9%

Microsoft Cognitive Services

Age 14
Gender Male

Microsoft Cognitive Services

Age 40
Gender Female

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Unlikely
Headwear Possible
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Unlikely
Joy Very unlikely
Headwear Very likely
Blurred Very unlikely

Feature analysis

Amazon

Person 99.8%
Hat 98.6%
Helmet 62.7%

Captions

Microsoft

a person wearing a hat 88.6%
a person wearing a black hat 87.1%
a group of people posing for the camera 85.9%