Human Generated Data

Title

Writer Susan Sontag with her son David, N.Y.C. 1965

Date

1965, printed later

People

Artist: Diane Arbus, American 1923 - 1971

Printer: Neil Selkirk,

Classification

Photographs

Credit Line

Harvard Art Museums/Fogg Museum, Gift of Jeffrey Fraenkel and Frish Brandt, 2017.308

Copyright

© The Estate of Diane Arbus, LLC

Human Generated Data

Title

Writer Susan Sontag with her son David, N.Y.C. 1965

People

Artist: Diane Arbus, American 1923 - 1971

Printer: Neil Selkirk,

Date

1965, printed later

Classification

Photographs

Machine Generated Data

Tags

Amazon
created on 2019-04-09

Apparel 100
Clothing 100
Overcoat 98
Coat 98
Human 94.9
Person 94.9
Person 93
Suit 66.2
Hat 65.7
Nature 62.1
Pants 60.6
Dating 59.3
Cap 56.9
Beanie 56.9

Clarifai
created on 2018-03-22

people 99.9
two 99.3
wear 98.8
portrait 98.6
adult 98
coat 96.5
man 96.2
woman 96.2
outerwear 95
street 92.3
outfit 89.5
child 88.3
affection 88.3
overcoat 88.2
facial expression 87.7
veil 85.8
movie 82.6
lid 82.3
jacket 81.2
actress 80.9

Imagga
created on 2018-03-22

statue 28.5
person 26.8
portrait 25.2
man 24.9
coat 22.4
adult 21.3
people 20.6
smile 19.2
attractive 18.2
jacket 18.2
trench coat 17
sculpture 16.4
face 16.3
happy 16.3
suit 15.9
raincoat 15.6
fashion 15.1
old 14.6
city 14.1
lady 13.8
one 13.4
clothing 13.4
pretty 13.3
outdoor 13
male 12.8
business 12.8
outdoors 12.7
religion 12.5
smiling 12.3
lifestyle 12.3
monument 12.1
architecture 11.8
model 11.7
antique 11.3
ancient 11.2
cold 11.2
culture 11.1
expression 11.1
hat 11.1
handsome 10.7
garment 10.7
crazy 10.5
standing 10.4
style 10.4
outside 10.3
stone 10.1
black 10.1
travel 9.9
history 9.8
building 9.7
success 9.7
urban 9.6
cute 9.3
street 9.2
tourism 9.1
scarf 9
world 8.6
happiness 8.6
art 8.6
religious 8.4
covering 8.2
tourist 8.1
landmark 8.1
looking 8
businessman 7.9
autumn 7.9
brunette 7.8
catholic 7.8
god 7.7
mother 7.6
human 7.5
successful 7.3
office 7.2
boy 7.1
hair 7.1
women 7.1
day 7.1

Google
created on 2018-03-22

Microsoft
created on 2018-03-22

person 99.8
outdoor 99.2
standing 82

Face analysis

Amazon

Microsoft

Google

AWS Rekognition

Age 20-38
Gender Male, 76.3%
Angry 0.4%
Sad 1.9%
Calm 95.8%
Confused 0.8%
Surprised 0.7%
Happy 0.1%
Disgusted 0.4%

AWS Rekognition

Age 26-43
Gender Female, 90.8%
Happy 0.8%
Disgusted 0.5%
Sad 5.1%
Confused 2.2%
Surprised 0.6%
Angry 0.5%
Calm 90.3%

Microsoft Cognitive Services

Age 19
Gender Female

Microsoft Cognitive Services

Age 24
Gender Female

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Possible
Blurred Very unlikely

Feature analysis

Amazon

Person 94.9%

Captions

Microsoft

Susan Sontag et al. posing for a photo 83.5%
Susan Sontag standing posing for the camera 83.4%
Susan Sontag and woman posing for a photo 76.5%