Human Generated Data

Title

Untitled (studio portrait of parents with three children dressed all in white)

Date

c. 1935, printed later

People

Artist: Durette Studio, American 20th century

Classification

Photographs

Credit Line

Harvard Art Museums/Fogg Museum, Transfer from the Carpenter Center for the Visual Arts, American Professional Photographers Collection, 4.2002.6080

Human Generated Data

Title

Untitled (studio portrait of parents with three children dressed all in white)

People

Artist: Durette Studio, American 20th century

Date

c. 1935, printed later

Classification

Photographs

Machine Generated Data

Tags

Amazon
created on 2019-11-16

Human 99.5
People 99.5
Family 99.5
Person 99.5
Person 99.5
Person 99.4
Person 97.4
Person 96.6
Person 95.8
Clothing 95.5
Coat 95.5
Suit 95.5
Apparel 95.5
Overcoat 95.5
Person 93.7
Person 93.6
Shoe 91
Footwear 91
Shoe 79.6
Helmet 65.2
Person 61.5
Helmet 51.2

Clarifai
created on 2019-11-16

people 100
group 99.7
group together 99.2
many 98.3
child 97.5
adult 97.2
family 97
woman 96.5
man 96.3
wear 95.4
several 95.1
chair 93.5
leader 93.1
offspring 92.5
room 92.5
five 90.9
administration 90.4
furniture 89.4
movie 88.1
music 87.3

Imagga
created on 2019-11-16

kin 100
man 34.9
couple 29.6
people 28.4
male 26.9
person 22.6
family 20.5
adult 19
men 18.9
happiness 18.8
love 17.4
women 16.6
mother 16.3
bride 15.3
happy 15
portrait 14.9
dress 14.5
home 14.4
room 13.4
fashion 12.8
two 12.7
wedding 12
husband 11.4
marriage 11.4
together 11.4
smiling 10.8
silhouette 10.8
handsome 10.7
smile 10.7
businessman 10.6
cheerful 10.6
married 10.5
attractive 10.5
wife 10.4
sitting 10.3
grandfather 10.3
groom 10.2
romance 9.8
romantic 9.8
lady 9.7
interior 9.7
father 9.5
nurse 9.5
bouquet 9.4
passion 9.4
joy 9.2
dark 9.2
child 9.1
world 9.1
black 9
fun 9
group 8.9
daughter 8.8
lifestyle 8.7
business 8.5
senior 8.4
holding 8.3
human 8.2
window 8.2
holiday 7.9
boy 7.8
face 7.8
chair 7.8
old 7.7
loving 7.6
elegance 7.6
worker 7.5
vintage 7.4
indoor 7.3
new 7.3
dirty 7.2
body 7.2
celebration 7.2
night 7.1
indoors 7

Google
created on 2019-11-16

Microsoft
created on 2019-11-16

clothing 98.9
wall 97.1
person 94.3
standing 88.9
posing 85.1
text 83.6
footwear 81.5
man 77.7
smile 72.3
black 70.6
group 69.2
human face 65.9
baby 65.7
boy 60.2
old 53.5

Face analysis

Amazon

Microsoft

Google

AWS Rekognition

Age 2-8
Gender Female, 54.4%
Disgusted 45.1%
Happy 45.2%
Fear 45.1%
Confused 47.7%
Surprised 46.3%
Calm 49.9%
Angry 45.1%
Sad 45.5%

AWS Rekognition

Age 17-29
Gender Male, 51.9%
Fear 45%
Calm 54.7%
Disgusted 45%
Angry 45%
Surprised 45%
Happy 45.1%
Sad 45.1%
Confused 45%

AWS Rekognition

Age 2-8
Gender Female, 54.3%
Calm 48%
Angry 45.5%
Fear 45.1%
Happy 45%
Sad 45.1%
Confused 45.5%
Disgusted 45.1%
Surprised 50.8%

AWS Rekognition

Age 12-22
Gender Female, 54.7%
Angry 46.4%
Fear 45.2%
Disgusted 45.1%
Sad 45.8%
Confused 45.5%
Calm 51.3%
Happy 45.4%
Surprised 45.3%

AWS Rekognition

Age 11-21
Gender Female, 50.8%
Fear 45.1%
Surprised 45.1%
Angry 45.2%
Happy 45%
Disgusted 45.1%
Confused 45.5%
Sad 46.3%
Calm 52.7%

AWS Rekognition

Age 3-9
Gender Female, 52.3%
Disgusted 45.1%
Sad 47%
Confused 45.3%
Happy 45.1%
Fear 45.2%
Surprised 45.2%
Calm 51.4%
Angry 45.8%

AWS Rekognition

Age 14-26
Gender Male, 52.4%
Fear 45%
Calm 54.6%
Sad 45.1%
Angry 45.1%
Disgusted 45.1%
Happy 45%
Surprised 45%
Confused 45%

AWS Rekognition

Age 18-30
Gender Male, 50.9%
Angry 45.1%
Fear 48.9%
Confused 45.7%
Surprised 49.4%
Calm 45.8%
Disgusted 45%
Sad 45.1%
Happy 45%

Microsoft Cognitive Services

Age 14
Gender Female

Microsoft Cognitive Services

Age 4
Gender Female

Microsoft Cognitive Services

Age 21
Gender Male

Microsoft Cognitive Services

Age 1
Gender Female

Microsoft Cognitive Services

Age 28
Gender Female

Microsoft Cognitive Services

Age 38
Gender Male

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Possible
Blurred Unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Likely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Unlikely
Blurred Very unlikely

Feature analysis

Amazon

Person 99.5%
Suit 95.5%
Shoe 91%
Helmet 65.2%

Captions

Microsoft

a group of people posing for a photo 95.5%
a group of people standing in front of a window posing for the camera 88.9%
a group of people posing for a photo in front of a window 88.8%