Human Generated Data

Title

John Sabatine and Molly at Pat Sabatine’s Eleventh Birthday Party
Date

April 1980, printed 1983
People

Artist: Larry Fink, American born 1941
Classification

Photographs
Credit Line

Harvard Art Museums/Fogg Museum, Gift of George and Alexandra Stephanopoulos, 2012.239
Copyright

© Larry Fink

Human Generated Data

Title

John Sabatine and Molly at Pat Sabatine’s Eleventh Birthday Party

People

Artist: Larry Fink, American born 1941

Date

April 1980, printed 1983

Classification

Photographs

Machine Generated Data

Tags

Amazon
created on 2019-04-08

Clothing 100
Apparel 100
Human 97.3
Person 97.3
Person 95.2
Person 94.7
Party Hat 91.9
Hat 91.9
Costume 59.6
Cake 55.7
Creme 55.7
Food 55.7
Icing 55.7
Cream 55.7
Dessert 55.7
Face 55.3
Cap 55.2
Elf 55.1

Clarifai
created on 2018-02-09

people 99.9
two 99.4
portrait 98.8
man 98.1
adult 97.9
three 97.2
group 96.7
child 96.1
monochrome 95.7
facial expression 94.2
administration 93.6
offspring 92.5
four 92.2
group together 92.1
family 92
leader 90.1
son 89.8
woman 89.7
street 89.6
wear 89.4

Imagga
created on 2018-02-09

grandfather 48.1
man 39.6
male 32.3
family 32
people 31.8
father 30.5
happy 29.4
portrait 29.1
dad 28.2
couple 27
parent 26.6
grandma 25.8
love 25.2
adult 24.6
smile 23.5
person 22.9
together 22.8
senior 22.5
mother 22
face 21.3
boy 20.9
smiling 19.5
elderly 18.2
old 18.1
lifestyle 17.3
child 17.2
home 16.7
husband 16.2
attractive 15.4
wife 15.2
mature 14.9
men 14.6
grandmother 13.7
fun 13.5
women 13.4
married 13.4
happiness 13.3
youth 12.8
two 12.7
brother 12.4
son 12.4
hair 11.9
casual 11.9
aged 11.8
handsome 11.6
holding 11.6
retirement 11.5
kid 11.5
black 11.4
cheerful 11.4
looking 11.2
sitting 11.2
suit 11.1
expression 11.1
pretty 10.5
hand 9.9
romantic 9.8
lady 9.7
retired 9.7
life 9.6
marriage 9.5
girls 9.1
romance 8.9
tie 8.5
kin 8.5
wine 8.3
indoors 7.9
affection 7.7
togetherness 7.6
fashion 7.5
human 7.5
glasses 7.4
business 7.3
sexy 7.2
gray 7.2
businessman 7.1

Google
created on 2018-02-09

photograph 95.2
man 94.9
person 94.8
facial expression 93.3
black and white 85.8
human behavior 83.8
male 82.7
snapshot 81.8
smile 76.9
child 76.6
standing 74.4
human 71.6
family 69.9
monochrome photography 68.7
grandparent 67.6
monochrome 65
father 64.5
senior citizen 60.1
fun 57
vintage clothing 51.2

Microsoft
created on 2018-02-09

person 99.9
man 95.1
old 47.4

Face analysis

Amazon

Microsoft

Google

AWS Rekognition

Age 1-5
Gender Male, 50.2%
Happy 72.2%
Confused 2.6%
Angry 5.9%
Surprised 9.8%
Disgusted 2.8%
Calm 5.8%
Sad 0.9%

AWS Rekognition

Age 48-68
Gender Male, 97.8%
Disgusted 21%
Sad 6.6%
Happy 21.6%
Surprised 3.5%
Calm 31.3%
Angry 6.8%
Confused 9.2%

Microsoft Cognitive Services

Age 4
Gender Female

Microsoft Cognitive Services

Age 72
Gender Male

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Likely
Headwear Very unlikely
Blurred Very unlikely

Feature analysis

Amazon

Person 97.3%

Captions

Microsoft

a black and white photo of a man 92.8%
an old photo of a man 92.7%
a man posing for a photo 92.6%