Human Generated Data

Title

Untitled (family portrait with seventeen children)
Date

c. 1955
People

Artist: Harry Annas, American 1897 - 1980
Classification

Photographs
Credit Line

Harvard Art Museums/Fogg Museum, Transfer from the Carpenter Center for the Visual Arts, American Professional Photographers Collection, 4.2002.10379

Human Generated Data

Title

Untitled (family portrait with seventeen children)

People

Artist: Harry Annas, American 1897 - 1980

Date

c. 1955

Classification

Photographs

Machine Generated Data

Tags

Amazon
created on 2022-01-23

Human 99.5
Person 99.5
Person 99.1
Person 98.9
Person 98.8
Person 97.9
Person 97.2
Person 96.9
Person 96.7
Person 96.5
Kindergarten 95.5
Person 95.5
Person 95.1
School 94.9
Room 94.7
Indoors 94.7
Classroom 94.7
Person 94.2
Person 94
Person 92
Person 88.2
Person 86.8
Person 85.2
Accessories 81.5
Accessory 81.5
Tie 81.5
People 77.8
Tie 75.6
Person 63.2
Clothing 56.5
Apparel 56.5

Imagga
created on 2022-01-23

nurse 100
people 35.7
happy 30
man 29.6
person 28.1
male 26.3
smiling 25.3
women 25.3
hospital 24.7
adult 24.3
group 24.2
men 24
family 23.1
kin 22.8
smile 22.8
home 22.3
team 21.5
room 19.2
happiness 18.8
together 18.4
mother 18.4
child 18
portrait 17.5
couple 16.5
business 16.4
professional 16
businessman 15.9
love 15
cheerful 14.6
businesswoman 14.5
standing 13.9
uniform 13.4
indoors 13.2
casual 12.7
boy 12.2
father 12.1
teamwork 12
sitting 12
fun 12
daughter 11.6
diversity 11.5
kid 11.5
interior 11.5
meeting 11.3
work 11
indoor 10.9
girls 10.9
joy 10.8
job 10.6
fashion 10.5
office 10.4
friendship 10.3
lifestyle 10.1
20s 10.1
house 10
school 9.9
attractive 9.8
medical 9.7
education 9.5
life 9.4
friends 9.4
two 9.3
classroom 9.3
children 9.1
student 9
childhood 8.9
new 8.9
success 8.8
parent 8.7
black 8.5
doctor 8.5
successful 8.2
brunette 7.8
corporate 7.7
clothing 7.7
husband 7.6
businesspeople 7.6
wife 7.6
elegance 7.5
human 7.5
holding 7.4
executive 7.4
occupation 7.3
looking 7.2
cute 7.2
holiday 7.2
kitchen 7.1
worker 7.1
working 7.1

Google
created on 2022-01-23

Family reunion 77.1
Event 73.7
Vintage clothing 73.7
Family taking photos together 70.2
Team 68.5
Suit 66.6
Child 65.2
History 64.1
Room 62.8
Font 62.5
Toddler 60.3
Family pictures 58.7
Happy 57.8
Hat 56.2
Crew 55.8
Retro style 55.1
Monochrome 54.9
Family 53
Sharing 51.4

Microsoft
created on 2022-01-23

posing 100
person 99.9
group 98.8
clothing 98.5
wall 97.6
smile 95.2
boy 94.7
human face 88.5
child 86.3
toddler 77.7
team 77.3
text 77.2
footwear 68.5
dressed 25
family 17.5

Face analysis

Amazon

Microsoft

Google

AWS Rekognition

Age 4-10
Gender Male, 99.9%
Calm 46.8%
Happy 41.4%
Surprised 5%
Confused 2.4%
Angry 1.9%
Sad 1.7%
Fear 0.4%
Disgusted 0.4%

AWS Rekognition

Age 0-4
Gender Female, 77.5%
Calm 83.7%
Surprised 4.5%
Sad 4.3%
Confused 2.6%
Happy 2.1%
Angry 1.5%
Disgusted 0.9%
Fear 0.5%

AWS Rekognition

Age 6-14
Gender Male, 99.2%
Happy 100%
Surprised 0%
Angry 0%
Calm 0%
Sad 0%
Confused 0%
Disgusted 0%
Fear 0%

AWS Rekognition

Age 0-4
Gender Male, 99.7%
Confused 57.9%
Calm 20.6%
Fear 12.7%
Sad 2.9%
Angry 2.3%
Happy 1.5%
Surprised 1.2%
Disgusted 0.8%

AWS Rekognition

Age 13-21
Gender Female, 93.1%
Happy 52.5%
Calm 25.7%
Surprised 9.4%
Sad 5.6%
Confused 4.1%
Angry 1.5%
Fear 0.6%
Disgusted 0.5%

AWS Rekognition

Age 4-10
Gender Male, 100%
Calm 77.6%
Sad 19.8%
Confused 1.7%
Angry 0.3%
Surprised 0.2%
Disgusted 0.1%
Fear 0.1%
Happy 0.1%

AWS Rekognition

Age 34-42
Gender Female, 98.6%
Calm 98.6%
Angry 0.4%
Sad 0.3%
Happy 0.3%
Confused 0.2%
Surprised 0.2%
Disgusted 0%
Fear 0%

AWS Rekognition

Age 1-7
Gender Male, 100%
Calm 52%
Happy 15.2%
Sad 13.8%
Confused 9%
Angry 2.9%
Surprised 2.9%
Fear 2.4%
Disgusted 1.8%

AWS Rekognition

Age 2-8
Gender Male, 100%
Calm 99.6%
Confused 0.2%
Sad 0.1%
Surprised 0.1%
Angry 0%
Happy 0%
Fear 0%
Disgusted 0%

AWS Rekognition

Age 20-28
Gender Female, 99.8%
Calm 35.9%
Happy 35.4%
Angry 8.1%
Confused 7.2%
Surprised 5.1%
Disgusted 4.7%
Sad 2.5%
Fear 1%

AWS Rekognition

Age 38-46
Gender Male, 96.3%
Happy 32.3%
Disgusted 18.6%
Sad 15.9%
Calm 11.3%
Confused 8.3%
Angry 6.1%
Fear 4.9%
Surprised 2.7%

AWS Rekognition

Age 18-24
Gender Female, 99.7%
Happy 49.1%
Calm 35.5%
Angry 5.1%
Surprised 3.6%
Confused 3.2%
Sad 1.4%
Disgusted 1%
Fear 1%

AWS Rekognition

Age 37-45
Gender Female, 100%
Happy 99%
Surprised 0.3%
Fear 0.2%
Confused 0.1%
Angry 0.1%
Calm 0.1%
Disgusted 0%
Sad 0%

AWS Rekognition

Age 18-26
Gender Male, 96.2%
Happy 97.5%
Calm 1.1%
Angry 0.5%
Surprised 0.4%
Fear 0.2%
Confused 0.2%
Sad 0.1%
Disgusted 0.1%

AWS Rekognition

Age 19-27
Gender Female, 100%
Happy 98.3%
Surprised 0.4%
Calm 0.4%
Confused 0.4%
Angry 0.3%
Fear 0.1%
Disgusted 0.1%
Sad 0.1%

AWS Rekognition

Age 22-30
Gender Male, 92.8%
Calm 98.9%
Happy 0.4%
Confused 0.2%
Sad 0.2%
Angry 0.1%
Surprised 0.1%
Disgusted 0%
Fear 0%

AWS Rekognition

Age 10-18
Gender Male, 85.1%
Calm 95.5%
Happy 2.3%
Confused 0.7%
Sad 0.6%
Surprised 0.5%
Angry 0.2%
Fear 0.1%
Disgusted 0%

AWS Rekognition

Age 1-7
Gender Male, 100%
Happy 92.6%
Confused 2.5%
Calm 2.4%
Angry 1.2%
Surprised 0.6%
Disgusted 0.3%
Sad 0.3%
Fear 0.1%

AWS Rekognition

Age 18-26
Gender Female, 98.3%
Sad 92%
Fear 2.1%
Angry 1.6%
Surprised 1.4%
Calm 1.2%
Happy 1.1%
Disgusted 0.5%
Confused 0.2%

Microsoft Cognitive Services

Age 28
Gender Female

Microsoft Cognitive Services

Age 26
Gender Male

Microsoft Cognitive Services

Age 47
Gender Female

Microsoft Cognitive Services

Age 27
Gender Female

Microsoft Cognitive Services

Age 27
Gender Female

Microsoft Cognitive Services

Age 6
Gender Male

Microsoft Cognitive Services

Age 38
Gender Female

Microsoft Cognitive Services

Age 38
Gender Male

Microsoft Cognitive Services

Age 28
Gender Male

Microsoft Cognitive Services

Age 12
Gender Male

Microsoft Cognitive Services

Age 12
Gender Female

Microsoft Cognitive Services

Age 7
Gender Male

Microsoft Cognitive Services

Age 5
Gender Male

Microsoft Cognitive Services

Age 48
Gender Male

Microsoft Cognitive Services

Age 6
Gender Male

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Possible
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very likely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Likely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very likely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Possible
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Possible
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very likely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Likely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very likely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Feature analysis

Amazon

Person 99.5%
Tie 81.5%

Captions

Microsoft

Timothy Wiltsey et al. posing for a photo 99%
Timothy Wiltsey et al. posing for the camera 98.9%
Timothy Wiltsey et al. posing for a picture 98.8%