Human Generated Data

Title

Untitled (family portrait)
Date

c. 1945
People

Artist: Martin Schweig, American 20th century
Classification

Photographs
Credit Line

Harvard Art Museums/Fogg Museum, Transfer from the Carpenter Center for the Visual Arts, American Professional Photographers Collection, 4.2002.1447

Human Generated Data

Title

Untitled (family portrait)

People

Artist: Martin Schweig, American 20th century

Date

c. 1945

Classification

Photographs

Machine Generated Data

Tags

Amazon
created on 2022-01-22

Accessories 99.6
Accessory 99.6
Tie 99.6
Person 99.1
Human 99.1
Person 99
Person 98.8
Person 98.7
Person 98.6
Person 98.4
Person 98
Person 96.9
Person 96.5
Tie 96.5
Person 95.4
People 93.8
Person 91.5
Person 88.9
Person 81.3
Person 80.9
Kindergarten 79.1
Person 78.5
Person 76.3
Indoors 75.5
Room 75.5
Tie 73.8
Person 73.5
School 69.5
Person 67.2
Classroom 67
Tie 61.2
Family 59.5

Imagga
created on 2022-01-22

nurse 92.2
people 36.2
person 28.4
man 27.5
male 25.5
room 22.3
adult 21.9
men 21.4
women 21.3
smiling 21
hospital 20.6
happy 20
couple 19.1
home 19.1
kin 18.8
family 18.7
group 16.9
indoors 16.7
happiness 16.4
professional 15.9
business 14.6
team 14.3
businessman 14.1
together 14
smile 13.5
standing 13
cheerful 13
child 12.3
two 11.8
mother 11.8
love 11.8
work 11.8
uniform 10.7
interior 10.6
job 10.6
new 10.5
fun 10.5
office 10.4
portrait 10.3
teamwork 10.2
classroom 10.2
businesswoman 10
dress 9.9
holding 9.9
life 9.8
pretty 9.8
education 9.5
teacher 9.5
meeting 9.4
casual 9.3
holiday 9.3
attractive 9.1
fashion 9
medical 8.8
house 8.3
inside 8.3
clothing 8.2
girls 8.2
student 8.2
suit 8.1
success 8
daughter 7.8
old 7.7
elegance 7.5
joy 7.5
friends 7.5
leisure 7.5
occupation 7.3
20s 7.3
school 7.3
lady 7.3
black 7.3
activity 7.2
working 7.1

Google
created on 2022-01-22

Family reunion 76.7
Vintage clothing 74
Event 70.8
Font 69.8
Toddler 66.8
Team 66.6
Child 64.9
Stock photography 61.8
Hat 61.6
Crew 60.7
History 58.2
Family taking photos together 57.9
Baby 57.4
Picture frame 57.3
Room 56.6
Classic 56.6
Family 55.6
Monochrome 53.4
Retro style 52.8
Musical instrument 51.1

Microsoft
created on 2022-01-22

posing 100
person 99.8
wall 99.3
group 98.2
clothing 97.6
smile 92.4
boy 89.8
standing 88.6
toddler 84.1
team 84.1
child 80.8
human face 78.4
old 68.4
family 66.8
footwear 66.6
baby 57.2
school 53.5
dressed 26.4
clothes 16.2

Face analysis

Amazon

Microsoft

Google

AWS Rekognition

Age 0-4
Gender Male, 99.9%
Calm 98.1%
Sad 0.7%
Confused 0.4%
Angry 0.2%
Disgusted 0.2%
Fear 0.1%
Happy 0.1%
Surprised 0.1%

AWS Rekognition

Age 14-22
Gender Female, 100%
Happy 87.1%
Surprised 4.8%
Angry 2.8%
Calm 2.7%
Confused 0.9%
Fear 0.7%
Sad 0.5%
Disgusted 0.4%

AWS Rekognition

Age 39-47
Gender Female, 99.9%
Calm 99.1%
Angry 0.3%
Confused 0.3%
Surprised 0.1%
Sad 0.1%
Happy 0.1%
Fear 0%
Disgusted 0%

AWS Rekognition

Age 29-39
Gender Female, 100%
Happy 97.6%
Surprised 0.8%
Fear 0.7%
Angry 0.3%
Confused 0.2%
Calm 0.2%
Sad 0.1%
Disgusted 0.1%

AWS Rekognition

Age 4-10
Gender Male, 99.8%
Calm 56.1%
Happy 26.6%
Confused 6.2%
Sad 5.3%
Surprised 3.1%
Angry 1.5%
Fear 0.8%
Disgusted 0.3%

AWS Rekognition

Age 2-10
Gender Male, 100%
Calm 98.6%
Confused 0.4%
Sad 0.4%
Surprised 0.3%
Happy 0.1%
Angry 0.1%
Fear 0%
Disgusted 0%

AWS Rekognition

Age 1-7
Gender Male, 100%
Happy 99%
Calm 0.2%
Confused 0.2%
Surprised 0.1%
Angry 0.1%
Disgusted 0.1%
Sad 0.1%
Fear 0%

AWS Rekognition

Age 18-24
Gender Male, 61.7%
Calm 74.4%
Happy 19.3%
Surprised 2.5%
Angry 1%
Sad 1%
Confused 1%
Fear 0.5%
Disgusted 0.5%

AWS Rekognition

Age 4-10
Gender Male, 98.3%
Calm 98.9%
Happy 0.9%
Confused 0.1%
Angry 0%
Sad 0%
Surprised 0%
Disgusted 0%
Fear 0%

AWS Rekognition

Age 1-7
Gender Male, 100%
Happy 79.6%
Calm 8%
Confused 7.9%
Angry 2.4%
Surprised 0.9%
Sad 0.8%
Disgusted 0.2%
Fear 0.1%

AWS Rekognition

Age 10-18
Gender Female, 91.3%
Happy 99.4%
Calm 0.2%
Confused 0.1%
Surprised 0.1%
Angry 0.1%
Sad 0%
Fear 0%
Disgusted 0%

AWS Rekognition

Age 18-24
Gender Female, 100%
Happy 98.3%
Surprised 0.7%
Angry 0.3%
Confused 0.2%
Fear 0.2%
Calm 0.1%
Disgusted 0.1%
Sad 0.1%

AWS Rekognition

Age 4-12
Gender Male, 100%
Happy 98.2%
Calm 0.6%
Sad 0.4%
Confused 0.3%
Surprised 0.3%
Angry 0.1%
Disgusted 0.1%
Fear 0.1%

AWS Rekognition

Age 6-16
Gender Female, 99.8%
Calm 76.4%
Happy 8.1%
Sad 5.6%
Surprised 3.3%
Confused 3.2%
Angry 2.1%
Disgusted 0.8%
Fear 0.5%

AWS Rekognition

Age 21-29
Gender Female, 100%
Happy 90.8%
Calm 3.5%
Angry 2%
Surprised 1.6%
Confused 0.9%
Sad 0.5%
Fear 0.5%
Disgusted 0.3%

AWS Rekognition

Age 23-33
Gender Female, 50.6%
Calm 86.8%
Happy 10%
Confused 1%
Surprised 0.8%
Sad 0.5%
Angry 0.4%
Fear 0.2%
Disgusted 0.2%

AWS Rekognition

Age 0-4
Gender Male, 90.5%
Sad 95.4%
Happy 1.5%
Angry 0.9%
Fear 0.7%
Surprised 0.6%
Calm 0.5%
Disgusted 0.3%
Confused 0.1%

AWS Rekognition

Age 39-47
Gender Male, 99.7%
Calm 83.9%
Sad 7.4%
Confused 3.9%
Disgusted 2.6%
Angry 0.8%
Surprised 0.5%
Fear 0.5%
Happy 0.4%

Microsoft Cognitive Services

Age 49
Gender Female

Microsoft Cognitive Services

Age 29
Gender Male

Microsoft Cognitive Services

Age 37
Gender Male

Microsoft Cognitive Services

Age 27
Gender Male

Microsoft Cognitive Services

Age 30
Gender Female

Microsoft Cognitive Services

Age 57
Gender Male

Microsoft Cognitive Services

Age 39
Gender Female

Microsoft Cognitive Services

Age 7
Gender Male

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very likely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very likely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very likely
Headwear Unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Likely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very likely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Possible
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very likely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very likely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very likely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Likely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Feature analysis

Amazon

Tie 99.6%
Person 99.1%

Captions

Microsoft

a group of people posing for a photo 99.1%
a group of people posing for the camera 99%
a group of people posing for a picture 98.9%