Human Generated Data

Title

Untitled (large family of eighteen people posing in small room with bookshelf)
Date

1938
People

Artist: Martin Schweig, American 20th century
Classification

Photographs
Credit Line

Harvard Art Museums/Fogg Museum, Transfer from the Carpenter Center for the Visual Arts, American Professional Photographers Collection, 4.2002.10155

Human Generated Data

Title

Untitled (large family of eighteen people posing in small room with bookshelf)

People

Artist: Martin Schweig, American 20th century

Date

1938

Classification

Photographs

Machine Generated Data

Tags

Amazon
created on 2022-01-22

Human 99.4
Person 99.4
Accessories 99.3
Tie 99.3
Accessory 99.3
People 99.3
Family 99.3
Person 99
Person 98.3
Person 98.2
Person 98
Person 97.5
Person 97.5
Person 97.4
Person 97
Tie 96.7
Person 96.3
Tie 96
Tie 95.9
Person 95.4
Person 94.7
Person 94.1
Person 93.3
Suit 92.8
Overcoat 92.8
Apparel 92.8
Clothing 92.8
Coat 92.8
Person 92.4
Person 92.2
Person 87.6
Suit 80.7
Suit 71.9
Tie 71.6
Tie 68.4
Tie 67.2
Person 66.2

Imagga
created on 2022-01-22

kin 100
family 28.4
people 27.3
happy 23.8
child 23.6
happiness 23.5
male 23.4
man 22.1
person 21.1
portrait 20.7
adult 19.8
fun 17.9
mother 17
couple 16.5
group 16.1
love 15.8
men 13.7
daughter 13.5
women 13.4
joy 13.3
together 13.1
smiling 13
smile 12.8
youth 12.8
girls 12.7
father 12.4
kid 12.4
outdoor 12.2
boy 12.2
black 12
home 12
childhood 11.6
parent 11.1
casual 11
two 11
leisure 10.8
room 10.3
summer 10.3
face 9.9
dress 9.9
attractive 9.8
cheerful 9.7
human 9.7
married 9.6
park 9
blond 9
active 9
outdoors 8.9
lifestyle 8.7
bride 8.6
enjoying 8.5
friendship 8.4
fashion 8.3
silhouette 8.3
holding 8.2
sport 8.2
playing 8.2
children 8.2
sexy 8
cute 7.9
sepia 7.8
son 7.7
husband 7.6
ball 7.5
emotion 7.4
wedding 7.3
life 7.3
looking 7.2
team 7.2
grass 7.1
to 7.1
interior 7.1
day 7.1

Google
created on 2022-01-22

Coat 85.4
Suit 80.2
Family reunion 78.4
Vintage clothing 75.5
Classic 75.4
Snapshot 74.3
Event 73.7
Crew 73.4
Team 71.4
Formal wear 67
History 65.8
Sitting 59.1
Family 55.1
Oval 54.4
Retro style 54.2

Microsoft
created on 2022-01-22

person 100
posing 99.9
clothing 98.3
wall 96.5
group 95.4
smile 93.6
man 92.2
standing 91.4
old 85.2
woman 81
team 60.1
people 59.4
dressed 31.4
male 15.9
clothes 15.1
crowd 0.9

Face analysis

Amazon

Microsoft

Google

AWS Rekognition

Age 16-24
Gender Male, 100%
Calm 52%
Happy 33.7%
Surprised 6.7%
Confused 3.6%
Angry 2.1%
Sad 0.9%
Fear 0.8%
Disgusted 0.2%

AWS Rekognition

Age 9-17
Gender Female, 75.9%
Calm 98.6%
Happy 0.7%
Confused 0.3%
Sad 0.1%
Surprised 0.1%
Angry 0.1%
Fear 0%
Disgusted 0%

AWS Rekognition

Age 16-24
Gender Male, 100%
Calm 71.3%
Surprised 13.9%
Happy 7.4%
Fear 2.6%
Confused 2.4%
Angry 1.3%
Sad 0.7%
Disgusted 0.4%

AWS Rekognition

Age 43-51
Gender Male, 100%
Calm 99.8%
Confused 0.1%
Surprised 0.1%
Angry 0%
Happy 0%
Sad 0%
Disgusted 0%
Fear 0%

AWS Rekognition

Age 24-34
Gender Female, 96.9%
Calm 98.2%
Sad 0.4%
Surprised 0.4%
Happy 0.3%
Angry 0.3%
Confused 0.2%
Fear 0.1%
Disgusted 0.1%

AWS Rekognition

Age 42-50
Gender Male, 97%
Calm 83%
Confused 4.6%
Happy 3.1%
Surprised 2.3%
Sad 2.2%
Angry 2.2%
Fear 1.5%
Disgusted 1%

AWS Rekognition

Age 42-50
Gender Female, 60.2%
Calm 80.6%
Confused 6.8%
Happy 5.9%
Angry 2.2%
Surprised 2%
Sad 0.9%
Fear 0.8%
Disgusted 0.7%

AWS Rekognition

Age 20-28
Gender Male, 97.4%
Calm 98.8%
Surprised 0.3%
Happy 0.2%
Confused 0.2%
Angry 0.2%
Sad 0.2%
Fear 0.1%
Disgusted 0%

AWS Rekognition

Age 29-39
Gender Male, 100%
Calm 99.5%
Happy 0.2%
Confused 0.1%
Angry 0.1%
Surprised 0%
Sad 0%
Disgusted 0%
Fear 0%

AWS Rekognition

Age 47-53
Gender Female, 99.6%
Calm 99.2%
Surprised 0.3%
Happy 0.1%
Angry 0.1%
Fear 0.1%
Sad 0.1%
Confused 0.1%
Disgusted 0%

AWS Rekognition

Age 35-43
Gender Female, 98.6%
Happy 55.8%
Calm 28.5%
Confused 8.7%
Angry 2.9%
Surprised 2.2%
Disgusted 0.8%
Sad 0.7%
Fear 0.5%

AWS Rekognition

Age 54-62
Gender Female, 99.6%
Happy 95.1%
Calm 3.1%
Surprised 0.6%
Confused 0.4%
Angry 0.3%
Sad 0.2%
Fear 0.2%
Disgusted 0.1%

AWS Rekognition

Age 73-83
Gender Female, 58.8%
Calm 80.6%
Angry 8.8%
Confused 5.2%
Sad 2.1%
Disgusted 0.9%
Fear 0.9%
Surprised 0.8%
Happy 0.6%

AWS Rekognition

Age 4-10
Gender Female, 97.9%
Calm 94.2%
Surprised 3.2%
Fear 0.9%
Confused 0.7%
Happy 0.4%
Angry 0.3%
Sad 0.3%
Disgusted 0.1%

AWS Rekognition

Age 38-46
Gender Male, 100%
Calm 98.2%
Happy 0.5%
Confused 0.4%
Sad 0.3%
Angry 0.2%
Surprised 0.2%
Fear 0.1%
Disgusted 0.1%

AWS Rekognition

Age 45-53
Gender Female, 99.8%
Calm 87.4%
Happy 5.2%
Surprised 2%
Angry 1.8%
Confused 1.7%
Sad 0.8%
Fear 0.6%
Disgusted 0.5%

AWS Rekognition

Age 4-12
Gender Male, 59.1%
Calm 97%
Confused 1.9%
Sad 0.4%
Angry 0.2%
Surprised 0.2%
Happy 0.1%
Disgusted 0.1%
Fear 0%

AWS Rekognition

Age 52-60
Gender Male, 100%
Calm 99.5%
Happy 0.1%
Angry 0.1%
Confused 0.1%
Sad 0.1%
Disgusted 0.1%
Surprised 0%
Fear 0%

Microsoft Cognitive Services

Age 27
Gender Male

Microsoft Cognitive Services

Age 44
Gender Male

Microsoft Cognitive Services

Age 53
Gender Female

Microsoft Cognitive Services

Age 31
Gender Female

Microsoft Cognitive Services

Age 30
Gender Male

Microsoft Cognitive Services

Age 30
Gender Female

Microsoft Cognitive Services

Age 44
Gender Male

Microsoft Cognitive Services

Age 7
Gender Male

Microsoft Cognitive Services

Age 58
Gender Male

Microsoft Cognitive Services

Age 33
Gender Female

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Likely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Unlikely
Joy Unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Possible
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very likely
Headwear Unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Feature analysis

Amazon

Person 99.4%
Tie 99.3%
Suit 92.8%

Captions

Microsoft

a group of people posing for a photo 99.2%
a group of people posing for a picture 99.1%
a group of people posing for the camera 99%