Human Generated Data

Title

Untitled (thirteen family members posed sitting on patio outdoors)

Date

1960

People

Artist: Martin Schweig, American 20th century

Classification

Photographs

Credit Line

Harvard Art Museums/Fogg Museum, Transfer from the Carpenter Center for the Visual Arts, American Professional Photographers Collection, 4.2002.10169

Human Generated Data

Title

Untitled (thirteen family members posed sitting on patio outdoors)

People

Artist: Martin Schweig, American 20th century

Date

1960

Classification

Photographs

Machine Generated Data

Tags

Amazon
created on 2022-01-22

Person 99.8
Human 99.8
Accessory 99.4
Tie 99.4
Accessories 99.4
Person 99.2
Person 99.2
Person 99
Person 98.5
Person 98.5
Person 98.4
Person 97.4
Person 96.5
Person 96.5
Tie 96.3
Person 94.8
Person 92.6
Person 91.2
Meal 87.7
Food 87.7
People 86.9
Advertisement 85.7
Poster 85.7
Clothing 79
Apparel 79
Leisure Activities 74.5
Tie 67
Overcoat 63.7
Coat 63.7
Suit 63.7
Shorts 63.6
Vacation 62.5
Picnic 62.5
Steamer 57.8
Crowd 57.3

Imagga
created on 2022-01-22

nurse 31.6
person 30.5
people 27.9
man 27.5
couple 27
male 24.8
adult 22.6
smiling 19.5
happy 19.4
men 18.9
teacher 18.5
happiness 18
bride 17.5
love 16.6
married 16.3
portrait 16.2
wedding 14.7
groom 14.5
cheerful 13.8
dress 13.5
family 12.4
together 12.3
holding 11.5
businessman 11.5
marriage 11.4
home 11.2
casual 11
human 10.5
bouquet 10.4
child 10.3
business 10.3
senior 10.3
sitting 10.3
two 10.2
smile 10
mother 9.8
gown 9.8
educator 9.7
professional 9.6
women 9.5
beach 9.3
clothing 9.2
attractive 9.1
outdoors 8.9
romance 8.9
new 8.9
romantic 8.9
group 8.9
to 8.8
sport 8.8
lifestyle 8.7
water 8.7
sky 8.3
fan 8.2
room 8.1
suit 8.1
team 8.1
patient 8
player 7.9
black 7.8
brass 7.8
old 7.7
youth 7.7
athlete 7.5
friendship 7.5
guy 7.5
blackboard 7.5
silhouette 7.4
teenager 7.3
student 7.3
sibling 7.2
kin 7.2
holiday 7.2
handsome 7.1
face 7.1
husband 7.1
summer 7.1

Google
created on 2022-01-22

Microsoft
created on 2022-01-22

text 99.9
person 98.8
posing 96.4
old 96
outdoor 88.5
group 87.8
clothing 84.3
black 74.5
people 66.2
man 66.1
smile 65.5
male 15.8

Face analysis

Amazon

Microsoft

Google

AWS Rekognition

Age 34-42
Gender Female, 99.8%
Happy 99.6%
Surprised 0.1%
Calm 0.1%
Fear 0%
Angry 0%
Confused 0%
Disgusted 0%
Sad 0%

AWS Rekognition

Age 37-45
Gender Female, 74.7%
Happy 99%
Surprised 0.4%
Calm 0.3%
Fear 0.2%
Confused 0.1%
Angry 0.1%
Sad 0%
Disgusted 0%

AWS Rekognition

Age 30-40
Gender Female, 100%
Happy 99.8%
Surprised 0.1%
Fear 0%
Calm 0%
Angry 0%
Sad 0%
Disgusted 0%
Confused 0%

AWS Rekognition

Age 4-10
Gender Male, 99.7%
Happy 98.7%
Surprised 0.4%
Angry 0.2%
Fear 0.2%
Confused 0.1%
Disgusted 0.1%
Calm 0.1%
Sad 0.1%

AWS Rekognition

Age 19-27
Gender Male, 100%
Happy 96.7%
Surprised 1.7%
Angry 0.5%
Calm 0.3%
Fear 0.3%
Confused 0.2%
Disgusted 0.1%
Sad 0.1%

AWS Rekognition

Age 20-28
Gender Female, 100%
Happy 97.3%
Calm 1.1%
Surprised 1%
Fear 0.2%
Confused 0.2%
Angry 0.1%
Sad 0%
Disgusted 0%

AWS Rekognition

Age 18-24
Gender Male, 99.9%
Calm 69.3%
Happy 12.6%
Confused 9.3%
Angry 2.4%
Surprised 2%
Fear 1.9%
Disgusted 1.3%
Sad 1.3%

AWS Rekognition

Age 42-50
Gender Male, 99.9%
Happy 99.2%
Surprised 0.2%
Fear 0.2%
Calm 0.2%
Angry 0.1%
Sad 0%
Disgusted 0%
Confused 0%

AWS Rekognition

Age 36-44
Gender Male, 100%
Happy 83.2%
Angry 5.6%
Calm 4.6%
Surprised 2.4%
Fear 1.3%
Disgusted 1.1%
Sad 1%
Confused 0.7%

AWS Rekognition

Age 13-21
Gender Male, 99.9%
Calm 92.1%
Happy 3.7%
Surprised 1.5%
Confused 1%
Fear 0.8%
Angry 0.5%
Sad 0.3%
Disgusted 0.1%

AWS Rekognition

Age 2-10
Gender Female, 99.9%
Happy 99.1%
Surprised 0.5%
Confused 0.2%
Fear 0.1%
Angry 0.1%
Calm 0%
Disgusted 0%
Sad 0%

AWS Rekognition

Age 2-8
Gender Female, 99.9%
Happy 53.6%
Fear 15.9%
Calm 15.1%
Surprised 5.9%
Angry 3.5%
Sad 2.4%
Confused 2.3%
Disgusted 1.2%

AWS Rekognition

Age 35-43
Gender Male, 99.9%
Happy 99.5%
Surprised 0.2%
Fear 0.1%
Confused 0.1%
Angry 0.1%
Sad 0%
Calm 0%
Disgusted 0%

Microsoft Cognitive Services

Age 30
Gender Female

Microsoft Cognitive Services

Age 44
Gender Male

Microsoft Cognitive Services

Age 22
Gender Male

Microsoft Cognitive Services

Age 39
Gender Male

Microsoft Cognitive Services

Age 10
Gender Female

Microsoft Cognitive Services

Age 29
Gender Female

Microsoft Cognitive Services

Age 32
Gender Male

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Possible
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very likely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very likely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very likely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very likely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Possible
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very likely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Likely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Likely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very likely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Likely
Headwear Very unlikely
Blurred Very unlikely

Feature analysis

Amazon

Person 99.8%
Tie 99.4%
Poster 85.7%
Suit 63.7%

Captions

Microsoft

a group of people posing for a photo 97.9%
a vintage photo of a group of people posing for the camera 97.1%
a vintage photo of a group of people posing for a picture 97%

Text analysis

Amazon

PROOF
SAINT
SAINT LOUIS
LOUIS
MARTIN
MARTIN SCHWEIG
SCHWEIG

Google

PROOF MARTIN SCHWEIG SAINT L OUIS
MARTIN
OUIS
PROOF
SAINT
SCHWEIG
L