Human Generated Data

Title

Untitled (family posed in front of foliage and candelabras)

Date

1941

People

Artist: Martin Schweig, American 20th century

Classification

Photographs

Credit Line

Harvard Art Museums/Fogg Museum, Transfer from the Carpenter Center for the Visual Arts, American Professional Photographers Collection, 4.2002.10234

Human Generated Data

Title

Untitled (family posed in front of foliage and candelabras)

People

Artist: Martin Schweig, American 20th century

Date

1941

Classification

Photographs

Machine Generated Data

Tags

Amazon
created on 2022-01-22

Family 99.5
People 99.5
Human 99.5
Person 99.4
Person 98.4
Person 98.1
Person 98
Person 95.9
Person 95.5
Person 94.8
Person 94
Accessories 93.2
Accessory 93.2
Tie 93.2
Person 90.6
Clothing 90
Suit 90
Overcoat 90
Coat 90
Apparel 90
Suit 85.5
Suit 83.6
Hat 73.6
Suit 65.2
Hat 63.6

Imagga
created on 2022-01-22

uniform 100
military uniform 100
clothing 83.6
consumer goods 57.1
covering 56.5
innocent 30.5
commodity 28.5
man 28.2
people 25.6
male 24.8
old 18.8
adult 16.2
person 16
men 14.6
group 14.5
happy 14.4
medical 14.1
doctor 14.1
portrait 13.6
hospital 13.3
senior 13.1
ancient 13
statue 12.3
home 12
health 11.8
aged 11.8
soldier 11.7
army 11.7
team 11.6
care 11.5
elderly 11.5
couple 11.3
mature 11.1
camera 11.1
smiling 10.8
vintage 10.7
history 10.7
family 10.7
military 10.6
medicine 10.6
private 10.4
business 10.3
nurse 10
face 9.9
religion 9.8
lifestyle 9.4
20s 9.1
indoors 8.8
antique 8.7
happiness 8.6
religious 8.4
kin 8.3
retro 8.2
looking 8
work 7.8
smile 7.8
professional 7.7
sculpture 7.6
city 7.5
monument 7.5
outdoors 7.5
historic 7.3
new 7.3
holiday 7.2
women 7.1
worker 7.1
love 7.1

Google
created on 2022-01-22

Outerwear 95.1
Photograph 94.1
Coat 89.9
Suit 83.3
People 77.6
Blazer 76.9
Classic 74.7
Snapshot 74.3
Vintage clothing 73.8
Event 73.1
Uniform 69.6
Family reunion 68.1
Collar 68
Team 66.4
History 65.6
Smile 64.2
Stock photography 64
Plant 63
Family pictures 62.2
Formal wear 62.1

Microsoft
created on 2022-01-22

person 100
posing 98.6
clothing 98.5
text 97.5
standing 96.4
group 93.2
old 92.2
man 91
military uniform 90.9
smile 90.1
outdoor 89.1
human face 81.2
people 77.8
black 66.3
woman 56.4
clothes 21.2
crowd 0.5

Face analysis

Amazon

Microsoft

Google

AWS Rekognition

Age 43-51
Gender Male, 99.8%
Calm 51.7%
Happy 38.3%
Surprised 3.1%
Confused 2.7%
Angry 2%
Disgusted 0.9%
Fear 0.7%
Sad 0.5%

AWS Rekognition

Age 54-64
Gender Female, 100%
Happy 81.7%
Calm 12.9%
Surprised 1.5%
Confused 1.3%
Sad 0.8%
Disgusted 0.8%
Angry 0.6%
Fear 0.5%

AWS Rekognition

Age 4-12
Gender Male, 99.7%
Happy 85.3%
Calm 9.9%
Confused 1.7%
Sad 1.1%
Angry 0.8%
Surprised 0.6%
Disgusted 0.3%
Fear 0.3%

AWS Rekognition

Age 35-43
Gender Male, 99.9%
Happy 99.6%
Surprised 0.2%
Confused 0.1%
Calm 0.1%
Fear 0%
Disgusted 0%
Angry 0%
Sad 0%

AWS Rekognition

Age 53-61
Gender Male, 100%
Happy 97.2%
Calm 0.9%
Surprised 0.6%
Angry 0.4%
Disgusted 0.3%
Fear 0.2%
Sad 0.2%
Confused 0.2%

AWS Rekognition

Age 4-10
Gender Male, 86.2%
Calm 84%
Sad 5.1%
Happy 3.7%
Confused 2.6%
Angry 2%
Surprised 1.4%
Disgusted 0.9%
Fear 0.3%

AWS Rekognition

Age 38-46
Gender Female, 100%
Happy 74%
Angry 6.9%
Calm 6.3%
Confused 3.1%
Disgusted 2.8%
Surprised 2.5%
Sad 2.3%
Fear 2.1%

AWS Rekognition

Age 34-42
Gender Female, 99.9%
Happy 84.9%
Angry 5.8%
Disgusted 3.2%
Surprised 2.7%
Fear 1.1%
Sad 1.1%
Calm 0.8%
Confused 0.4%

AWS Rekognition

Age 39-47
Gender Female, 100%
Calm 69.8%
Happy 19.1%
Surprised 4.6%
Confused 1.8%
Angry 1.5%
Sad 1.5%
Disgusted 1%
Fear 0.6%

Microsoft Cognitive Services

Age 47
Gender Male

Microsoft Cognitive Services

Age 63
Gender Male

Microsoft Cognitive Services

Age 40
Gender Female

Microsoft Cognitive Services

Age 40
Gender Male

Microsoft Cognitive Services

Age 67
Gender Female

Microsoft Cognitive Services

Age 7
Gender Male

Microsoft Cognitive Services

Age 29
Gender Female

Microsoft Cognitive Services

Age 33
Gender Female

Microsoft Cognitive Services

Age 6
Gender Male

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Likely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Possible
Headwear Very likely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Unlikely
Headwear Possible
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very likely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Likely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very likely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very likely
Headwear Likely
Blurred Very unlikely

Feature analysis

Amazon

Person 99.4%
Tie 93.2%
Suit 90%
Hat 73.6%

Captions

Microsoft

Zog I of Albania et al. posing for a photo 99.4%
a vintage photo of Zog I of Albania et al. posing for the camera 99%
a vintage photo of Zog I of Albania et al. posing for a picture 98.9%

Text analysis

Amazon

PROOF
to
SCHWEIG
MARTIN SCHWEIG
MARTIN
hair
take
away
W
Cut
darkun hair
brewa hair
W runkle
runkle
and
lair
helmen lair
darkun
helmen
brewa

Google

to
MARTIN
mabe
dehene
SCHWEIG
lbrevg
aauken
tokr
PROOF
awael
Aut
lbrevg hair aauken hair to mabe dehene lour Aut tokr PROOF MARTIN SCHWEIG awael
hair
lour