Human Generated Data

Title

Untitled (women talking around couch by man playing piano)

Date

1941

People

Artist: Martin Schweig, American 20th century

Classification

Photographs

Credit Line

Harvard Art Museums/Fogg Museum, Transfer from the Carpenter Center for the Visual Arts, American Professional Photographers Collection, 4.2002.10184

Human Generated Data

Title

Untitled (women talking around couch by man playing piano)

People

Artist: Martin Schweig, American 20th century

Date

1941

Classification

Photographs

Machine Generated Data

Tags

Amazon
created on 2022-01-22

Human 99.7
Person 99.7
Person 99.6
Person 99.5
Person 99.2
Person 99.1
Person 97.6
Person 97.2
Room 94.5
Indoors 94.5
Person 94.3
Crowd 94.2
Audience 92.4
Person 90.6
Person 90.6
People 80.3
School 70.4
Classroom 70.4
Jury 68.6
Court 67.2
Interior Design 59.1
Sitting 59
Lecture 57.6
Speech 57.6
Person 57.6
Press Conference 56.4

Imagga
created on 2022-01-22

people 36.2
person 33.2
room 31.9
man 29.6
group 26.6
professional 26.4
male 26.3
business 25.5
adult 25.5
teacher 24.7
classroom 23.1
meeting 20.7
businessman 19.4
home 19.1
team 18.8
office 18.6
happy 17.5
men 17.2
businesswoman 16.4
corporate 16.3
together 15.8
life 15.4
educator 14.7
smiling 14.5
adults 14.2
interior 14.1
indoors 14.1
couple 13.9
indoor 13.7
laptop 13.7
suit 13.6
house 13.4
work 13.3
attractive 13.3
working 13.3
education 13
computer 12.9
women 12.6
family 12.4
job 12.4
lifestyle 12.3
smile 12.1
executive 12
wind instrument 11.8
happiness 11.7
silhouette 11.6
chair 11.4
looking 11.2
sitting 11.2
love 11
portrait 11
conference 10.7
fun 10.5
businesspeople 10.4
student 10.2
teamwork 10.2
20s 10.1
girls 10
child 10
board 9.9
handsome 9.8
black 9.7
couch 9.7
table 9.5
desk 9.4
musical instrument 9.3
two 9.3
leisure 9.1
kin 9.1
pretty 9.1
success 8.8
sofa 8.6
youth 8.5
casual 8.5
friends 8.5
modern 8.4
communication 8.4
manager 8.4
presentation 8.4
boy 7.8
expression 7.7
buddy 7.7
mother 7.6
females 7.6
brass 7.5
harmonica 7.5
technology 7.4
cheerful 7.3
lady 7.3
dress 7.2
to 7.1

Google
created on 2022-01-22

Font 80.1
Chair 79.9
Piano 74.2
Event 72.6
Vintage clothing 71.2
Room 68.7
Suit 67.8
Sitting 65.4
Classic 64.7
Stock photography 64.2
Art 63.2
History 58.7
Formal wear 58
Photo caption 53.4
Retro style 53.2
Team 53
Recital 52.6
Hat 51.6
Advertising 51.5
Visual arts 51.5

Microsoft
created on 2022-01-22

person 99.7
wall 99.2
text 97.6
clothing 97.2
indoor 95.6
smile 88.3
woman 87
man 75.4
group 71.9
old 50.7
footwear 50

Face analysis

Amazon

Microsoft

Google

AWS Rekognition

Age 23-33
Gender Female, 100%
Happy 82%
Confused 3.7%
Surprised 3.6%
Angry 3.2%
Disgusted 2.8%
Fear 1.7%
Calm 1.6%
Sad 1.4%

AWS Rekognition

Age 27-37
Gender Female, 100%
Happy 85.1%
Calm 8.7%
Surprised 2.4%
Confused 1.6%
Angry 0.7%
Sad 0.7%
Fear 0.5%
Disgusted 0.3%

AWS Rekognition

Age 24-34
Gender Female, 100%
Calm 90.3%
Confused 2.1%
Angry 2%
Fear 1.6%
Happy 1.3%
Surprised 1.2%
Sad 1.2%
Disgusted 0.3%

AWS Rekognition

Age 25-35
Gender Female, 100%
Angry 78.9%
Confused 12.3%
Surprised 2.5%
Fear 2.1%
Disgusted 1.9%
Happy 1.1%
Calm 0.7%
Sad 0.5%

AWS Rekognition

Age 18-24
Gender Female, 100%
Calm 62.2%
Surprised 12.1%
Fear 11%
Angry 4.5%
Happy 3.4%
Sad 2.6%
Confused 2.4%
Disgusted 1.9%

AWS Rekognition

Age 9-17
Gender Female, 55.5%
Calm 96.9%
Sad 1.3%
Confused 0.5%
Angry 0.5%
Surprised 0.4%
Fear 0.2%
Happy 0.1%
Disgusted 0.1%

AWS Rekognition

Age 24-34
Gender Female, 62.4%
Calm 56.9%
Sad 26.7%
Disgusted 6.8%
Angry 3.4%
Confused 2.2%
Surprised 2.1%
Fear 1.1%
Happy 0.8%

AWS Rekognition

Age 27-37
Gender Male, 67.5%
Sad 89.8%
Calm 8.4%
Surprised 0.7%
Fear 0.3%
Disgusted 0.3%
Angry 0.2%
Happy 0.2%
Confused 0.1%

AWS Rekognition

Age 16-22
Gender Female, 99.7%
Calm 99.7%
Surprised 0.1%
Disgusted 0.1%
Sad 0%
Confused 0%
Fear 0%
Angry 0%
Happy 0%

AWS Rekognition

Age 16-22
Gender Female, 98.9%
Calm 83.9%
Sad 6.3%
Confused 2.6%
Surprised 1.9%
Fear 1.9%
Happy 1.6%
Disgusted 1%
Angry 0.9%

Microsoft Cognitive Services

Age 28
Gender Female

Microsoft Cognitive Services

Age 12
Gender Female

Microsoft Cognitive Services

Age 32
Gender Female

Microsoft Cognitive Services

Age 27
Gender Female

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very likely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Likely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Likely
Headwear Possible
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Feature analysis

Amazon

Person 99.7%

Captions

Microsoft

Sibylle Baier et al. in a room 97.3%
Sibylle Baier et al. standing in a room 96.5%
Sibylle Baier et al. posing for a photo 91.7%

Text analysis

Amazon

PROOF
SAINT
MARTIN
SAINT LOUIS
MARTIN SCHWEIG
LOUIS
SCHWEIG

Google

PROOF MARTIN SCHWEIG SAINT LOUIS
PROOF
MARTIN
SCHWEIG
SAINT
LOUIS