Human Generated Data

Title

Joost Schmidt and Mrs. Scheper at a Costume Party
Date

c. 1929
People

Artist: Unidentified Artist,

Classification

Photographs
Credit Line

Harvard Art Museums/Busch-Reisinger Museum, Gift of Howard Dearstyne, BR50.21

Human Generated Data

Title

Joost Schmidt and Mrs. Scheper at a Costume Party

People

Artist: Unidentified Artist,

Date

c. 1929

Classification

Photographs

Machine Generated Data

Tags

Amazon
created on 2022-01-09

Person 99.6
Human 99.6
Person 99.6
Face 96.9
Clothing 82
Apparel 82
People 74.9
Performer 65.1
Portrait 63.6
Photography 63.6
Photo 63.6

Imagga
created on 2022-01-09

man 43.7
male 42.6
person 38.5
portrait 33.7
businessman 33.6
business 32.8
office 31.8
people 30.7
director 30.3
suit 29.6
corporate 26.7
adult 26
executive 25.2
professional 24.3
happy 23.2
bow tie 21.5
phone 20.3
looking 20
tie 19.9
face 19.9
attractive 19.6
manager 19.6
smiling 19.5
necktie 19.4
work 18.9
handsome 18.7
smile 18.5
expression 17.9
glasses 16.7
look 16.7
successful 16.5
senior 15.9
job 15.9
mature 15.8
businesswoman 15.5
success 15.3
desk 15.1
confident 14.6
talk 14.4
shirt 14
old 13.9
call 13.8
sitting 13.8
lifestyle 13.7
clothing 13.3
businesspeople 13.3
indoor 12.8
communication 12.6
elderly 12.5
cell 12.4
mustache 12.3
mobile 12.3
telephone 12.1
computer 12
guy 11.9
modern 11.9
laptop 11.9
happiness 11.8
worker 11.6
couple 11.3
black 11.1
casual 11
alone 11
hand 10.6
working 10.6
talking 10.5
love 10.3
garment 10.2
friendly 10.1
holding 9.9
cheerful 9.8
human 9.8
masculine 9.7
cellphone 9.7
hair 9.5
eyes 9.5
buddy 9.4
room 9.1
entrepreneur 9.1
family 8.9
salesman 8.9
cellular 8.8
conversation 8.7
older 8.7
secretary 8.7
hands 8.7
sales 8.7
serious 8.6
thinking 8.6
jacket 8.5
one person 8.5
attire 8.5
finance 8.5
pretty 8.4
company 8.4
gray 8.1
home 8
to 8
indoors 7.9
table 7.8
head 7.6
fashion 7.5
meeting 7.5
one 7.5
occupation 7.3
stylish 7.2
women 7.1
wig 7

Google
created on 2022-01-09

Sleeve 86.9
Coat 85.4
Collar 79.8
Art 78.1
Beard 72.6
Vintage clothing 70.2
Eyewear 68.9
Room 67.5
Monochrome photography 66.2
Event 65.6
Font 64.6
Classic 63.2
History 61.3
Suit 60.9
Moustache 60.9
Monochrome 59.9
Visual arts 58.2
Formal wear 57.7
Photographic paper 57.3
Portrait 55.9

Microsoft
created on 2022-01-09

wall 99.9
human face 99.6
person 99.6
clothing 96.6
text 94.4
woman 94.1
indoor 92.5
smile 88.3
portrait 88.3
black 71.8
old 64.4
man 63
posing 50.2
picture frame 6

Face analysis

Amazon

Microsoft

Google

AWS Rekognition

Age 27-37
Gender Female, 99.8%
Calm 95.5%
Confused 2%
Surprised 0.7%
Sad 0.6%
Angry 0.6%
Disgusted 0.3%
Fear 0.2%
Happy 0.1%

AWS Rekognition

Age 38-46
Gender Male, 99.9%
Confused 45.1%
Calm 30.4%
Angry 11.2%
Fear 5.2%
Sad 3.8%
Surprised 1.8%
Disgusted 1.4%
Happy 1.1%

Microsoft Cognitive Services

Age 60
Gender Male

Microsoft Cognitive Services

Age 36
Gender Female

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Feature analysis

Amazon

Person 99.6%

Captions

Microsoft

a man and woman standing in front of a mirror posing for the camera 53.7%
a person standing in front of a mirror posing for the camera 53.6%
a man and woman in front of a mirror posing for the camera 53.5%