Human Generated Data

Title

Untitled (family portrait, outside house)
Date

c. 1950
People

Artist: Lucian and Mary Brown, American

Classification

Photographs
Credit Line

Harvard Art Museums/Fogg Museum, Transfer from the Carpenter Center for the Visual Arts, American Professional Photographers Collection, 4.2002.16450

Human Generated Data

Title

Untitled (family portrait, outside house)

People

Artist: Lucian and Mary Brown, American

Date

c. 1950

Classification

Photographs

Machine Generated Data

Tags

Amazon
created on 2022-02-11

Person 99.4
Human 99.4
Person 99.3
People 98.5
Person 97.9
Tie 97.2
Accessory 97.2
Accessories 97.2
Family 96.9
Person 96.7
Person 96.3
Clinic 90.8
Chair 83.2
Furniture 83.2
Tie 77.2
Apparel 64.9
Clothing 64.9
Photo 57.6
Photography 57.6
Doctor 57.6
Portrait 57.1
Face 57.1
Hospital 55.9
Coat 55.8

Imagga
created on 2022-02-11

person 32.2
man 31.6
people 31.2
male 31.2
nurse 26.2
businessman 22.9
business 22.5
adult 21.6
crutch 21.4
staff 21.4
patient 21.2
room 21
group 20.9
men 19.7
colleagues 14.6
happy 14.4
happiness 14.1
office 13.6
sick person 13.5
30s 13.5
women 13.4
case 13.2
indoors 13.2
stick 12.6
family 12.4
businesspeople 12.3
lifestyle 12.3
sitting 12
smiling 11.6
life 11.5
four 11.5
couple 11.3
meeting 11.3
senior 11.2
classroom 11.2
home 11.2
mature 11.1
professional 11.1
casual 11
businesswoman 10.9
team 10.7
40s 10.7
together 10.5
clothing 10.4
two 10.2
20s 10.1
medical 9.7
standing 9.6
corporate 9.4
teamwork 9.3
worker 9.2
black 9
suit 9
cheerful 8.9
job 8.8
working 8.8
chair 8.8
smile 8.5
kin 8.4
old 8.4
color 8.3
camera 8.3
human 8.2
brass 8.2
coworkers 7.8
forties 7.8
day 7.8
grandfather 7.8
table 7.8
hospital 7.7
elderly 7.7
talking 7.6
desk 7.5
joy 7.5
planner 7.3
portrait 7.1
wind instrument 7.1
interior 7.1

Google
created on 2022-02-11

Chair 87.4
Adaptation 79.3
Vintage clothing 75.6
Hat 75.4
Classic 73.2
Monochrome 70.3
Sitting 69.8
Room 67
Suit 66.8
Monochrome photography 65.8
History 65.5
Event 64.7
Window 58.6
Photo caption 57.7
Retro style 55.8
Team 55.4
Family 54.2

Microsoft
created on 2022-02-11

person 99.4
text 97.5
outdoor 97.1
clothing 88.5
group 61.5
man 59.8
smile 55
posing 46.5

Face analysis

Amazon

Google

AWS Rekognition

Age 48-54
Gender Male, 99.7%
Happy 96.4%
Calm 3.3%
Surprised 0.1%
Confused 0.1%
Disgusted 0.1%
Sad 0%
Angry 0%
Fear 0%

AWS Rekognition

Age 34-42
Gender Male, 90.6%
Calm 28.5%
Happy 27.4%
Fear 10.4%
Disgusted 8.8%
Surprised 8%
Angry 6.3%
Confused 6.2%
Sad 4.4%

AWS Rekognition

Age 51-59
Gender Female, 91.2%
Calm 99.6%
Surprised 0.1%
Happy 0.1%
Disgusted 0%
Fear 0%
Angry 0%
Sad 0%
Confused 0%

AWS Rekognition

Age 25-35
Gender Male, 98.1%
Surprised 52.2%
Happy 26.6%
Calm 16%
Confused 1.6%
Sad 1.4%
Disgusted 1.2%
Fear 0.5%
Angry 0.5%

AWS Rekognition

Age 52-60
Gender Male, 100%
Calm 86.4%
Happy 11%
Surprised 1.2%
Disgusted 0.6%
Sad 0.3%
Angry 0.3%
Confused 0.2%
Fear 0.1%

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Feature analysis

Amazon

Person 99.4%
Tie 97.2%
Chair 83.2%

Captions

Microsoft

a group of people posing for a photo 96.5%
a group of people posing for the camera 96.4%
a group of people posing for a picture 96.3%

Text analysis

Amazon

7