Human Generated Data

Title

Untitled (three girls at tea with their dolls)
Date

1959, printed later
People

Artist: Francis J. Sullivan, American 1916 - 1996
Classification

Photographs
Credit Line

Harvard Art Museums/Fogg Museum, Transfer from the Carpenter Center for the Visual Arts, American Professional Photographers Collection, 4.2002.147

Human Generated Data

Title

Untitled (three girls at tea with their dolls)

People

Artist: Francis J. Sullivan, American 1916 - 1996

Date

1959, printed later

Classification

Photographs

Machine Generated Data

Tags

Amazon
created on 2022-01-08

Clothing 99.8
Apparel 99.8
Chair 99.6
Furniture 99.6
Person 99.3
Human 99.3
Person 98.5
Person 94.3
Table 89.2
Meal 81.9
Food 81.9
Icing 81.2
Cream 81.2
Creme 81.2
Dessert 81.2
Cake 81.2
Face 78.8
Dish 75.5
Shoe 73.2
Footwear 73.2
People 72.9
Dining Table 71.9
Shoe 71.7
Photography 69.7
Photo 69.7
Portrait 69.5
Hat 64.2
Sitting 62.6
Kid 56.9
Child 56.9
Indoors 56.8
Room 55

Imagga
created on 2022-01-08

man 41.6
person 39.3
male 37.6
people 35.7
nurse 30.5
room 27.8
home 26.3
stretcher 24.6
classroom 23.5
group 23.3
happy 23.2
adult 22.9
men 21.4
table 21.1
sitting 20.6
office 20.3
smiling 20.2
together 20.1
couple 20
meeting 19.8
litter 19.6
smile 19.2
team 18.8
senior 18.7
patient 18.5
indoors 18.4
women 18.2
business 17.6
teamwork 16.7
work 16.5
talking 16.1
lifestyle 15.9
happiness 15.7
portrait 15.5
professional 15.3
family 15.1
communication 15.1
conveyance 15
worker 14.9
mature 14.9
desk 14.7
cheerful 14.6
teacher 14.6
laptop 14.1
businessman 14.1
businesswoman 13.6
chair 13.6
case 13.6
elderly 13.4
working 13.2
student 13
sick person 12.8
child 12.4
computer 12.1
indoor 11.9
love 11.8
education 11.2
corporate 11.2
retired 10.7
school 10.5
book 10.1
colleagues 9.7
retirement 9.6
husband 9.5
learning 9.4
friends 9.4
modern 9.1
health 9
job 8.8
boy 8.7
class 8.7
30s 8.6
grandfather 8.6
workplace 8.6
enjoying 8.5
casual 8.5
mother 8.5
friendship 8.4
horizontal 8.4
old 8.4
inside 8.3
successful 8.2
lady 8.1
suit 8.1
success 8
interior 8
medical 7.9
black 7.8
two people 7.8
older 7.8
four 7.7
loving 7.6
reading 7.6
businesspeople 7.6
laughing 7.6
togetherness 7.5
uniform 7.5
enjoyment 7.5
presentation 7.4
coffee 7.4
care 7.4
phone 7.4
occupation 7.3
color 7.2
handsome 7.1
kid 7.1

Google
created on 2022-01-08

Footwear 98
Hat 92
Table 85.3
Chair 74
Sharing 73.8
Vintage clothing 73.6
Toddler 73.5
Event 70.1
Classic 67.6
Sitting 65.5
Recreation 65.3
Monochrome 64.8
Stock photography 64.3
Monochrome photography 64.3
Art 61.5
Room 59.5
Sun hat 58.5
Retro style 56.5
Child 54.4
Photo caption 53.3

Microsoft
created on 2022-01-08

person 99.7
text 98.4
sitting 97.6
clothing 91.2
furniture 89.5
table 88.6
smile 78.5
chair 71
human face 62.7
people 57.6
old 48.6
posing 48.6

Face analysis

Amazon

Microsoft

Google

AWS Rekognition

Age 4-10
Gender Female, 95.8%
Calm 98.6%
Confused 0.6%
Surprised 0.3%
Disgusted 0.1%
Sad 0.1%
Angry 0.1%
Happy 0.1%
Fear 0%

AWS Rekognition

Age 2-8
Gender Male, 99.7%
Calm 79.3%
Confused 15.9%
Angry 2.2%
Fear 0.8%
Sad 0.6%
Surprised 0.5%
Disgusted 0.4%
Happy 0.2%

AWS Rekognition

Age 4-10
Gender Female, 91.2%
Happy 74.5%
Calm 11.7%
Confused 9.1%
Surprised 3.4%
Disgusted 0.4%
Fear 0.4%
Angry 0.3%
Sad 0.3%

AWS Rekognition

Age 0-4
Gender Female, 95.8%
Surprised 71.3%
Calm 12.7%
Fear 7.5%
Sad 4.7%
Confused 1.3%
Disgusted 1%
Happy 0.8%
Angry 0.7%

AWS Rekognition

Age 21-29
Gender Female, 96.6%
Sad 91.6%
Calm 7.2%
Angry 0.4%
Fear 0.2%
Happy 0.2%
Confused 0.2%
Disgusted 0.1%
Surprised 0.1%

Microsoft Cognitive Services

Age 4
Gender Male

Microsoft Cognitive Services

Age 5
Gender Female

Microsoft Cognitive Services

Age 7
Gender Female

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Likely
Headwear Possible
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Possible
Blurred Very unlikely

Feature analysis

Amazon

Person 99.3%
Shoe 73.2%
Hat 64.2%

Captions

Microsoft

a vintage photo of a group of people sitting posing for the camera 95.4%
a group of people sitting posing for the camera 95.3%
a vintage photo of a group of people sitting on a bench 90.6%