Human Generated Data

Title

Untitled (large family portrait on stairway)

Date

c. 1950

People

Artist: Lucian and Mary Brown, American

Classification

Photographs

Credit Line

Harvard Art Museums/Fogg Museum, Transfer from the Carpenter Center for the Visual Arts, American Professional Photographers Collection, 4.2002.17961

Human Generated Data

Title

Untitled (large family portrait on stairway)

People

Artist: Lucian and Mary Brown, American

Date

c. 1950

Classification

Photographs

Machine Generated Data

Tags

Amazon
created on 2022-03-04

Human 99.6
Person 99.6
Apparel 99.6
Clothing 99.6
Person 99.5
Person 99.4
Person 99.3
Person 99.2
Person 99.2
Person 99
Person 98.6
Person 98.6
Accessories 97.3
Tie 97.3
Accessory 97.3
Dress 95.8
People 92.9
Female 92
Face 91.3
Overcoat 91
Coat 91
Suit 91
Chair 88.6
Furniture 88.6
Shorts 86.5
Woman 77.5
Flooring 70.9
Family 70.2
Floor 69.4
Portrait 69.3
Photo 69.3
Photography 69.3
Girl 67.9
Costume 64.8
Child 63.7
Kid 63.7
Food 61.5
Cake 61.5
Cream 61.5
Creme 61.5
Icing 61.5
Dessert 61.5
Stage 60.6
Table 58.8
Door 57.7
Tuxedo 55.2

Imagga
created on 2022-03-04

kin 33.2
people 29
person 24.6
man 24.2
nurse 22.1
male 22.1
men 20.6
couple 19.2
adult 19
women 16.6
businessman 15
happy 14.4
happiness 14.1
shop 13.8
business 13.4
barbershop 13.3
life 12.8
two 12.7
bride 12.5
family 12.5
portrait 12.3
smiling 12.3
player 11.6
room 11.5
together 11.4
professional 11.1
worker 10.9
patient 10.9
suit 10.8
contestant 10.1
wedding 10.1
dress 9.9
clothing 9.9
group 9.7
groom 9.6
mercantile establishment 9.5
love 9.5
corporate 9.4
smile 9.3
mother 9.2
black 9
human 9
new 8.9
office 8.8
world 8.8
daughter 8.8
home 8.8
celebration 8.8
day 8.6
married 8.6
elegance 8.4
old 8.4
ballplayer 8.3
fashion 8.3
cheerful 8.1
job 8
medical 7.9
standing 7.8
ceremony 7.8
adults 7.6
traditional 7.5
athlete 7.3
girls 7.3
color 7.2
uniform 7.2
team 7.2
to 7.1

Google
created on 2022-03-04

Microsoft
created on 2022-03-04

person 98.9
clothing 94.5
text 93.7
posing 92.4
group 81
man 72.8

Face analysis

Amazon

Google

AWS Rekognition

Age 26-36
Gender Male, 96.7%
Happy 77.6%
Surprised 10.5%
Calm 3.5%
Angry 3.2%
Sad 2.3%
Fear 1.5%
Disgusted 1%
Confused 0.3%

AWS Rekognition

Age 52-60
Gender Female, 75.1%
Calm 98.6%
Happy 0.7%
Surprised 0.5%
Sad 0.1%
Confused 0.1%
Disgusted 0%
Angry 0%
Fear 0%

AWS Rekognition

Age 54-64
Gender Male, 99.6%
Happy 82.3%
Surprised 12.2%
Calm 3.9%
Angry 0.5%
Sad 0.4%
Disgusted 0.3%
Fear 0.3%
Confused 0.2%

AWS Rekognition

Age 25-35
Gender Female, 99.4%
Calm 63.2%
Happy 27.4%
Surprised 4%
Disgusted 2.2%
Sad 1.6%
Fear 0.6%
Angry 0.5%
Confused 0.5%

AWS Rekognition

Age 24-34
Gender Male, 100%
Surprised 85.4%
Calm 7.9%
Sad 1.9%
Disgusted 1.3%
Confused 1%
Fear 0.9%
Angry 0.8%
Happy 0.8%

AWS Rekognition

Age 38-46
Gender Male, 99.6%
Happy 63.3%
Calm 13%
Surprised 10.4%
Sad 4.6%
Confused 3.1%
Fear 2.5%
Angry 1.6%
Disgusted 1.4%

AWS Rekognition

Age 30-40
Gender Male, 97.3%
Calm 91.2%
Surprised 4.2%
Happy 2.4%
Confused 0.9%
Disgusted 0.5%
Sad 0.3%
Angry 0.3%
Fear 0.3%

AWS Rekognition

Age 31-41
Gender Male, 98.1%
Calm 76.7%
Happy 15.4%
Surprised 4.3%
Sad 1.5%
Disgusted 0.8%
Confused 0.8%
Fear 0.3%
Angry 0.2%

AWS Rekognition

Age 29-39
Gender Female, 52.9%
Happy 48.7%
Sad 20.9%
Surprised 17%
Calm 4.3%
Fear 3.7%
Confused 2%
Disgusted 1.9%
Angry 1.5%

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Unlikely
Headwear Very unlikely
Blurred Very unlikely

Feature analysis

Amazon

Person 99.6%
Tie 97.3%

Captions

Microsoft

a group of people posing for a photo 95.2%
a group of people posing for the camera 95.1%
a group of people posing for a picture 95%

Text analysis

Amazon

42
2

Google

-XA
MJI7- -YT3RA2- -XA GO
-YT3RA2-
GO
MJI7-