Human Generated Data

Title

Untitled (varsity football team with coach, Friends School)

Date

1939

People

Artist: Harris & Ewing, American 1910s-1940s

Classification

Photographs

Credit Line

Harvard Art Museums/Fogg Museum, Transfer from the Carpenter Center for the Visual Arts, American Professional Photographers Collection, 4.2002.22277

Human Generated Data

Title

Untitled (varsity football team with coach, Friends School)

People

Artist: Harris & Ewing, American 1910s-1940s

Date

1939

Classification

Photographs

Machine Generated Data

Tags

Amazon
created on 2022-03-11

Person 99.8
Human 99.8
Person 98.8
Person 98.2
Person 97.9
Clothing 96.4
Apparel 96.4
Person 96.1
Person 95.9
Person 95.1
Person 91.9
Helmet 91.8
Person 91.8
Person 91.7
Shorts 89.9
Person 89.3
Person 88.7
Person 87.4
Helmet 87.1
People 84.8
Person 82.1
Helmet 81
Text 79.8
Person 78.8
Sport 69.3
Sports 69.3

Imagga
created on 2022-03-11

graffito 97.2
decoration 64.9
freight car 62.3
car 48.4
wheeled vehicle 38.6
grunge 32.3
old 30.6
vintage 28.1
vehicle 26.9
aged 21.7
texture 20.8
art 20.1
ancient 18.2
retro 18
black 17.4
antique 17.3
wall 17
grungy 16.1
dirty 14.5
space 14
paint 13.6
dark 13.4
city 13.3
graphic 13.1
text 13.1
pattern 13
design 12.9
conveyance 12.6
frame 11.6
building 11.6
artistic 11.3
empty 11.2
textured 10.5
damaged 10.5
urban 10.5
architecture 10.1
grain 10.1
concrete 9.6
structure 9.5
wallpaper 9.2
border 9
history 8.9
landscape 8.9
scene 8.7
stain 8.6
paper 8.6
worn 8.6
old fashioned 8.6
weathered 8.5
drawing 8.2
rough 8.2
style 8.2
backgrounds 8.1
colorful 7.9
grime 7.8
mottled 7.8
color 7.8
travel 7.7
decay 7.7
culture 7.7
textures 7.6
house 7.5
silhouette 7.4
street 7.4
brown 7.4
artwork 7.3
industrial 7.3
landmark 7.2
scenery 7.2
material 7.1
night 7.1
sky 7

Google
created on 2022-03-11

Photograph 94.2
Black 89.6
Standing 86.4
Sleeve 83
Headgear 82
Font 77.9
Crew 75.6
Team 70
Vintage clothing 70
Monochrome 69.5
Player 68.4
Uniform 68.2
History 67.2
Suit 66.1
T-shirt 65.9
Team sport 65.7
Room 64.6
Hat 63.2
Monochrome photography 63
Visual arts 57.5

Microsoft
created on 2022-03-11

outdoor 96.9
soccer 95.8
text 91.1
group 79.5
person 55.3
black and white 55
posing 38.5

Face analysis

Amazon

Google

AWS Rekognition

Age 23-31
Gender Female, 89.4%
Calm 91%
Happy 6.8%
Sad 0.7%
Surprised 0.5%
Angry 0.3%
Confused 0.3%
Disgusted 0.2%
Fear 0.2%

AWS Rekognition

Age 23-33
Gender Male, 90.7%
Calm 99.9%
Surprised 0%
Happy 0%
Confused 0%
Angry 0%
Sad 0%
Disgusted 0%
Fear 0%

AWS Rekognition

Age 36-44
Gender Male, 98.6%
Calm 73.9%
Happy 24.3%
Surprised 0.9%
Fear 0.5%
Sad 0.2%
Confused 0.1%
Angry 0.1%
Disgusted 0.1%

AWS Rekognition

Age 19-27
Gender Male, 99.7%
Calm 99.4%
Surprised 0.2%
Confused 0.1%
Disgusted 0.1%
Happy 0.1%
Angry 0.1%
Sad 0%
Fear 0%

AWS Rekognition

Age 24-34
Gender Male, 96.6%
Happy 38.7%
Surprised 36.1%
Calm 16.5%
Fear 4.4%
Sad 2.1%
Confused 0.9%
Disgusted 0.8%
Angry 0.6%

AWS Rekognition

Age 26-36
Gender Male, 96.2%
Calm 97.4%
Sad 1%
Happy 0.8%
Disgusted 0.2%
Angry 0.2%
Confused 0.1%
Surprised 0.1%
Fear 0.1%

AWS Rekognition

Age 39-47
Gender Male, 53.3%
Calm 99%
Surprised 0.4%
Sad 0.2%
Confused 0.1%
Angry 0.1%
Disgusted 0.1%
Fear 0.1%
Happy 0.1%

AWS Rekognition

Age 21-29
Gender Male, 92.9%
Calm 95.5%
Happy 4.2%
Surprised 0.1%
Fear 0.1%
Sad 0.1%
Disgusted 0%
Confused 0%
Angry 0%

AWS Rekognition

Age 24-34
Gender Male, 73.5%
Calm 66%
Happy 26.7%
Surprised 3.2%
Sad 1.9%
Angry 0.7%
Fear 0.5%
Confused 0.5%
Disgusted 0.4%

AWS Rekognition

Age 21-29
Gender Male, 98.4%
Calm 91%
Happy 7.3%
Surprised 0.7%
Fear 0.4%
Sad 0.2%
Disgusted 0.1%
Angry 0.1%
Confused 0.1%

AWS Rekognition

Age 24-34
Gender Male, 88.2%
Calm 99.9%
Surprised 0%
Sad 0%
Confused 0%
Disgusted 0%
Happy 0%
Angry 0%
Fear 0%

AWS Rekognition

Age 29-39
Gender Male, 96.2%
Calm 88.8%
Happy 10.6%
Surprised 0.3%
Disgusted 0.1%
Confused 0.1%
Sad 0.1%
Angry 0%
Fear 0%

AWS Rekognition

Age 38-46
Gender Male, 99.1%
Calm 95.8%
Happy 2.1%
Sad 1.3%
Surprised 0.4%
Disgusted 0.2%
Confused 0.1%
Angry 0.1%
Fear 0.1%

AWS Rekognition

Age 27-37
Gender Male, 96.7%
Calm 94.6%
Happy 3.2%
Surprised 1%
Sad 0.6%
Disgusted 0.2%
Confused 0.2%
Angry 0.2%
Fear 0.1%

AWS Rekognition

Age 23-33
Gender Male, 90.2%
Calm 92.6%
Happy 3.1%
Fear 1.2%
Sad 0.9%
Angry 0.7%
Disgusted 0.6%
Surprised 0.6%
Confused 0.3%

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Unlikely

Feature analysis

Amazon

Person 99.8%
Helmet 91.8%

Captions

Microsoft

a group of people posing for a photo 93.4%
a group of people posing for a picture 93.3%
a group of people posing for the camera 93.2%

Text analysis

Amazon

F
E
IF
E IF IF F F
Friends
'39
15
FI
IF A
15-
A