Human Generated Data

Title

Untitled (Fifteenth Street or Bethune Street, New York City)
Date

1932-1934
People

Artist: Ben Shahn, American 1898 - 1969
Classification

Photographs
Credit Line

Harvard Art Museums/Fogg Museum, Gift of Bernarda Bryson Shahn, P1970.2857
Copyright

© President and Fellows of Harvard College

Human Generated Data

Title

Untitled (Fifteenth Street or Bethune Street, New York City)

People

Artist: Ben Shahn, American 1898 - 1969

Date

1932-1934

Classification

Photographs

Machine Generated Data

Tags

Amazon
created on 2019-04-07

Person 99.3
Human 99.3
Handrail 99
Banister 99
Person 98.5
Brick 95.5
Wall 90
Home Decor 79.5
Clothing 70.5
Apparel 70.5
Indoors 65.7
Fireplace 57.5
Staircase 55.1

Clarifai
created on 2018-03-23

people 97.8
step 95.1
monochrome 94.8
no person 94.8
wall 91.7
adult 90
one 89.8
architecture 88.2
brick 88.1
street 85.6
home 85.2
wear 84.7
building 84
two 82.4
window 80.8
group 80.5
man 79.7
woman 79.3
shadow 78
indoors 77.8

Imagga
created on 2018-03-23

blackboard 63.2
building 34.7
architecture 30.4
brick 25.2
old 25.1
wall 25.1
structure 23.3
door 20.9
city 18.3
center 17.9
house 17.5
construction 16.2
window 15.8
stone 15.4
cell 14.8
exterior 14.7
billboard 14.7
urban 14
pattern 13
antique 13
shopping cart 12.8
texture 12.5
travel 12
street 12
signboard 11.4
buildings 11.3
ancient 11.2
grunge 11.1
historic 11
aged 10.9
dirty 10.8
style 10.4
handcart 10.2
town 10.2
tourism 9.9
modern 9.8
new 9.7
home 9.6
culture 9.4
vintage 9.1
retro 9
gate 8.9
wooden 8.8
business 8.5
design 8.4
famous 8.4
roof 8.3
sky 8.3
brown 8.1
tower 8.1
history 8
art 7.9
brass 7.9
scene 7.8
room 7.7
architectural 7.7
wheeled vehicle 7.7
damaged 7.6
perspective 7.5
sign 7.5
memorial 7.5
frame 7.5
church 7.4
rough 7.3
classroom 7.2
religion 7.2

Google
created on 2018-03-23

wall 88.6
infrastructure 88.6
brick 80
window 67.1
building 64.3
facade 58.3
black and white 55.8
street 54.9
angle 54.8

Microsoft
created on 2018-03-23

building 99.2
brick 93.6

Face analysis

Amazon

Microsoft

Google

AWS Rekognition

Age 26-43
Gender Female, 53.7%
Happy 45.2%
Confused 45.1%
Angry 46.5%
Disgusted 46%
Sad 45.3%
Calm 51.8%
Surprised 45.2%

AWS Rekognition

Age 4-9
Gender Female, 53.7%
Disgusted 45%
Calm 48%
Confused 45%
Surprised 45%
Sad 51.6%
Happy 45.1%
Angry 45.2%

Microsoft Cognitive Services

Age 33
Gender Male

Microsoft Cognitive Services

Age 4
Gender Female

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Feature analysis

Amazon

Person 99.3%

Captions

Microsoft

a person sitting in front of a brick building 69.1%
a person sitting in front of a brick building 64.9%
a person sitting in front of a brick wall 64.8%