Human Generated Data

Title

Untitled (Bowery, New York City)
Date

1932-1934
People

Artist: Ben Shahn, American 1898 - 1969
Classification

Photographs
Credit Line

Harvard Art Museums/Fogg Museum, Gift of Bernarda Bryson Shahn, P1970.2460
Copyright

© President and Fellows of Harvard College

Human Generated Data

Title

Untitled (Bowery, New York City)

People

Artist: Ben Shahn, American 1898 - 1969

Date

1932-1934

Classification

Photographs

Machine Generated Data

Tags

Amazon
created on 2019-04-07

Human 97.9
Person 97.9
Text 97.4
Clothing 96
Apparel 96
Hat 96
Person 95.6
Advertisement 95.6
Poster 94.1
Overcoat 90.2
Suit 90.2
Coat 90.2
Flyer 84.8
Paper 84.8
Brochure 84.8
Accessory 71
Tie 71
Accessories 71
Suit 61

Clarifai
created on 2018-03-23

people 99
adult 95.3
text 92.3
man 89.9
war 89.4
music 87.7
woman 87.1
one 86.3
group 85.3
education 84.9
bill 84.8
conceptual 84.6
business 84.5
retro 84.5
portrait 84.5
musician 80.6
monochrome 80.3
wear 78.9
administration 78.7
vehicle 78.6

Imagga
created on 2018-03-23

blackboard 100
menu 40.8
vintage 28.1
fare 27.6
old 25.8
grunge 24.7
retro 24.6
design 23.1
text 21.8
art 21.7
antique 21.6
frame 19.2
graphic 18.2
texture 18.1
letter 17.4
card 17
book jacket 16.7
black 16.2
word 16
business 14.6
aged 14.5
decoration 14.5
food 13.8
cloud 13.8
stamp 13.5
style 13.4
paper 13.3
book 13.2
jacket 13
ancient 13
message 11.9
definition 11.8
collection 11.7
element 11.6
backdrop 11.5
advertising 11.5
information 11.5
creative 11.5
conceptual 11.5
symbol 11.5
sign 11.3
data 11
wrapping 10.9
border 10.9
decorative 10.9
mail 10.5
pattern 10.3
cover 10.2
artwork 10.1
dirty 9.9
postmark 9.9
chalkboard 9.8
chalk 9.7
collage 9.6
memorial 9.5
board 9.2
template 9.2
structure 8.9
decor 8.8
brass 8.8
brochure 8.8
advertise 8.8
product 8.7
advertisement 8.7
wall 8.6
label 8.4
dollar 8.4
dark 8.4
brown 8.1
religion 8.1
circa 7.9
words 7.8
envelope 7.8
empty 7.7
blank 7.7
culture 7.7
notes 7.7
tag 7.7
wallpaper 7.7
damaged 7.6
covering 7.6
grungy 7.6
composition 7.5
close 7.4
new 7.3
creation 7.2
copy 7.1
textured 7

Google
created on 2018-03-23

black and white 84.3
poster 77.6
human behavior 71
font 68.1
monochrome 65.2
design 65.1
advertising 64.5
art 60.1
monochrome photography 52.7
vintage clothing 50.2

Microsoft
created on 2018-03-23

outdoor 98.5
text 87.7

Face analysis

Amazon

Microsoft

AWS Rekognition

Age 60-90
Gender Male, 54.5%
Sad 47%
Confused 45.2%
Happy 45.1%
Disgusted 45.2%
Angry 45.5%
Surprised 45.3%
Calm 51.7%

AWS Rekognition

Age 20-38
Gender Male, 54.7%
Disgusted 45.6%
Surprised 45.5%
Sad 48.9%
Calm 47.4%
Angry 46.6%
Happy 45.4%
Confused 45.4%

Microsoft Cognitive Services

Age 32
Gender Male

Feature analysis

Amazon

Person 97.9%
Hat 96%
Suit 90.2%
Tie 71%

Captions

Microsoft

a person standing in front of a sign 76.2%
a person standing in front of a building 69.2%
a person holding a sign 66.3%

Text analysis

Amazon

GREENPER
ALL
CRULER
PIGSFEET
PIGSHEAD
BOILEDHAM
SPARERIBS
REG.
OFFEE
FRESARM
20
LAMBSTEW
5
ROLLS
ROAST
CORNEOBEEF
POTATOPANCAKES2BRON
BRERSTOFLAMB
8
NOODLES.
PIGSKNUCKCLES
SHOKTRIBS
RESTAURANT
MEATBALLS2MRRRO PIGSHEAD PIGSSNOUTS
ASTLEGOFLAMB
VORL
BUNS ROLLS ROAST FOASTZIR FRESARM OICI NOODLES. ORSTSIRLOINOFBEEF GREENPER
POTATOPANCAKES2BRON BILEDBACON
BUNS
OHIE CORNEOBEEF PIGSKNUCKCLES 20 BEEFSTEW LAMBSTEW
NNER
ORSTSIRLOINOFBEEF
MEATBALLS2MRRRO
OFFEE CRULER SOUPS:VEG.BOUITL9N REG.OINNER1525. NNER BOUJLLN&
PORKBITTS
5 SHOKTRIBS RORSTSHOUD. -OOFPORK ASTLEGOFLAMB VORL
OXTAILGOULPSH
PIGSSNOUTS
SMRPRON
SITTEREDNOODLESIBALON
BILEDBACON
REG.OINNER1525.
BEEFSTEW
PiE 8 BOILEDHAM SPARERIBS BRERSTOFLAMB OXTAILGOULPSH 1
SITTEREDNOODLESIBALON SMRPRON PIGSFEET PORKBITTS SIERCPRRIR oe
-OOFPORK
FOASTZIR
SOUPS:VEG.BOUITL9N
RORSTSHOUD.
BOUJLLN&
OHIE
OICI
SIERCPRRIR
PiE
oe
B
1

Google

ESTAURANT A SOUPS VEGB00LN&NOODLES GREENPER CeuLeRs SHORTRİBSe SEEFA BEEFSTen 飣 KNUCKLES CORNEOBEEF AMB STeh OXTAILGOULASH MEATBALLS&RARONPIGSFEE HASHANDONTONS VERANDONIONS CHILI CONCARNE RPESTen BRAISED3EEF RIB OEYEA MULLER DAIRI BUTTERMIL
ESTAURANT
A
SOUPS
VEGB00LN&NOODLES
GREENPER
CeuLeRs
SHORTRİBSe
SEEFA
BEEFSTen
飣
KNUCKLES
CORNEOBEEF
AMB
STeh
OXTAILGOULASH
MEATBALLS&RARONPIGSFEE
HASHANDONTONS
VERANDONIONS
CHILI
CONCARNE
RPESTen
BRAISED3EEF
RIB
OEYEA
MULLER
DAIRI
BUTTERMIL