Human Generated Data

Title

Untitled (Artists' Union demonstration?, New York City)

Date

1935

People

Artist: Ben Shahn, American 1898 - 1969

Classification

Photographs

Credit Line

Harvard Art Museums/Fogg Museum, Gift of Bernarda Bryson Shahn, P1970.5596

Human Generated Data

Title

Untitled (Artists' Union demonstration?, New York City)

People

Artist: Ben Shahn, American 1898 - 1969

Date

1935

Classification

Photographs

Machine Generated Data

Tags

Amazon
created on 2019-11-11

Apparel 99.9
Clothing 99.9
Human 99.5
Person 99.5
Person 99.5
Hat 99.3
Person 99
Person 98.7
Person 97.8
Person 97.4
Person 97.4
Person 97.1
Person 96.6
Hat 96.1
Hat 92.4
Tie 89.8
Accessory 89.8
Accessories 89.8
Person 89.1
Sun Hat 88.1
Crowd 87.4
Person 87.3
Face 84.4
People 75.5
Hat 67.6
Cowboy Hat 67.1
Coat 65.9
Suit 65.9
Overcoat 65.9
Suit 64.7
Hat 60
Person 59.9
Audience 58
Photography 57.9
Photo 57.9
Portrait 57.9
Parade 57.2

Clarifai
created on 2019-11-11

people 99.8
group 98.8
many 98.1
group together 97.4
wear 96.8
adult 96.4
administration 94.4
man 94.2
military 93.5
outfit 93.4
veil 92.8
war 92.4
lid 90.9
leader 90.1
woman 89.7
crowd 88.3
several 87.7
uniform 87
fedora 80.4
recreation 80.3

Imagga
created on 2019-11-11

sculpture 32.8
old 23
statue 22.8
art 18.9
person 18.2
ancient 18.1
man 17.5
male 17
history 17
people 16.7
stone 15.6
newspaper 14.2
monument 14
religion 13.4
vintage 13.2
antique 13
architecture 12.5
uniform 11.9
cash 11.9
bank 11.8
bill 11.4
dollar 11.1
culture 11.1
money 11
room 10.8
travel 10.6
business 10.3
banking 10.1
product 10
aged 9.9
brass 9.9
currency 9.9
portrait 9.7
marble 9.7
bust 9.6
men 9.4
symbol 9.4
religious 9.4
finance 9.3
financial 8.9
wind instrument 8.9
creation 8.7
clothing 8.6
historical 8.5
daily 8.4
tourism 8.2
one 8.2
adult 8
building 7.7
spirituality 7.7
spiritual 7.7
pay 7.7
god 7.6
happy 7.5
retro 7.4
historic 7.3
lady 7.3
protection 7.3
smiling 7.2
paper 7.2
looking 7.2
wealth 7.2
musical instrument 7

Google
created on 2019-11-11

Crew 63.6
Crowd 57.8

Microsoft
created on 2019-11-11

text 99.8
person 99.2
clothing 96.7
outdoor 92.9
hat 92.3
man 89.8
human face 69.7
fashion accessory 65.2
fedora 56.2
crowd 1.3

Face analysis

Amazon

Microsoft

Google

AWS Rekognition

Age 33-49
Gender Male, 98.6%
Disgusted 0.9%
Sad 0.9%
Confused 1.1%
Calm 8.7%
Angry 4.6%
Fear 1.4%
Happy 80%
Surprised 2.4%

AWS Rekognition

Age 50-68
Gender Female, 52.4%
Angry 46.1%
Surprised 45.1%
Calm 45.2%
Confused 45.3%
Happy 45%
Disgusted 45.4%
Sad 52.2%
Fear 45.7%

AWS Rekognition

Age 31-47
Gender Male, 54.9%
Angry 45%
Disgusted 45%
Sad 45%
Fear 45%
Happy 54.6%
Confused 45%
Calm 45.3%
Surprised 45%

AWS Rekognition

Age 22-34
Gender Female, 54.9%
Happy 48.3%
Angry 45.3%
Calm 46.8%
Confused 46.2%
Disgusted 45.3%
Sad 45.6%
Fear 46.2%
Surprised 46.3%

AWS Rekognition

Age 32-48
Gender Male, 54.4%
Confused 45.1%
Calm 47%
Sad 45.2%
Angry 52.3%
Happy 45%
Fear 45.1%
Surprised 45.2%
Disgusted 45.1%

AWS Rekognition

Age 31-47
Gender Male, 96.7%
Surprised 2.9%
Happy 10.1%
Fear 0.9%
Angry 10.1%
Confused 1.1%
Calm 43.4%
Sad 0.5%
Disgusted 31.1%

AWS Rekognition

Age 32-48
Gender Male, 54.9%
Sad 45%
Calm 54.8%
Angry 45%
Happy 45%
Disgusted 45%
Fear 45%
Surprised 45.2%
Confused 45%

AWS Rekognition

Age 36-52
Gender Female, 51.8%
Calm 54.7%
Disgusted 45.1%
Angry 45%
Fear 45%
Surprised 45%
Happy 45.1%
Sad 45%
Confused 45%

AWS Rekognition

Age 32-48
Gender Male, 54.8%
Surprised 45%
Calm 54.9%
Angry 45%
Fear 45%
Happy 45%
Disgusted 45%
Sad 45%
Confused 45%

AWS Rekognition

Age 22-34
Gender Female, 54.9%
Fear 46.9%
Sad 52.7%
Calm 45%
Disgusted 45%
Happy 45%
Confused 45%
Surprised 45%
Angry 45.2%

AWS Rekognition

Age 24-38
Gender Male, 54.4%
Disgusted 45.1%
Fear 49.6%
Angry 48.6%
Sad 45.1%
Calm 45.6%
Surprised 45.9%
Confused 45.2%
Happy 45%

AWS Rekognition

Age 32-48
Gender Female, 50.1%
Calm 52.4%
Confused 45.1%
Sad 45.1%
Angry 45.1%
Fear 45%
Disgusted 45%
Surprised 46.2%
Happy 46.1%

AWS Rekognition

Age 27-43
Gender Female, 50.9%
Angry 46.1%
Calm 50.1%
Happy 45.8%
Fear 45.8%
Surprised 45.3%
Disgusted 45.1%
Sad 46.7%
Confused 45.1%

Microsoft Cognitive Services

Age 38
Gender Male

Microsoft Cognitive Services

Age 29
Gender Female

Microsoft Cognitive Services

Age 61
Gender Male

Microsoft Cognitive Services

Age 45
Gender Male

Microsoft Cognitive Services

Age 44
Gender Female

Microsoft Cognitive Services

Age 27
Gender Female

Microsoft Cognitive Services

Age 31
Gender Male

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Likely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very likely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very likely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Likely
Headwear Very likely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very likely
Blurred Very unlikely

Feature analysis

Amazon

Person 99.5%
Hat 99.3%
Tie 89.8%
Suit 65.9%

Captions

Microsoft

a group of people standing in front of a crowd 89.8%
a group of people in front of a crowd 89.7%
a group of people standing in front of a crowd of people 88.2%

Text analysis

Amazon

PRODUCTS
UNGER'S
DAIRY
BUTTER
27
mames
31
CHEESE
35 31 BUTTER
GREAM
iAKERS 158 UNGER'S DAIRY
iAKERS
GREAM 6 27
35
UNGE
CHEESE SVEET BUTTER CREAT
SVEET
158
6
CREAT
aoy

Google

27
NS8
BUTTER
PRODUCTS
SVEET
CHEESE
Gea
CREAM
31
man's BAKERS NS8 UNGER'S DAIRY PRODUCTS ONGER SVEET CHEESE Gea C 35 15AT CREAM 31 BUTTER 27
UNGER'S
DAIRY
ONGER
C
35
man's
BAKERS
15AT