Human Generated Data

Title

Self-Portrait (with Cows)

Date

1796

People

Artist: Jean-Jacques de Boissieu, French 1736 - 1810

Classification

Prints

Credit Line

Harvard Art Museums/Fogg Museum, Gift of Belinda L. Randall from the collection of John Witt Randall, R4097

Human Generated Data

Title

Self-Portrait (with Cows)

People

Artist: Jean-Jacques de Boissieu, French 1736 - 1810

Date

1796

Classification

Prints

Machine Generated Data

Tags

Amazon
created on 2019-11-09

Person 99
Human 99
Art 98.7
Hat 97.5
Apparel 97.5
Clothing 97.5
Painting 71.5

Clarifai
created on 2019-11-09

people 100
one 99.8
lid 99.7
adult 99.3
veil 98.7
portrait 98.7
print 97.8
wear 97.6
art 97.6
outerwear 96.7
painting 92.1
man 91.8
two 91.1
outfit 90.8
engraving 90.2
bonnet 90.1
woman 89.6
writer 89
furniture 88.8
elderly 88.4

Imagga
created on 2019-11-09

money 36.6
currency 35.9
cowboy hat 32.7
hat 32.1
cash 32
dollar 28.8
bank 27.8
bill 27.6
banking 24.8
finance 23.7
headdress 23.3
exchange 21
paper 20.6
wealth 19.8
financial 19.6
portrait 18.1
business 17.6
close 16.6
savings 15.9
rich 15.8
face 15.6
note 15.6
clothing 15.6
banknote 15.5
us 15.4
pay 15.3
art 15.3
economy 14.8
ancient 14.7
culture 14.5
dollars 14.5
man 14.3
vintage 14.1
loan 13.4
investment 12.8
banknotes 12.7
one 12.7
newspaper 12.6
religion 12.6
old 12.5
person 12.5
closeup 12.1
male 12.1
product 11.8
tattoo 11.8
hundred 11.6
market 11.5
covering 11.4
historical 11.3
bills 10.7
payment 10.6
success 10.5
statue 10.5
decoration 10.4
stone 10.1
sculpture 10.1
world 10
creation 9.7
notes 9.6
antique 9.5
design 9.5
golden 9.5
stock 9.4
church 9.3
history 8.9
people 8.9
franklin 8.9
twenty 8.9
funds 8.8
symbol 8.8
economic 8.7
states 8.7
finances 8.7
god 8.6
profit 8.6
united 8.6
consumer goods 8.5
commerce 8.4
sale 8.3
black 7.9
president 7.9
child 7.8
pound 7.7
price 7.7
painter 7.6
head 7.6
human 7.5
religious 7.5
retro 7.4
cemetery 7.1
architecture 7

Google
created on 2019-11-09

Microsoft
created on 2019-11-09

hat 98.7
drawing 96.2
wall 95.5
old 94.2
clothing 93.8
text 93.8
person 91.5
human face 90.6
man 90.5
sketch 90.3
painting 86.8
cowboy hat 81.7
fedora 81.6
fashion accessory 78.6
photograph 60.1
gallery 56.5
room 51.4
vintage 39.2
older 30.6
picture frame 14.9

Face analysis

Amazon

Microsoft

Google

AWS Rekognition

Age 49-67
Gender Male, 99.3%
Disgusted 0%
Angry 0.1%
Surprised 0%
Happy 0%
Sad 1.2%
Calm 98.6%
Fear 0%
Confused 0.1%

AWS Rekognition

Age 20-32
Gender Male, 98.6%
Confused 0.7%
Calm 37.6%
Fear 7.1%
Happy 0.1%
Surprised 18.6%
Disgusted 33.2%
Sad 0.5%
Angry 2.1%

Microsoft Cognitive Services

Age 54
Gender Male

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Unlikely
Blurred Very unlikely

Feature analysis

Amazon

Person 99%
Hat 97.5%
Painting 71.5%

Captions

Microsoft

a vintage photo of a man 88%
a vintage photo of a man wearing a hat 82.4%
a vintage photo of a man wearing a black hat 80.4%

Text analysis

Amazon

21810.
yonr
Sauert
His
323.116
a o F . yonr 21810.
His Cur
Cur
F .
o
a

Google

II DS-1796
DS-1796
II