Human Generated Data

Title

Untitled (Mrs. Curtis?, seated, half-length, front view)

Date

c. 1858

People

Artist: Unidentified Artist,

Classification

Photographs

Credit Line

Harvard Art Museums/Fogg Museum, Gift of Janet and Daniel Tassel, 2007.219.54.22.A

Human Generated Data

Title

Untitled (Mrs. Curtis?, seated, half-length, front view)

People

Artist: Unidentified Artist,

Date

c. 1858

Classification

Photographs

Machine Generated Data

Tags

Amazon
created on 2019-11-07

Person 99.8
Human 99.8
Person 99.5
Art 97.8
Painting 87.4
Face 70.6
Apparel 67.6
Clothing 67.6
Portrait 64.8
Photo 64.8
Photography 64.8

Clarifai
created on 2019-11-07

people 99.6
one 99
adult 98.8
painting 98.2
art 97.8
portrait 97.4
print 96.5
wear 92.8
leader 92.6
illustration 91.3
woman 91.2
picture frame 90.1
two 88.1
veil 87.1
monarch 87.1
man 86.9
religion 84.3
crown 83.2
royalty 80.1
group 79.6

Imagga
created on 2019-11-07

mosaic 100
art 47.2
transducer 46
tile 41.2
electrical device 34.5
money 28.1
currency 27.8
dollar 26.9
cash 26.6
religion 26
paper 25.1
old 23.7
finance 23.7
device 23.1
creation 22.1
ancient 21.6
bank 21.5
financial 21.4
one 20.9
vintage 20.7
banking 20.2
antique 19.4
gold 18.9
culture 18.8
golden 18.1
wealth 18
stamp 17.4
close 17.1
postmark 16.8
postage 16.7
dollars 16.4
business 16.4
mail 16.3
bill 16.2
museum 16
decoration 15.9
church 15.7
letter 15.6
closeup 15.5
history 15.2
postal 14.7
god 14.4
architecture 14.3
symbol 14.2
religious 14.1
envelope 13.7
us 13.5
post 13.4
savings 13.1
rich 13
masterpiece 12.9
palace 12.8
banknote 12.6
loan 12.5
detail 12.1
icon 11.9
shows 11.8
printed 11.8
hundred 11.6
pay 11.5
instrumentality 11.5
retro 11.5
exchange 11.5
note 11
historic 11
pattern 10.9
painter 10.8
design 10.7
face 10.7
spirituality 10.6
painted 10.5
prayer rug 10.4
economy 10.2
philately 9.9
renaissance 9.8
profit 9.6
temple 9.4
commerce 9.3
sculpture 9.3
investment 9.2
travel 9.2
global 9.1
success 8.9
interior 8.9
paintings 8.8
finances 8.7
payment 8.7
fine 8.6
east 8.4
frame 8.3
traditional 8.3
rug 8.3
ornate 8.2
style 8.2
aged 8.1
office 8
post mail 7.9
zigzag 7.9
fame 7.9
known 7.9
franklin 7.9
funds 7.8
banknotes 7.8
arabesque 7.8
delivery 7.8
portrait 7.8
grand 7.8
cutting 7.7
wall 7.7
notes 7.7
communications 7.7
unique 7.6
sign 7.5
famous 7.4
object 7.3
color 7.2
landmark 7.2
black 7.2
collection 7.2

Google
created on 2019-11-07

Picture frame 92.5
Collection 68.7
Gentleman 66.5
Portrait 65.9
Painting 64.6
History 64.5
Art 62.5
Antique 60.5

Microsoft
created on 2019-11-07

person 97.6
human face 97.5
clothing 93.6
mirror 90.1
man 86.7
text 86.4
picture frame 63.7
reflection 59.8
painting 59.3
old 58
decorated 46.5

Face analysis

Amazon

Microsoft

Google

AWS Rekognition

Age 50-68
Gender Male, 98.7%
Disgusted 0.5%
Fear 0.2%
Happy 0.4%
Confused 1.4%
Sad 0.6%
Surprised 3.2%
Angry 1.5%
Calm 92.1%

AWS Rekognition

Age 35-51
Gender Female, 89.8%
Disgusted 0%
Happy 0%
Sad 0.1%
Angry 0%
Fear 0%
Calm 99.9%
Confused 0%
Surprised 0%

Microsoft Cognitive Services

Age 58
Gender Male

Microsoft Cognitive Services

Age 49
Gender Female

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Feature analysis

Amazon

Person 99.8%
Painting 87.4%

Captions

Microsoft

a painting of William Henry Rinehart et al. standing in front of a mirror 60%
a painting of William Henry Rinehart in front of a mirror posing for the camera 59.9%
a painting of William Henry Rinehart in front of a mirror posing for the camera 59.8%