Human Generated Data

Title

Colonel Ernst August von Hugo and Lt. Colonel von Schlepegrell

Date

1787

People

Artist: John Singleton Copley, American 1738 - 1815

Classification

Paintings

Credit Line

Harvard Art Museums/Fogg Museum, Gift of Mrs. Gordon Dexter, 1942.180

Human Generated Data

Title

Colonel Ernst August von Hugo and Lt. Colonel von Schlepegrell

People

Artist: John Singleton Copley, American 1738 - 1815

Date

1787

Classification

Paintings

Machine Generated Data

Tags

Amazon
created on 2019-04-08

Art 96.4
Painting 96.4
Human 94.1
Person 94.1
Apparel 82.1
Clothing 82.1
Hat 62.4
Linen 55.7
Home Decor 55.7
Face 55.6
Photography 55.6
Photo 55.6
Portrait 55.6

Clarifai
created on 2018-03-16

people 99.9
adult 99.2
lid 98.5
two 98.2
portrait 97.1
man 96.5
one 96
veil 95.4
wear 95.2
woman 93.8
painting 93.5
three 92.2
facial expression 89.9
group 87.4
art 86.9
son 86.8
jacket 85.3
child 84.4
famous 82.8
retro 80.1

Imagga
created on 2018-03-16

hat 68.5
portrait 33
person 30.1
headdress 28
face 27.7
cowboy hat 27.3
man 24.2
clothing 23.3
model 22.5
people 21.7
hair 19
fashion 18.1
happy 17.5
sombrero 17.5
male 17
attractive 16.1
looking 16
pretty 15.4
old 15.3
smile 15
adult 14.9
senior 14.1
lady 13.8
smiling 13.7
sexy 13.6
black 13.2
cute 12.9
make 12.7
covering 12.7
posing 12.4
eyes 12
costume 12
style 11.9
head 11.8
fun 11.2
outdoors 11.2
consumer goods 10.7
retro 10.6
vintage 9.9
eye 9.8
mysterious 9.7
close 9.7
beard 9.6
art 9.6
mystery 9.6
brunette 9.6
culture 9.4
expression 9.4
sensual 9.1
human 9
currency 9
cowboy 8.8
look 8.8
straw 8.7
hairstyle 8.6
wearing 8.6
makeup 8.2
one 8.2
gorgeous 8.2
dress 8.1
handsome 8
romantic 8
happiness 7.8
western 7.7
elegant 7.7
elderly 7.7
money 7.7
guy 7.6
casual 7.6
skin 7.6
hand 7.6
shirt 7.5
lips 7.4
nice 7.3
girls 7.3

Google
created on 2018-03-16

painting 86.9
portrait 79.8
art 73.8
headgear 61.9
gentleman 59.7
modern art 55.4
artwork 53.9
hat 51.7
illustration 50.3

Microsoft
created on 2018-03-16

person 97.2
old 72.9

Face analysis

Amazon

Microsoft

Google

AWS Rekognition

Age 23-38
Gender Male, 94.9%
Confused 3.6%
Sad 12%
Surprised 3.3%
Calm 70.8%
Happy 4.1%
Disgusted 2.7%
Angry 3.4%

AWS Rekognition

Age 49-69
Gender Male, 71%
Angry 1.2%
Surprised 1.8%
Happy 1.3%
Sad 1.1%
Calm 90.4%
Disgusted 3.6%
Confused 0.7%

AWS Rekognition

Age 26-43
Gender Female, 97%
Sad 38.4%
Disgusted 0.9%
Angry 2.7%
Happy 2.9%
Surprised 3.1%
Calm 50%
Confused 2.1%

Microsoft Cognitive Services

Age 71
Gender Male

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Possible
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Feature analysis

Amazon

Painting 96.4%
Person 94.1%

Captions

Microsoft

a man and a woman wearing a hat 65.5%
a person wearing a hat 65.4%
a close up of a man and a woman wearing a hat 59.2%