Human Generated Data

Title

Virgin and Child with the Infant Saint John the Baptist, Saint Jerome, and an Unidentified Male Saint
Date

16th century
People

Artist: Bartolommeo Neroni, called il Riccio, Italian c. 1500 - before 1571
Classification

Paintings
Credit Line

Harvard Art Museums/Fogg Museum, Gift of Evelyn S. Griswold, Gladys S. May, and Eirene W. Smyth, 1958.85

Human Generated Data

Title

Virgin and Child with the Infant Saint John the Baptist, Saint Jerome, and an Unidentified Male Saint

People

Artist: Bartolommeo Neroni, called il Riccio, Italian c. 1500 - before 1571

Date

16th century

Classification

Paintings

Machine Generated Data

Tags

Amazon
created on 2020-04-24

Human 98.7
Person 98.7
Person 96.7
Person 93.9
Art 92.1
Person 92
Painting 76.8
Photography 67.5
Photo 67.5
Portrait 67.5
Face 67.5
Jaw 63.2

Clarifai
created on 2020-04-24

people 100
portrait 99.2
adult 98.9
two 98.6
man 98.5
three 97.9
group 97.8
affection 97
baby 95.2
facial expression 93.2
art 93.2
administration 93
son 92.8
four 91.6
child 90.9
group together 90.8
interaction 90.2
actor 90
family 89.8
music 89.2

Imagga
created on 2020-04-24

family 63.2
child 60.8
father 55.7
mother 50.9
male 46
happy 45.8
kin 44.5
love 44.2
together 42.1
parent 41.7
grandfather 39
dad 37.6
man 36.3
smiling 34.7
people 31.8
couple 31.4
husband 28.6
portrait 28.5
son 28.2
daughter 28
kid 27.5
fun 26.9
adult 26.5
home 25.5
smile 25
wife 24.7
happiness 24.3
children 22.8
senior 22.5
brother 21.6
cute 21.5
boy 20.9
sibling 20.9
togetherness 20.8
casual 19.5
baby 19.2
kids 18.8
lifestyle 18.8
hug 18.4
person 17.8
loving 17.2
laughing 17
childhood 16.1
two 16.1
joy 15.9
affectionate 15.5
sitting 15.5
cheerful 15.4
married 15.3
mature 14.9
old 14.6
mom 14.5
face 14.2
grandmother 13.7
group 13.7
parents 13.7
playing 13.7
couch 13.5
affection 13.5
little 13.3
interior 12.4
hugging 11.7
30s 11.5
elderly 11.5
attractive 11.2
juvenile 11
life 10.9
handsome 10.7
outdoors 10.5
men 10.3
relaxing 10
studio 9.9
generations 9.9
sister 9.8
retired 9.7
retirement 9.6
generation 9.6
enjoying 9.5
camera 9.2
adorable 9.2
leisure 9.1
holding 9.1
park 9.1
60s 8.8
embracing 8.8
indoors 8.8
diversity 8.7
four 8.6
race 8.6
black 8.4
pretty 8.4
blond 8.3
clothing 8.2
gray 8.1
grandma 7.9
women 7.9
grandparents 7.9
mommy 7.9
bond 7.9
parenthood 7.8
color 7.8
boys 7.8
play 7.8
infant 7.7
comfort 7.7
laugh 7.7
youth 7.7
ethnic 7.6
house 7.5
friends 7.5
relationship 7.5
girls 7.3
room 7.3
day 7.1

Google
created on 2020-04-24

Portrait 65.1
Photography 62.4
Photo caption 54.7
Art 50.2

Microsoft
created on 2020-04-24

text 100
book 99.7
human face 97.1
baby 94.7
sketch 94.2
person 92.3
drawing 90.9
painting 77.8
posing 68.2
toddler 60.5
man 50.6

Face analysis

Amazon

Microsoft

Google

AWS Rekognition

Age 13-25
Gender Female, 95.4%
Calm 67.7%
Fear 0.3%
Angry 1%
Sad 29.6%
Surprised 0.1%
Disgusted 0.2%
Happy 0.1%
Confused 0.9%

AWS Rekognition

Age 35-51
Gender Male, 98.6%
Calm 20.6%
Disgusted 0.4%
Surprised 0.3%
Fear 0.7%
Happy 0.2%
Angry 2.8%
Sad 71.5%
Confused 3.6%

AWS Rekognition

Age 34-50
Gender Male, 89.9%
Disgusted 5.4%
Fear 3.1%
Sad 18.2%
Calm 48.5%
Happy 1.8%
Confused 5.2%
Surprised 4.9%
Angry 12.8%

AWS Rekognition

Age 51-69
Gender Male, 97.1%
Disgusted 0.9%
Fear 0.4%
Sad 86.7%
Happy 0.4%
Angry 2.2%
Surprised 0.3%
Calm 7.1%
Confused 2.1%

AWS Rekognition

Age 21-33
Gender Female, 80.8%
Surprised 0.2%
Sad 1.4%
Confused 0.6%
Disgusted 0.6%
Fear 0.1%
Angry 60.9%
Happy 0.2%
Calm 35.9%

Microsoft Cognitive Services

Age 22
Gender Female

Microsoft Cognitive Services

Age 49
Gender Male

Microsoft Cognitive Services

Age 68
Gender Male

Microsoft Cognitive Services

Age 2
Gender Female

Microsoft Cognitive Services

Age 34
Gender Female

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Feature analysis

Amazon

Person 98.7%
Painting 76.8%

Captions

Microsoft

a group of people posing for a photo 78.4%
a man holding a book 57.5%
a group of people posing for the camera 57.4%

Text analysis

Amazon

in
in e 0M6
0M6
e