Human Generated Data

Title

The Family Home
Date

19th century
People

Artist: Unidentified Artist,

Classification

Paintings
Credit Line

Harvard Art Museums/Fogg Museum, Bequest of Mrs. William Hayes Fogg, 1895.687

Human Generated Data

Title

The Family Home

People

Artist: Unidentified Artist,

Date

19th century

Classification

Paintings

Machine Generated Data

Tags

Amazon
created on 2020-04-24

Person 99.8
Human 99.8
Person 98.9
Art 98.3
Painting 98.3
Person 91.1
People 79.2
Drawing 73.4
Apparel 61.9
Clothing 61.9

Clarifai
created on 2020-04-24

people 100
canine 99.3
group 99.3
adult 98.9
two 98.9
three 98
interaction 97.6
child 97.5
man 97.1
family 96.8
son 96.5
woman 96.2
four 96.1
sit 95.7
offspring 94.7
dog 94.3
furniture 94.2
group together 93.9
administration 90.9
wear 90.6

Imagga
created on 2020-04-24

person 29.5
man 24.9
male 24.8
adult 23.6
people 22.3
couple 18.3
portrait 17.5
sitting 17.2
mother 17
happy 16.3
child 16
patient 14.8
family 14.2
smiling 13.7
love 13.4
room 13
smile 12.8
religion 12.5
old 12.5
happiness 12.5
father 12.4
brother 12.3
together 12.3
home 12
parent 11.4
chair 11.3
fun 11.2
art 10.7
sculpture 10.6
statue 10.6
interior 10.6
sick person 10.6
fashion 10.6
husband 10.5
sit 10.4
case 10.4
men 10.3
face 9.9
dress 9.9
cadaver 9.8
attractive 9.8
posing 9.8
couch 9.7
boy 9.6
ancient 9.5
wife 9.5
kin 9.4
outdoors 9
sofa 8.7
lifestyle 8.7
culture 8.5
casual 8.5
two 8.5
human 8.2
one 8.2
hospital 8.2
childhood 8.1
expression 7.7
married 7.7
youth 7.7
grunge 7.7
god 7.7
pair 7.6
togetherness 7.5
relaxation 7.5
house 7.5
dark 7.5
traditional 7.5
leisure 7.5
vintage 7.4
life 7.4
lady 7.3
sexy 7.2
son 7.2
body 7.2
electric chair 7.1
architecture 7
indoors 7

Google
created on 2020-04-24

Sitting 66.1
Child 57.3
Family 55.3
Art 50.2

Microsoft
created on 2020-04-24

person 95.6
baby 92.9
human face 91
clothing 88.1
toddler 75
painting 74.9
smile 70.4
child 61.1
old 50.3
posing 36.4
crowd 0.7

Face analysis

Amazon

Microsoft

Google

AWS Rekognition

Age 5-15
Gender Female, 98%
Angry 0.3%
Fear 0.2%
Calm 87.9%
Sad 0.7%
Happy 9.5%
Confused 0.3%
Surprised 0.5%
Disgusted 0.5%

AWS Rekognition

Age 51-69
Gender Male, 65%
Angry 0%
Calm 0.2%
Disgusted 0%
Happy 0.1%
Fear 0%
Confused 0%
Surprised 0%
Sad 99.7%

AWS Rekognition

Age 0-4
Gender Female, 83.1%
Angry 0%
Calm 99.1%
Happy 0.6%
Confused 0%
Surprised 0.1%
Fear 0%
Sad 0.1%
Disgusted 0.1%

Microsoft Cognitive Services

Age 36
Gender Male

Microsoft Cognitive Services

Age 17
Gender Female

Microsoft Cognitive Services

Age 1
Gender Female

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Feature analysis

Amazon

Person 99.8%
Painting 98.3%

Captions

Microsoft

a group of people sitting posing for the camera 94.7%
a group of people sitting on a bench 83.2%
a group of people sitting posing for a photo 83.1%