Human Generated Data

Title

Untitled (Mulhall Family, Ozarks, Arkansas)

Date

1935

People

Artist: Ben Shahn, American 1898 - 1969

Classification

Photographs

Credit Line

Harvard Art Museums/Fogg Museum, Transfer from the Carpenter Center for the Visual Arts, 2.2002.1322

Human Generated Data

Title

Untitled (Mulhall Family, Ozarks, Arkansas)

People

Artist: Ben Shahn, American 1898 - 1969

Date

1935

Classification

Photographs

Machine Generated Data

Tags

Amazon
created on 2021-12-15

Person 99.7
Human 99.7
Person 95.7
Face 95.5
Outdoors 89.5
Clothing 84
Apparel 84
Baby 78.9
People 77.6
Nature 75.6
Female 74.2
Field 72.2
Photo 70.4
Photography 70.4
Portrait 70.4
Advertisement 65
Grass 62.9
Plant 62.9
Poster 59.6
Smile 59.6
Hat 58.3
Countryside 56.5
Collage 55.5
Dress 55.2

Imagga
created on 2021-12-15

kin 87.7
mother 71.2
parent 60.8
family 41.9
child 41.1
happy 40.8
portrait 36.9
outdoors 34.4
park 33.8
happiness 32.9
together 32.4
love 32.4
people 31.3
smiling 31.1
adult 28.5
daughter 25.8
man 25.6
senior 24.4
father 24.3
male 23.6
couple 23.5
lifestyle 23.2
smile 22.8
sibling 21.4
joy 20.1
person 18.7
fun 18.7
summer 18.7
outdoor 18.4
grass 18.2
outside 18
togetherness 18
childhood 17.9
cute 17.2
day 16.5
two 16.1
kid 16
affection 15.5
cheerful 15.5
laughing 15.1
boy 14.8
husband 14.3
women 14.3
countryside 13.7
face 13.5
leisure 13.3
mature 13
dad 12.8
youth 12.8
girls 12.8
pretty 12.6
garden 12.6
old 12.6
loving 12.4
wife 12.3
relaxed 12.2
healthy 12
son 12
hug 11.6
affectionate 11.6
little 11.5
enjoying 11.4
blond 11.3
relationship 11.3
hair 11.1
camera 11.1
relaxing 10.9
60s 10.8
retired 10.7
horizontal 10.1
baby 9.6
married 9.6
elderly 9.6
grandma 9.5
casual 9.3
adorable 9.2
children 9.1
holding 9.1
aged 9.1
black 9
romance 8.9
romantic 8.9
grandfather 8.9
embracing 8.8
autumn 8.8
two people 8.8
sepia 8.8
sitting 8.6
model 8.6
attractive 8.4
field 8.4
fashion 8.3
human 8.3
playing 8.2
vacation 8.2
meadow 8.1
group 8.1
having fun 7.9
seniors 7.9
sixties 7.9
50s 7.8
parenthood 7.8
sister 7.8
hugging 7.8
mom 7.8
eyes 7.8
retirement 7.7
walking 7.6
females 7.6
friendship 7.5
rest 7.4
joyful 7.4
lady 7.3
active 7.2
pensioner 7.1

Google
created on 2021-12-15

Microsoft
created on 2021-12-15

outdoor 99.4
human face 99.4
person 99
grass 98.9
baby 98.5
smile 97.5
clothing 95.7
toddler 95.7
text 94.3
child 87
boy 66.9
picture frame 7

Face analysis

Amazon

Microsoft

Google

AWS Rekognition

Age 28-44
Gender Female, 98.5%
Happy 30.2%
Sad 18.4%
Calm 14.4%
Confused 10.4%
Fear 10.2%
Angry 9%
Disgusted 5.7%
Surprised 1.7%

AWS Rekognition

Age 0-3
Gender Male, 72.9%
Sad 88%
Calm 7.5%
Angry 2.7%
Confused 0.6%
Fear 0.4%
Happy 0.3%
Surprised 0.3%
Disgusted 0.2%

Microsoft Cognitive Services

Age 1
Gender Female

Microsoft Cognitive Services

Age 58
Gender Female

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Likely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Feature analysis

Amazon

Person 99.7%

Captions

Microsoft

a person holding a baby 85.3%
a little girl posing for a photo 85.2%
a little girl posing for a picture 85.1%