Human Generated Data

Title

Untitled (couple sitting at counter in restaurant, waitress serving bowl of soup)
Date

c. 1950
People

Artist: Joseph Janney Steinmetz, American 1905 - 1985
Classification

Photographs
Credit Line

Harvard Art Museums/Fogg Museum, Transfer from the Carpenter Center for the Visual Arts, American Professional Photographers Collection, 4.2002.4564
Copyright

© Estate of Joseph Janney Steinmetz

Human Generated Data

Title

Untitled (couple sitting at counter in restaurant, waitress serving bowl of soup)

People

Artist: Joseph Janney Steinmetz, American 1905 - 1985

Date

c. 1950

Classification

Photographs

Machine Generated Data

Tags

Amazon
created on 2021-12-14

Human 99.7
Person 99.7
Person 99.6
Person 99.1
Person 99.1
Restaurant 94.5
Food 85.6
Meal 85.6
Clothing 82.3
Apparel 82.3
Cafeteria 77.9
Sitting 68.5
People 67.4
Cafe 63.1
Indoors 62
Room 59.5
Worker 59.4
Workshop 56.7
Lab 56.5

Imagga
created on 2021-12-14

shop 40
man 34.2
male 30.5
people 28.4
mercantile establishment 27.9
person 26.1
work 26
adult 23.1
business 22.5
men 22.3
worker 22.2
professional 22.1
medical 21.2
clinic 20.8
office 18.2
place of business 17.8
job 17.7
waiter 17.6
indoors 16.7
happy 16.3
team 16.1
restaurant 16
working 15.9
businessman 15.9
seller 15.1
doctor 15
barbershop 14.8
medicine 14.1
salon 13.8
supermarket 13.7
laboratory 13.5
customer 13.3
employee 13.3
building 13.2
portrait 12.9
smile 12.8
looking 12.8
lab 12.6
technology 12.6
profession 12.4
smiling 12.3
counter 12.3
manager 12.1
occupation 11.9
20s 11.9
women 11.9
dining-room attendant 11.7
assistant 11.7
lifestyle 11.6
room 11.3
teamwork 11.1
chair 10.5
group 10.5
research 10.5
grocery store 10.3
patient 10.2
modern 9.8
scientist 9.8
coat 9.7
chemistry 9.7
serious 9.5
corporate 9.4
casual 9.3
service 9.3
indoor 9.1
attractive 9.1
holding 9.1
health 9
suit 9
barber chair 8.8
table 8.7
standing 8.7
chemical 8.7
desk 8.6
hospital 8.6
tobacco shop 8.6
hairdresser 8.6
sitting 8.6
businesspeople 8.5
biology 8.5
store 8.5
meeting 8.5
two 8.5
student 8.1
cheerful 8.1
seat 8.1
interior 8
colleagues 7.8
marketplace 7.7
test 7.7
talking 7.6
retail 7.6
establishment 7.6
plan 7.6
instrument 7.5
one 7.5
study 7.5
coffee 7.4
stall 7.4
computer 7.2
home 7.2
architecture 7

Google
created on 2021-12-14

Shirt 93.8
Black-and-white 84.9
Style 83.8
Adaptation 79.3
T-shirt 78.9
Monochrome photography 77.3
Monochrome 75.9
Customer 71.5
Table 70.9
Room 68
Font 67.8
Vintage clothing 66.3
Art 64.8
Stock photography 63.4
History 63.2
Sitting 62.2
Cooking 61.6
Job 61.3
Suit 60.3
Photographic paper 57.3

Microsoft
created on 2021-12-14

text 99.8
clothing 94.6
person 92.7
table 90.5
man 89.4
bottle 75
black and white 72
food 63.7
shop 9

Face analysis

Amazon

Google

AWS Rekognition

Age 22-34
Gender Female, 92.6%
Happy 65.3%
Calm 29.7%
Sad 2.1%
Angry 1%
Surprised 0.7%
Confused 0.5%
Disgusted 0.4%
Fear 0.4%

AWS Rekognition

Age 24-38
Gender Female, 87.2%
Calm 66.1%
Happy 28.3%
Sad 3%
Angry 1.1%
Confused 0.8%
Surprised 0.3%
Fear 0.2%
Disgusted 0.2%

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Unlikely
Blurred Very unlikely

Feature analysis

Amazon

Person 99.7%

Captions

Microsoft

a group of people standing in front of a store 74.8%
a group of people in front of a store 73.6%
a group of people performing on a counter 73.5%

Text analysis

Amazon

SOUPS
HEINZ SOUPS
HEINZ
119306
LARGE
STYLE
PLATE
BOWL
PEA
CREAM
COUNTRY STYLE CHICKEN
TOMATO
Bowl
CHICKEN
OLD
COUNTRY
CREAM OF TOMATO
CHICKEN NOODLE
MUSHROOM
OLD FASHIONED
VEGETABLE
OF
NOODLE
20
FASHIONED
CREAM OF MUSHROOM
119306.
CHOWDER
VEGETABLE VEGETABLE BEEF
CLAM
15c
Coca-Cola
BEEF
CLAM GENUINE
Large Bowl 15c
CREAM OF
-
TURTLE
GENUINE
.
Hot
CHOWDER TURTLE
Het
Large
Piping
SYS
SQUP
the
LARGE BOWL 15€
adidas
CLAN CHOWDER
MACARONI
Piping 22 Hot minister!
OLD PASHIONED SEAN
PASHIONED
Staaklandwich adidas NEW 20
Staaklandwich
CREAM DE TOMATO
SEAN
minister!
CHOC
BEER
CLAN
GENUINE TURTLE
Piping 2- Het
VEGETABLE PER BEER
- - casa the . - TOMATO EXCIR
DE
CHOCO .
22
15€
2-
CLASS
MUSIROOM
CAS
CHEAM CH MUSIROOM
CHEAM CH
O)OH)
LND
- BICK
large
BICK
NEW
PER
CHOCOLATE
CHOCO
EXCIR
TURIS
casa
VISCONS

Google

*
SQUS
SOUPS
BEAN
TURTLE
CICN
OF
COUNTRY
CREAM
PEA
CHICKEN
SYE
॥१३०६. CocaCola MACARON PLATE HEINZ * SQUS HEINZ SOUPS HEINZ SOUPS Pain O RA Eo BEAN CREAN OOMA CREAM OF HuooM GENINE TURTLE Piping Hot CAN O PA VEGETABLE SLAM CHOWD CHICKEN NOOOLE CICN wICE CREAM OF MUSHROOM COUNTRY STYE CHICKEN CREAM OF TOMATO OLD FASHIONED BEAN VEGETABLE BEEF VEGETABLE GIAN CHOWDER GEMUINE TURm CREAM OF PEA CHICKEN NOODLE *LARGE BOWL 15 co c Longe Boul 15 CHOC SYE 119306.
HEINZ
Pain
O
CREAN
PA
wICE
MUSHROOM
BEEF
BOWL
Boul
CHOC
॥१३०६.
Eo
GENINE
FASHIONED
GIAN
GEMUINE
TURm
MACARON
PLATE
RA
OOMA
HuooM
Piping
Hot
CAN
VEGETABLE
CHOWD
OLD
CHOWDER
NOODLE
*LARGE
c
Longe
119306.
CocaCola
SLAM
NOOOLE
STYE
TOMATO
15
co