Human Generated Data

Title

Untitled (waitress holding Heinz products behind diner counter)

Date

1941

People

Artist: Joseph Janney Steinmetz, American 1905 - 1985

Classification

Photographs

Credit Line

Harvard Art Museums/Fogg Museum, Transfer from the Carpenter Center for the Visual Arts, American Professional Photographers Collection, 4.2002.4888

Copyright

© Estate of Joseph Janney Steinmetz

Human Generated Data

Title

Untitled (waitress holding Heinz products behind diner counter)

People

Artist: Joseph Janney Steinmetz, American 1905 - 1985

Date

1941

Classification

Photographs

Credit Line

Harvard Art Museums/Fogg Museum, Transfer from the Carpenter Center for the Visual Arts, American Professional Photographers Collection, 4.2002.4888

Copyright

© Estate of Joseph Janney Steinmetz

Machine Generated Data

Tags

Amazon
created on 2022-01-23

Human 99.2
Clothing 92.9
Apparel 92.9
Shop 83
Restaurant 82.7
Meal 81.9
Food 81.9
Cafeteria 68.6
Indoors 68.5
Room 64.3
People 61.1

Clarifai
created on 2023-10-26

people 99.8
monochrome 99.7
adult 98.5
woman 96.1
man 94.4
indoors 92.2
furniture 89.7
stock 84.2
barber 82.9
group 80.7
room 77.6
cashier 76.8
sewing machine 76.7
child 72.8
shelf 71.9
department store 71.6
commerce 70.8
refrigerator 66
owner (law) 65.8
merchandise 65.4

Imagga
created on 2022-01-23

barbershop 100
shop 100
mercantile establishment 77.4
place of business 51.5
barber chair 41.5
chair 38.4
man 29.6
interior 26.5
establishment 25.7
furniture 25.5
seat 25.3
medical 24.7
people 22.9
male 22.7
doctor 22.6
person 22
work 21.2
room 21
indoors 20.2
hospital 19.1
health 18.8
clinic 17.3
modern 16.8
medicine 16.7
patient 16.6
equipment 16.3
working 15.9
professional 15.4
adult 15
kitchen 14.4
job 14.2
office 13.9
restaurant 13.4
senior 13.1
table 13
inside 12.9
indoor 12.8
smiling 12.3
home 12
happy 11.9
portrait 11.6
laboratory 11.6
exam 11.5
men 11.2
occupation 11
business 10.9
food 10.9
lifestyle 10.8
worker 10.7
lab 9.7
cooking 9.6
women 9.5
machine 9.5
counter 9.4
industry 9.4
smile 9.3
service 9.3
team 9
science 8.9
scientist 8.8
salon 8.8
standing 8.7
research 8.6
furnishing 8.3
treatment 8.3
holding 8.3
industrial 8.2
technology 8.2
looking 8
surgery 7.8
experiment 7.8
education 7.8
glass 7.8
factory 7.7
test 7.7
30s 7.7
profession 7.7
customer 7.6
casual 7.6
illness 7.6
dinner 7.6
house 7.5
care 7.4
cook 7.3
computer 7.2
to 7.1
decor 7.1
steel 7.1

Google
created on 2022-01-23

Microsoft
created on 2022-01-23

text 99.3
person 92.4
indoor 88.2
clothing 69.4
kitchen 54.4
store 35.2
shop 8.2

Color Analysis

Face analysis

Amazon

Google

AWS Rekognition

Age 40-48
Gender Female, 94%
Happy 99.9%
Surprised 0%
Sad 0%
Angry 0%
Calm 0%
Confused 0%
Fear 0%
Disgusted 0%

AWS Rekognition

Age 34-42
Gender Female, 67.5%
Sad 60.5%
Calm 31.6%
Happy 2%
Angry 1.5%
Disgusted 1.5%
Confused 1.1%
Fear 1%
Surprised 0.9%

AWS Rekognition

Age 26-36
Gender Male, 66.2%
Sad 55.9%
Confused 25.9%
Calm 7.2%
Happy 5.6%
Angry 2.1%
Disgusted 1.4%
Surprised 1.2%
Fear 0.8%

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very likely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Likely

Feature analysis

Amazon

Person

Clarifai

Clothing
Man
Human face
Building
Food
Table
Human head
Human arm
Human hair
Drawer
Footwear

AWS Rekognition

Person 99.2%

AWS Rekognition

Person 98.6%

AWS Rekognition

Person 98.6%

AWS Rekognition

Person 92.6%

AWS Rekognition

Person 82.7%

Clarifai

Clothing 86.2%

Clarifai

Clothing 83.7%

Clarifai

Clothing 80.6%

Clarifai

Clothing 67.9%

Clarifai

Clothing 53.5%

Clarifai

Clothing 34.6%

Clarifai

Clothing 34.5%

Clarifai

Man 82.6%

Clarifai

Human face 61.7%

Clarifai

Human face 57.6%

Clarifai

Human face 52.5%

Clarifai

Building 50.9%

Clarifai

Food 44.5%

Clarifai

Food 40.5%

Clarifai

Table 40.5%

Clarifai

Human head 40.3%

Clarifai

Human arm 39.3%

Clarifai

Human hair 35.6%

Clarifai

Human hair 33.8%

Clarifai

Drawer 34.9%

Clarifai

Footwear 33.8%

Categories

Imagga
created on 2022-01-23

streetview architecture 48.4%
people portraits 26.6%
paintings art 9.7%
text visuals 7.9%
beaches seaside 2.7%
events parties 2.4%

Captions

Clarifai

Created by general-english-image-caption-clip on 2025-07-12

photograph of a woman looking at a mirror.

Salesforce

Created by general-english-image-caption-blip on 2025-05-22

a photograph of a woman in a kitchen with a man in a apron

OpenAI GPT

Created by gpt-4o-2024-11-20 on 2025-06-12

This is a black-and-white, negative-style image of a diner or soda fountain scene, showing patrons seated at a counter while being served by a waitress. The counter is lined with plates, bottles, and cups, and the background features shelves stocked with various items such as food or condiments. A menu board displaying options like soup can be seen along with decorative elements. There appears to be a view of the street outside through the window, with lettering visible on the glass. Other details include lighting fixtures and a fan hanging from the ceiling.

Created by gpt-4o-2024-08-06 on 2025-06-12

This image depicts a scene inside a diner, captured in negative format. The setting features a long counter with stools where several individuals are seated, enjoying food and conversation. A waitress in a classic diner uniform stands behind the counter, possibly taking an order or serving food. The counter is lined with plates of food, beverages, and condiments. Behind the counter, shelves are stocked with various items, including cups, jars, and other diner essentials. Additionally, there is a menu board displaying different soup options. A large window in the background shows signage from outside, and a visible fan and light fixture are part of the interior decor. The image gives a nostalgic glimpse into mid-20th century diner culture.

Anthropic Claude

Created by anthropic.claude-3-haiku-20240307-v1:0 on 2025-06-20

The image appears to be a black and white photograph of the interior of a store or shop. The scene depicts a woman, likely an employee, standing in front of a display of various products and merchandise. The store appears to be well-stocked, with shelves and cabinets filled with a variety of items. In the background, there are other people, possibly customers, interacting with the products and the employee. The overall atmosphere suggests a bustling and active retail environment.

Created by us.anthropic.claude-3-5-sonnet-20241022-v2:0 on 2025-06-20

This is a black and white photograph that appears to be from a diner or restaurant counter, likely from the 1950s or early 1960s. The image shows an elegant counter setup with display cases and cabinets behind it. There's a waitress in a typical diner uniform with an apron serving customers who are seated at the counter. The counter appears to have various bottles and place settings, and there are decorative display cases with what look like bottles or products arranged on shelves. The overall aesthetic is very characteristic of mid-century American restaurant design, with clean lines and organized displays. A large fan can be seen in the upper right corner of the image. The photograph has reference numbers "H16003" visible at the top and bottom of the frame.

Meta Llama

Created by us.meta.llama3-2-11b-instruct-v1:0 on 2025-06-03

The image is a black-and-white photograph of a diner or restaurant, with a woman in a uniform standing behind the counter and several men seated at the counter.

The woman is wearing a white dress with a black collar and a black apron. She has short hair and is holding a glass in her right hand. The men are all wearing white shirts and have short hair. They are all looking at the woman as she speaks to them.

The counter is made of wood and has various items on it, including bottles, glasses, and plates. There are also several stools lined up along the counter. In the background, there is a large window that lets in natural light. The overall atmosphere of the image suggests a casual, everyday scene in a diner or restaurant.

Created by us.meta.llama3-2-90b-instruct-v1:0 on 2025-06-03

This black-and-white photograph depicts a diner scene, with a waitress standing at the counter, surrounded by patrons. The waitress, dressed in a uniform consisting of a short-sleeved dress and a hat, holds a glass in each hand. She stands behind the counter, which features a display case showcasing various items, including bottles and cans.

In the foreground, several patrons are seated at the counter, engaged in conversation or enjoying their meals. The atmosphere appears lively and bustling, with the waitress attending to the customers' needs. The overall setting suggests a classic American diner from a bygone era, with its retro decor and nostalgic charm.

Amazon Nova

Created by amazon.nova-lite-v1:0 on 2025-06-02

The image is a black-and-white photograph showing a scene inside a bar or a restaurant. The image appears to be a historical photograph, possibly from the mid-20th century. The photograph shows a woman standing behind a bar counter, smiling and holding a glass in her hands. There are several bottles and glasses on the counter, and a few customers are seated at the bar, engaged in conversation. In the background, there are shelves with various items, including bottles, glasses, and other bar supplies. The image also shows a fan hanging from the ceiling and a lamp on the wall. The photograph has a watermark in the top corners, indicating that it is a historical image.

Created by amazon.nova-pro-v1:0 on 2025-06-02

A black-and-white photo of a restaurant with a woman standing in front of a counter. She is wearing a hat and apron and holding a glass. Behind her are cabinets with bottles and a shelf with plates. On the right side, a man is sitting at a table with a bowl, a plate, and a cup. A man standing beside him is looking at something. There is a fan and a mirror in the distance.

Google Gemini

Created by gemini-2.0-flash on 2025-05-18

This is a black and white interior shot of what appears to be a diner. The image is a negative, reversing the colors of the scene. In the foreground, there are a few patrons seated at a counter, being served by a waitress. The waitress is standing behind the counter, wearing a uniform with an apron and cap. She is holding a couple of glasses.

The counter runs the length of the frame, and it is lined with condiments, cups, and plates. Behind the counter, there are shelves displaying various items, including bottles and cans, and a menu board. Another woman is standing in the background near the diner's window with her arms crossed.

A large fan is mounted on the ceiling, and there are hanging light fixtures. The overall atmosphere of the diner seems vintage, possibly from the mid-20th century. The image is labeled "H16003" at the top and bottom.

Created by gemini-2.0-flash-lite on 2025-05-18

Certainly! Here's a description of the image:

Overall Impression:

The photograph appears to be a negative image of a vintage diner or restaurant scene. It depicts a moment in time, capturing the interior and activity of the establishment.

Key Elements:

  • Setting: The primary setting is a diner with a long counter. The decor seems classic, with wooden cabinets, shelving, and possibly some display cases.
  • People: The photo features a waitress who appears to be serving customers. There are a few customers seated at the counter, engaged in their meal or drink. Another person in the back of the restaurant may be another staff member.
  • Details: The image has details like condiments bottles, cups, pies, and other items that point toward an era of the diner. A sign with a name that seems to be "Jeffrey Soups" is also visible.
  • Composition: The angle is from a side view. The main focus is on the waitress and the people eating at the counter.

Overall, the image evokes a sense of nostalgia, suggesting a moment from a bygone era.

Mistral Pixtral

Created by us.mistral.pixtral-large-2502-v1:0 on 2025-05-31

The image appears to be a black-and-white photograph taken in a commercial setting, likely a bar or a restaurant. The central figure is a woman dressed in what looks like a uniform, possibly a waitress or bartender, standing behind a counter. She is holding a bottle in one hand and a glass in the other, seemingly pouring a drink.

The background shows a well-stocked bar with various bottles of liquor displayed on shelves. The shelves are filled with an assortment of alcoholic beverages, including whiskey, gin, and other spirits. There is also a sign visible in the background that reads "Bourbon," indicating the presence of bourbon whiskey.

To the right of the woman, there are several patrons seated at the bar. They appear to be engaged in conversation or enjoying their drinks. The bar counter has various items on it, including glasses, bottles, and possibly some food items.

The setting suggests a lively and bustling atmosphere, typical of a bar or restaurant during its operating hours. The photograph captures a moment in time, providing a glimpse into the social and commercial activities of the era.

Qwen

Created by Qwen/Qwen2.5-VL-7B-Instruct on 2025-07-05

This is a black-and-white photograph depicting a vintage diner or soda fountain scene. The setting suggests a mid-20th-century era, possibly from the 1940s or 1950s, based on the style of the decor and attire.

In the foreground, a server wearing a traditional diner uniform—consisting of a dress or apron with a black belt and a hat—is holding a tray with two glasses, likely containing a drink, and smiling. The server's posture and expression suggest a welcoming and attentive demeanor.

The background features a well-organized diner counter with various condiments, bottles, and food items. A display case on the left side showcases several bottles, possibly containing sodas or other beverages. The counter is equipped with a large. metal shaker, a common fixture in soda fountains of that era.

On the right side of the image, there is a group of customers seated at the counter. They appear to be engaged in conversation and enjoying their food. A sign on the wall behind the server reads "Diner" in bold letters, confirming the setting as a diner. The overall ambiance is lively and nostalgic, capturing a moment of social interaction in a mid-century diner.

Created by Qwen/Qwen2.5-VL-72B-Instruct on 2025-07-05

This is a black-and-white photograph depicting a scene inside a diner or restaurant. The image shows a waitress, dressed in a white uniform with a black apron and a black cap, smiling and holding a glass in her hand. Behind her, there is a large display case filled with various bottles and containers, likely containing condiments and other items for customers. The display case has multiple shelves and compartments, and it is well-organized.

In the background, there is a sign that reads "DINER" in reverse, indicating that the photo was taken from the outside looking in through a window. Another person, possibly a chef or another staff member, is standing behind the counter, wearing a white uniform. There are also a few customers seated at the counter, engaged in conversation or waiting for their food.

The overall atmosphere of the image suggests a busy and cheerful dining environment, typical of a mid-20th-century American diner. The photograph is marked with the number "H 16003" in the top right corner and bottom left corner.

Text analysis

Amazon

FIRE
VICE
AUF
SOUPS
INC.-
URE
URE FIRE CAROLENE
VICE INC.- DISTRIBUTOR
H 16003
H 16003.
CAROLENE
2ЭЯ
DISTRIBUTOR
تسميمين
THE

Google

H1600 3 FIRE AL NI SOCPS H16003,
H1600
3
FIRE
AL
NI
SOCPS
H16003,