Human Generated Data

Title

Untitled (Welfare Hospital, Welfare Island, New York City)

Date

1934-1935

People

Artist: Ben Shahn, American 1898 - 1969

Classification

Photographs

Credit Line

Harvard Art Museums/Fogg Museum, Gift of Bernarda Bryson Shahn, P1970.2836

Copyright

© President and Fellows of Harvard College

Human Generated Data

Title

Untitled (Welfare Hospital, Welfare Island, New York City)

People

Artist: Ben Shahn, American 1898 - 1969

Date

1934-1935

Classification

Photographs

Credit Line

Harvard Art Museums/Fogg Museum, Gift of Bernarda Bryson Shahn, P1970.2836

Copyright

© President and Fellows of Harvard College

Machine Generated Data

Tags

Amazon
created on 2023-10-06

Clothing 100
Adult 99.2
Male 99.2
Man 99.2
Person 99.2
Adult 99.2
Male 99.2
Man 99.2
Person 99.2
Accessories 97.8
Bag 97.8
Handbag 97.8
Sitting 97
Stick 86.5
Overcoat 85.3
Coat 83
Glasses 77.7
Face 76.6
Head 76.6
Footwear 71.4
High Heel 71.4
Shoe 61.6
Hat 59.3
Shoe 57.5
Tripod 56.9
City 55.4
Photography 55.2

Clarifai
created on 2018-05-10

people 100
adult 98.7
one 98
two 96.9
wear 96.7
woman 96.4
man 96
administration 95.8
military 91.5
lid 90.2
veil 89.3
child 88.2
portrait 87.4
war 87.3
outfit 86.9
group together 85
three 83.9
group 83.6
actor 83.1
sit 81.2

Imagga
created on 2023-10-06

crutch 100
staff 100
stick 92.5
man 29.6
person 20.2
male 19.1
adult 17.5
bass 13.9
protection 13.6
portrait 13.6
people 12.8
one 12.7
trombone 12.6
sport 12.4
black 12
brass 11.2
old 11.1
clothing 11
walking 10.4
men 10.3
danger 10
equipment 9.9
music 9.9
holding 9.9
hand 9.9
worker 9.8
dark 9.2
city 9.1
studio 9.1
playing 9.1
dirty 9
suit 9
musician 8.9
standing 8.7
work 8.6
mask 8.6
play 8.6
musical instrument 8.6
safety 8.3
street 8.3
outdoors 8.2
industrial 8.2
working 8
indoors 7.9
wind instrument 7.8
protective 7.8
military 7.7
chemical 7.7
gas 7.7
instrument 7.7
fashion 7.5
happy 7.5
business 7.3
smile 7.1
interior 7.1

Google
created on 2018-05-10

Microsoft
created on 2018-05-10

outdoor 99.5
person 97
old 50.3
curb 7.8

Color Analysis

Face analysis

Amazon

Microsoft

Google

Imagga

AWS Rekognition

Age 57-65
Gender Male, 99.9%
Sad 88.3%
Calm 30%
Angry 13.9%
Confused 11.6%
Surprised 6.7%
Fear 6.2%
Disgusted 1.5%
Happy 0.5%

AWS Rekognition

Age 59-67
Gender Female, 99.9%
Calm 89.2%
Surprised 8.1%
Fear 6%
Sad 2.9%
Angry 1.9%
Confused 1.3%
Happy 1.2%
Disgusted 0.7%

Microsoft Cognitive Services

Age 55
Gender Male

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Unlikely
Joy Very unlikely
Headwear Very likely
Blurred Very unlikely

Imagga

Traits no traits identified

Imagga

Traits no traits identified

Feature analysis

Amazon

Adult
Male
Man
Person
Handbag
Coat
Glasses
High Heel
Shoe
Hat
Adult 99.2%
Adult 99.2%
Male 99.2%
Male 99.2%
Man 99.2%
Man 99.2%
Person 99.2%
Person 99.2%
Handbag 97.8%
Coat 83%
Glasses 77.7%
High Heel 71.4%
Shoe 61.6%
Shoe 57.5%
Hat 59.3%

Captions

Clarifai
created by general-english-image-caption-blip on 2025-05-20

a photograph of a man and woman sitting on a bench in front of a building -100%

OpenAI GPT

Created by gpt-4o-2024-11-20 on 2025-06-08

The image depicts two individuals sitting on stone steps outdoors in front of a building entrance. One of them is holding multiple wooden crutches, while the other holds a handbag on their lap. Both are dressed in long coats, hats, and formal shoes, suggesting a historical setting. The surrounding includes a metal radiator and glass-panel doors, indicative of urban architecture. A small detail like a circular sign is visible on the wall. There is also a piece of paper or fabric lying on the ground near them.

Created by gpt-4o-2024-08-06 on 2025-06-08

The image shows two elderly individuals seated on the steps outside a building. They are dressed in long coats and hats, with the person on the left holding four crutches, suggesting mobility assistance. The individual on the right holds a handbag on their lap. The steps are made of concrete, and there are doors behind them, indicating the entrance of a building. A newspaper is strewn on the ground next to the steps. The style and clothing suggest the photograph may have been taken in the early to mid-20th century.

Anthropic Claude

Created by anthropic.claude-3-haiku-20240307-v1:0 on 2025-06-12

The image shows two elderly individuals sitting on the steps of a building. They appear to be a man and a woman, both wearing coats and hats. The man is using a cane or crutches to support himself. The image has a black and white, vintage aesthetic, suggesting it was taken some time in the past. The individuals look to be in a state of poverty or hardship, sitting on the steps with some papers or documents scattered around them.

Created by us.anthropic.claude-3-5-sonnet-20241022-v2:0 on 2025-06-12

This is a black and white historical photograph showing two people sitting on what appears to be front steps of a building. They are both wearing dark clothing typical of an earlier era, including hats. One person has crutches leaning nearby, suggesting they may have had mobility challenges. The image appears to be from the early to mid-20th century, based on the clothing styles and photographic quality. The architectural detail visible shows a brick building with steps leading to what might be an entrance or porch area.

Meta Llama

Created by us.meta.llama3-2-11b-instruct-v1:0 on 2025-05-27

The image depicts two elderly individuals sitting on a concrete step outside a building, with one of them using crutches for support. The person on the left is wearing a dark coat and hat, while the person on the right is dressed in a dark coat and hat, holding a purse in their lap. The background features a building with a door and windows, with a newspaper lying on the ground nearby. The overall atmosphere suggests a quiet, everyday moment captured in time.

Created by us.meta.llama3-2-90b-instruct-v1:0 on 2025-05-27

This image depicts two elderly women sitting on the steps of a building, with one woman using crutches. The woman on the left is dressed in a long coat and hat, while the woman on the right wears a long coat and holds a purse. Both women are seated on the steps, with the woman on the left positioned slightly higher than the other. The background features a building with a door and windows, as well as a radiator visible through an open door to the left. The overall atmosphere of the image suggests that it was taken in the early 20th century, likely during the 1920s or 1930s.

Amazon Nova

Created by amazon.nova-lite-v1:0 on 2025-05-27

The image is a black-and-white photograph of two elderly women sitting on the steps of a building. The woman on the left is holding a crutch in her right hand and has a serious expression on her face. The woman on the right is holding a handbag in her left hand and is wearing glasses. They are both wearing hats and long dresses. Behind them is a building with a door and a window.

Created by amazon.nova-pro-v1:0 on 2025-05-27

The black-and-white photograph depicts two women sitting on a concrete bench in front of a building. The woman on the left is wearing a hat, glasses, and leather shoes. She is holding a handbag and a pair of crutches. The woman on the right is wearing a hat, leather shoes, and eyeglasses. She is holding a handbag. There are pieces of paper on the ground. The building has a brick wall.

Google Gemini

Created by gemini-2.0-flash-lite on 2025-04-30

Here is a description of the image:

This black and white photograph features two elderly African American women seated on the steps of a building. The woman on the left is wearing a dark coat and hat, and is leaning on a pair of crutches. The woman on the right is also wearing a dark coat and hat, and she is holding a purse. Both women are wearing glasses and seem to be looking off into the distance. The background includes a doorway, steps, and a partially visible sign. There is a newspaper on the ground in front of them. The overall impression is of a quiet, dignified scene.

Created by gemini-2.0-flash on 2025-06-16

The black and white image shows two older African American women sitting on the steps of a building. One woman is holding a pair of crutches, and the other is holding a purse. Both women are wearing hats and dark clothing. The building has a double door with glass panels, and there is a radiator visible through the open door on the left. There are some papers scattered on the ground in front of the women.

Mistral Pixtral

Created by us.mistral.pixtral-large-2502-v1:0 on 2025-06-01

The image is a black-and-white photograph depicting two older individuals sitting on a bench or steps outside a building. The building has a brick facade with large windows and doors, and there is a grate or vent visible on the wall.

The individual on the left is a man wearing a suit, a hat, and glasses. He is using crutches, which are positioned next to him. He appears to be resting or waiting.

The individual on the right is a woman wearing a long coat, a hat, and glasses. She is holding a handbag and has a blanket or cloth draped over her legs. She also appears to be resting or waiting.

There are some papers or newspapers on the ground near their feet, suggesting they might be spending some time there, possibly waiting for someone or something. The overall atmosphere of the image conveys a sense of quiet patience and perhaps a hint of weariness.