Human Generated Data

Title

Residents of Columbus, Ohio

Date

1938

People

Artist: Ben Shahn, American 1898 - 1969

Classification

Photographs

Credit Line

Harvard Art Museums/Fogg Museum, Transfer from the Carpenter Center for the Visual Arts, 2.2002.3006

Human Generated Data

Title

Residents of Columbus, Ohio

People

Artist: Ben Shahn, American 1898 - 1969

Date

1938

Classification

Photographs

Credit Line

Harvard Art Museums/Fogg Museum, Transfer from the Carpenter Center for the Visual Arts, 2.2002.3006

Machine Generated Data

Tags

Amazon
created on 2021-12-15

Human 99.8
Footwear 99
Clothing 99
Apparel 99
Stick 87.3
Tarmac 85.6
Asphalt 85.6
Cane 79
Road 71.4
Path 70.3
Sitting 56.9

Clarifai
created on 2023-10-15

people 99.8
elderly 98.7
two 98.2
lid 98.1
man 97.7
three 97.5
adult 97.2
monochrome 97
portrait 95.9
street 95.3
veil 95.2
group together 92.8
group 92.7
four 89.1
wear 87.4
documentary 87.4
walking stick 84.4
seat 82.2
old 82.1
chair 81.9

Imagga
created on 2021-12-15

man 30.9
people 25.1
hat 23.2
male 22.7
person 22.3
cowboy hat 21.6
sky 20.4
beach 20.2
water 19.3
travel 18.3
sea 18
ocean 16.6
sunset 16.2
summer 15.4
vacation 14.7
clothing 14.5
outdoor 14.5
sun 14.5
couple 13.9
headdress 13.7
adult 13.6
sand 13.3
world 12.6
walking 12.3
men 12
coast 11.7
silhouette 11.6
lifestyle 11.6
building 11.4
crutch 11.3
outdoors 11.2
landscape 11.2
old 11.1
love 11
relax 10.9
architecture 10.9
life 10.7
tourism 10.7
two 10.2
scholar 9.7
working 9.7
together 9.6
walk 9.5
work 9.4
happy 9.4
industry 9.4
waves 9.3
environment 9
staff 8.9
family 8.9
job 8.8
seller 8.5
stone 8.4
black 8.4
father 8.4
lady 8.1
child 8
business 7.9
holiday 7.9
intellectual 7.8
sitting 7.7
outside 7.7
newspaper 7.7
senior 7.5
leisure 7.5
shore 7.4
worker 7.4
safety 7.4
occupation 7.3
peaceful 7.3
alone 7.3
tourist 7.2
looking 7.2
portrait 7.1
day 7.1
parent 7

Google
created on 2021-12-15

Microsoft
created on 2021-12-15

text 99.7
outdoor 99.6
clothing 99
ground 98.4
person 94.7
man 93.4
hat 87
footwear 78.4
black and white 77.8
sidewalk 73.8
fashion accessory 71.3
smile 60.7
street 54.4

Color Analysis

Face analysis

Amazon

Google

AWS Rekognition

Age 37-55
Gender Male, 95.8%
Calm 86.7%
Angry 5%
Sad 4.3%
Happy 2.1%
Disgusted 0.6%
Surprised 0.5%
Fear 0.4%
Confused 0.4%

AWS Rekognition

Age 53-71
Gender Male, 96.2%
Calm 99.1%
Surprised 0.5%
Happy 0.1%
Angry 0.1%
Sad 0.1%
Fear 0%
Disgusted 0%
Confused 0%

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very likely
Blurred Very unlikely

Feature analysis

Amazon

Person
Shoe
Hat

Clarifai

Man
Footwear
Human face
Human leg
Human arm
Fedora

AWS Rekognition

Person 99.8%

AWS Rekognition

Person 99.5%

AWS Rekognition

Person 96.8%

AWS Rekognition

Person 94.8%

AWS Rekognition

Shoe 99%

AWS Rekognition

Shoe 96.4%

AWS Rekognition

Hat 98.3%

Clarifai

Man 92.3%

Clarifai

Man 88.9%

Clarifai

Man 56.2%

Clarifai

Man 35.8%

Clarifai

Footwear 91.5%

Clarifai

Footwear 90.8%

Clarifai

Footwear 89.3%

Clarifai

Footwear 72.8%

Clarifai

Footwear 68.3%

Clarifai

Footwear 38.6%

Clarifai

Human face 54.9%

Clarifai

Human face 49.5%

Clarifai

Human leg 49.4%

Clarifai

Human leg 38%

Clarifai

Human leg 37.3%

Clarifai

Human arm 49.1%

Clarifai

Human arm 43.2%

Clarifai

Human arm 35.1%

Clarifai

Fedora 33.1%

Categories

Imagga
created on 2021-12-15

people portraits 68.3%
paintings art 25.2%
pets animals 2.1%
streetview architecture 1.3%
nature landscape 1.1%

Captions

Microsoft
created by unknown on 2021-12-15

a man sitting on a bench 88.9%
a man and a woman sitting on a bench 75.5%
a man sitting on a sidewalk 75.4%

Clarifai

Created by general-english-image-caption-clip on 2025-07-12

photograph of men sitting on a bench.

Salesforce

Created by general-english-image-caption-blip on 2025-05-22

a photograph of a man and woman sitting on a bench in front of a brick wall

OpenAI GPT

Created by gpt-4o-2024-11-20 on 2025-06-12

The image depicts two men sitting outdoors on a sidewalk. One man is seated on a folding chair, dressed in a buttoned-down shirt, suspenders, and a hat, holding a walking stick. He appears to be resting and engaged in conversation. The other man, wearing a white shirt, dark pants, and lace-up boots, sits directly on the ground against a wall. The background reveals a brick wall with a painted "25¢" and partial text, and the sidewalk has paving and a metal grate. Objects and blurred individuals are seen farther in the background, suggesting an urban setting.

Created by gpt-4o-2024-08-06 on 2025-06-12

The image depicts two men sitting side by side on the street. The man on the left is wearing a light-colored button-up shirt with rolled-up sleeves, suspenders, and a fedora-style hat, and his hands are resting on his lap. He is seated on a folding chair. The man on the right is wearing a light-colored shirt and dark pants, and he is sitting on the curb with one leg crossed over the other. He is also wearing boots and a flat cap. The background features a brick wall with part of a painted advertisement displaying "25¢" and some partially obscured text "TO ORI". There is a sidewalk and street scene in the background with people walking by.

Anthropic Claude

Created by anthropic.claude-3-haiku-20240307-v1:0 on 2025-06-18

The image shows two elderly men sitting on a bench outside a building. They appear to be resting or conversing, with one man leaning forward and the other sitting back. The background includes a brick wall with a sign that says "25c E TO OPER" which seems to be an advertisement or sign. The scene has a black and white, vintage aesthetic, suggesting this is an older photograph.

Created by us.anthropic.claude-3-5-sonnet-20241022-v2:0 on 2025-06-18

This is a black and white photograph that appears to be from an earlier era, possibly the 1930s or 1940s. Two men are sitting on what looks like a sidewalk or storefront step. Behind them is a brick wall with "25¢" painted on it. One man is wearing a fedora hat and light-colored shirt with work clothes, while the other wears a flat cap and similar working attire. They appear to be taking a break or resting, with one man smoking. The image has a documentary or street photography style and captures what seems to be a candid moment of working-class life during that period. The setting appears urban, with sidewalk grating visible in the frame.

Meta Llama

Created by us.meta.llama3-2-11b-instruct-v1:0 on 2025-06-05

The image depicts two men sitting on a sidewalk, with one man wearing a hat and holding a cane. The background features a brick wall with a sign that reads "25¢" and "THE STO TO ORI," suggesting a vintage or retro setting. The overall atmosphere appears to be relaxed and casual, with the men engaged in conversation or simply enjoying each other's company. The image may have been taken in the early 20th century, given the clothing and accessories worn by the men.

Created by us.meta.llama3-2-90b-instruct-v1:0 on 2025-06-05

The image depicts two men sitting on a sidewalk, with one man holding a cane. The man on the left is wearing a hat and has his hand resting on his chin, while the man on the right is also wearing a hat and has his legs crossed. In the background, there is a brick wall with white lettering that reads "25c" and other partially visible words. The overall atmosphere of the image suggests a casual and relaxed setting, possibly during a break or leisure time.

Amazon Nova

Created by amazon.nova-lite-v1:0 on 2025-06-03

The black-and-white image shows two elderly men sitting on a brick sidewalk. The man on the left is wearing a hat and holding a cane, while the man on the right is wearing a cap and holding his hands together. They are sitting next to a brick wall with a black sign that reads "25" and "POSTO ORDI" in white letters. Behind them, there is a person walking on the sidewalk. The image captures a moment of quiet reflection and companionship between the two men.

Created by amazon.nova-pro-v1:0 on 2025-06-03

The black-and-white photo features two men sitting on the edge of a sidewalk. The man on the left is wearing a hat and holding a cane in his left hand, while his right hand is near his mouth. The man on the right is wearing a hat and holding his hands together. Behind them is a brick wall with the words "25c" and "STO OR" written on it.

Google Gemini

Created by gemini-2.0-flash on 2025-05-16

Here is a description of the image:

This black and white photograph features two older men sitting outside, seemingly taking a break. On the left, an older man is seated in a folding chair, holding a cane. He is wearing a light-colored, button-down shirt with rolled-up sleeves, suspenders, and a fedora. He has a cigarette in his mouth and his elbow rests on the cane. Behind him, there is a brick wall with a faded sign that reads "25 cents TO ORDER."

To the right, another older man is sitting on the edge of what appears to be a sidewalk, with his legs crossed. He is wearing a flat cap, a long-sleeved white shirt, and dark pants tucked into high-laced boots.

The setting appears to be an outdoor urban scene. Other people can be seen passing by in the background. The overall tone of the image is candid and suggests a moment of quiet observation of everyday life.

Created by gemini-2.0-flash-lite on 2025-05-16

Here's a description of the image:

This is a black and white photograph depicting two older men, likely in a public space. The photo's composition suggests a candid, documentary style.

Key elements:

  • Subject: Two men are the primary focus.
    • The man on the left is sitting on a folding chair, wearing a hat, and holding a cane. He has his hand up near his mouth.
    • The man on the right is sitting on a ledge with his legs crossed, also with a focused expression.
  • Setting: The setting appears to be a street or outdoor area. The background shows a brick wall with the large numbers "25" and the letter "c". The other wall is made of an old and weathered brick. There are other people in the background of the photo.
  • Tone/Atmosphere: The photograph evokes a sense of quiet observation and a possible moment of rest or contemplation. The lighting and overall composition give a classic, timeless feel.

Mistral Pixtral

Created by us.mistral.pixtral-large-2502-v1:0 on 2025-06-01

The image is a black-and-white photograph depicting two elderly men sitting on a sidewalk. The man on the left is wearing a light-colored hat, a light-colored shirt with rolled-up sleeves, and suspenders. He is holding a cane in his right hand and appears to be smoking a cigarette. The man on the right is wearing a flat cap, a light-colored shirt, dark pants, and lace-up boots. He is sitting with his arms resting on his knees and appears to be looking off to the side.

In the background, there is a brick wall with a partially visible sign that reads "25¢" and "STO OR." The sidewalk they are sitting on is made of bricks, and there is a metal grate near the man on the right. The scene suggests a casual, possibly urban setting, and the men seem to be engaged in a relaxed conversation or simply passing the time. The overall mood of the photograph is calm and reflective, capturing a moment of everyday life.

Qwen

Created by Qwen/Qwen2.5-VL-7B-Instruct on 2025-07-01

The image is a black-and-white photograph featuring two elderly men sitting on a bench in what appears to be an outdoor setting, possibly a street or public area. The man on the left is wearing a fedora hat, a long-sleeved shirt, suspenders, and trousers. He is holding a walking cane in one hand and appears to be deep in thought or conversation. The man on the right is wearing a cap, a long-sleeved shirt, trousers, and boots. He is seated on the same bench and is gesturing with his hands, possibly in the middle of a discussion.

In the background, there is a brick wall with the text "25c" painted prominently, suggesting a price or a sign for a shop or service. Further in the background, part of a store or shop with a window is visible. The sidewalk and street are paved with brick and concrete, and there are a few other people in the background who are not the main focus of the image. The overall atmosphere of the photo conveys a sense of everyday life and community interaction.

Created by Qwen/Qwen2.5-VL-72B-Instruct on 2025-07-01

This black-and-white photograph captures a candid street scene with two older men sitting in front of a storefront. The man on the left, wearing a hat and suspenders, is seated on a folding chair and appears to be resting his chin in his hand, looking contemplative. The man on the right, wearing a cap and a long-sleeve shirt, is seated on the step of a building and has his hands clasped together, looking off to the side. Behind them, a painted sign on the brick wall reads "25¢" and "TO OR," suggesting a historical setting. The sidewalk and building details give the image a vintage feel.

Text analysis

Amazon

TO
E
ORE
Г TO ORE
25e
Г

Google

25. E STO OR
25.
E
STO
OR