Human Generated Data

Title

Family on relief, Lancaster, Ohio

Date

1938

People

Artist: Ben Shahn, American 1898 - 1969

Classification

Photographs

Credit Line

Harvard Art Museums/Fogg Museum, Transfer from the Carpenter Center for the Visual Arts, 2.2002.3019

Human Generated Data

Title

Family on relief, Lancaster, Ohio

People

Artist: Ben Shahn, American 1898 - 1969

Date

1938

Classification

Photographs

Credit Line

Harvard Art Museums/Fogg Museum, Transfer from the Carpenter Center for the Visual Arts, 2.2002.3019

Machine Generated Data

Tags

Amazon
created on 2021-12-15

Human 99.6
Porch 93
Furniture 84.2
Sitting 70.4
Outdoors 64.9
Leisure Activities 63.8
Meal 61.7
Food 61.7
People 61.1
Chair 60.8
Deck 60.4
Cafe 58.2
Restaurant 58.2

Clarifai
created on 2023-10-15

people 100
adult 99.1
child 98
woman 96.6
nostalgia 96.5
group together 96.2
two 95.9
three 95.6
group 95.3
wear 95
furniture 93.2
man 91.4
offspring 89.5
actress 89.5
portrait 89.5
nostalgic 88.2
retro 88
actor 87.3
four 87
vintage 85.9

Imagga
created on 2021-12-15

man 26.2
male 23.9
people 21.7
room 20.9
family 20.4
person 19.8
happy 18.8
classroom 17.9
portrait 17.5
chair 17.5
barbershop 17.1
adult 15.8
love 15.8
black 15.6
shop 15.2
child 15.2
shopping cart 14.7
wheeled vehicle 14.3
couple 13.9
lifestyle 13.7
handcart 13.6
old 12.5
seat 12.5
mother 12.2
sitting 12
happiness 11.7
smiling 11.6
outdoors 11.2
window 11
parent 10.7
mercantile establishment 10.6
home 10.4
senior 10.3
women 10.3
youth 10.2
building 9.8
kid 9.7
fun 9.7
together 9.6
urban 9.6
married 9.6
husband 9.5
men 9.4
father 9.3
smile 9.3
city 9.1
kin 9.1
one 9
world 8.9
boy 8.7
day 8.6
wife 8.5
joy 8.3
care 8.2
indoor 8.2
cheerful 8.1
bench 8.1
indoors 7.9
retired 7.8
daughter 7.7
outside 7.7
casual 7.6
relax 7.6
conveyance 7.5
relationship 7.5
leisure 7.5
wheelchair 7.5
floor 7.4
alone 7.3
playing 7.3
dirty 7.2
place of business 7.1

Google
created on 2021-12-15

Microsoft
created on 2021-12-15

text 98.6
window 94.2
clothing 91.9
person 91.6
furniture 86.9
chair 80.5
black 79.6
white 77.2
old 45

Color Analysis

Face analysis

Amazon

Microsoft

Google

AWS Rekognition

Age 22-34
Gender Female, 92.8%
Calm 56.5%
Sad 27.4%
Angry 12.8%
Confused 1.4%
Fear 1%
Disgusted 0.4%
Surprised 0.3%
Happy 0.2%

AWS Rekognition

Age 20-32
Gender Male, 62.7%
Calm 86.6%
Sad 7.9%
Angry 3.8%
Disgusted 0.6%
Confused 0.5%
Surprised 0.2%
Fear 0.2%
Happy 0.1%

AWS Rekognition

Age 36-54
Gender Male, 99.2%
Confused 56.2%
Calm 22.3%
Sad 16.2%
Disgusted 1.8%
Surprised 1.3%
Angry 1%
Fear 0.7%
Happy 0.5%

AWS Rekognition

Age 0-3
Gender Male, 80.1%
Fear 82.3%
Surprised 8.4%
Calm 6.8%
Happy 0.8%
Confused 0.6%
Angry 0.4%
Disgusted 0.4%
Sad 0.3%

AWS Rekognition

Age 49-67
Gender Female, 50%
Calm 60.9%
Happy 30.3%
Angry 4.9%
Fear 1.3%
Sad 1.3%
Surprised 0.9%
Disgusted 0.3%
Confused 0.1%

Microsoft Cognitive Services

Age 49
Gender Male

Microsoft Cognitive Services

Age 5
Gender Female

Microsoft Cognitive Services

Age 0
Gender Female

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Feature analysis

Amazon

Person

Clarifai

Man
Woman
Footwear
Clothing
Human face
Chair
Human arm
House
Human leg
Tree

AWS Rekognition

Person 99.6%

AWS Rekognition

Person 99.5%

AWS Rekognition

Person 99.3%

AWS Rekognition

Person 97.1%

AWS Rekognition

Person 76%

Clarifai

Man 85.7%

Clarifai

Woman 82.1%

Clarifai

Footwear 81.6%

Clarifai

Footwear 80.6%

Clarifai

Footwear 74.6%

Clarifai

Footwear 69.2%

Clarifai

Footwear 50.6%

Clarifai

Footwear 37.4%

Clarifai

Footwear 35.6%

Clarifai

Footwear 35.2%

Clarifai

Clothing 76.5%

Clarifai

Clothing 68.9%

Clarifai

Clothing 44%

Clarifai

Clothing 41.8%

Clarifai

Human face 64.5%

Clarifai

Human face 62.6%

Clarifai

Human face 57.9%

Clarifai

Human face 47.1%

Clarifai

Human face 41.6%

Clarifai

Chair 47.4%

Clarifai

Chair 37.7%

Clarifai

Human arm 42.9%

Clarifai

Human arm 41.3%

Clarifai

House 42.4%

Clarifai

Human leg 38.8%

Clarifai

Human leg 33.7%

Clarifai

Tree 35.4%

Categories

Imagga
created on 2021-12-15

paintings art 98.7%

Captions

Clarifai

Created by general-english-image-caption-clip on 2025-07-11

photograph of a family sitting on a porch.

Salesforce

Created by general-english-image-caption-blip on 2025-05-22

a photograph of a group of people sitting on rocking chairs

OpenAI GPT

Created by gpt-4o-2024-11-20 on 2025-06-14

The black-and-white image depicts a scene of daily life on a porch. A woman stands with her hands on her hips, dressed in a patterned dress, while another woman sits in a rocking chair holding a small child. A man is seated in another rocking chair, leaning slightly to the side. Another small child sits in a separate rocking chair, closer to the center of the composition. The porch is constructed with wooden planks and features brick walls, a window, a doorway, and graffiti markings behind the figures. The overall atmosphere suggests a moment of quiet relaxation or conversation.

Created by gpt-4o-2024-08-06 on 2025-06-14

The image depicts a group of people sitting on a porch with a wooden railing. The building has brick walls and there are two windows visible on either side of the door. One of the windows reflects trees from the outside. A man in a dark outfit is seated to the left of the image on a rocking chair, resting his elbow on one knee. A woman standing in the middle, wearing a patterned dress cinched at the waist with a belt, has her hands on her hips. In front of her, a small child with light hair is seated in a small rocking chair. To the right, another woman is sitting on a chair with her legs crossed, holding a baby in light clothing. The porch has worn wooden floorboards. Visible on the brick wall are some chalk marks or writing.

Anthropic Claude

Created by anthropic.claude-3-haiku-20240307-v1:0 on 2025-06-18

The image appears to be a black and white photograph depicting a family on the porch of a rustic, wooden building. In the foreground, there is a woman sitting in a rocking chair holding a baby. Next to her, a young child is also seated in a rocking chair. In the background, a man is visible sitting on the porch. The building has a brick exterior and the scene suggests a rural or small-town setting. The overall atmosphere of the image conveys a sense of domestic tranquility and family life.

Created by us.anthropic.claude-3-5-sonnet-20241022-v2:0 on 2025-06-18

This is a black and white photograph that appears to be from an earlier era, possibly the 1930s or 1940s. The image shows people on what appears to be a front porch of a brick building. There are several rocking chairs on the porch. A woman stands in the middle wearing a patterned dress with a belt at the waist. There are two small children visible, and another adult seated on the porch. The setting suggests this might be a rural or working-class neighborhood, with the simple wooden porch and brick exterior visible. The composition has a documentary-style quality typical of Depression-era photography.

Meta Llama

Created by us.meta.llama3-2-11b-instruct-v1:0 on 2025-06-09

The image is a black-and-white photograph of a family sitting on a porch.

The family consists of a man, a woman, and two babies. The man is sitting in a rocking chair on the left side of the porch. He has short hair and is wearing a collared shirt, pants, and shoes. The woman is standing behind the man, with her hands on her hips. She is wearing a dress with a floral pattern and has her hair pulled back. One baby is sitting in a chair in front of the woman, and the other baby is being held by the woman.

The porch is made of wood and has a railing along the edge. There is a brick wall behind the porch, with a window on the left side and a door on the right side. The overall atmosphere of the image is one of relaxation and contentment, as the family enjoys each other's company on the porch.

Created by us.meta.llama3-2-90b-instruct-v1:0 on 2025-06-09

This image is a black-and-white photograph of a group of people on the porch of a brick building. The group consists of two women, two men, and three children.

On the left side of the image, a man sits in a rocking chair, wearing dark pants and a light-colored shirt. Next to him is a woman standing with her hands on her hips, wearing a light-colored dress with a dark belt. In front of her is a child sitting in a rocking chair.

On the right side of the image, a woman sits in a rocking chair, holding a baby in her lap. She is wearing a light-colored dress and has her legs crossed. The background of the image shows the brick building, with windows and a door visible. The overall atmosphere of the image suggests a casual gathering or social event, possibly a family reunion or community gathering.

Amazon Nova

Created by amazon.nova-lite-v1:0 on 2025-06-08

The image is a black-and-white photograph showing a family on the porch of a house. The family consists of four members: a man, a woman, and two children. The man is seated in a rocking chair on the left side of the image, while the woman is standing in front of the open door, leaning on the door frame with her hands on her hips. The two children are sitting in a rocking chair in the middle of the image, with one child in the lap of the other. The porch has a wooden railing and a brick wall in the background.

Created by amazon.nova-pro-v1:0 on 2025-06-08

A black-and-white photo shows a family of four sitting on the porch of a house. The woman is standing, with her hands on her hips, looking at the man on the left. The man is sitting on a chair, wearing boots, and holding a cigarette. The woman is sitting on a chair, holding a baby. The baby is wearing a pacifier. A brick wall and a window are on the side of the house.

Google Gemini

Created by gemini-2.0-flash-lite on 2025-05-16

Here is a description of the image:

The image is a black and white photograph depicting a group of people on what appears to be a porch. It is likely taken outdoors during the day. The scene is divided into two parts.

On the left side, a man is seated in a rocking chair, leaning his elbow on his leg and holding his chin. Next to him is a woman in a floral dress with a belt standing with her hands on her hips. There is a young child in another rocking chair.

On the right side, a woman is seated in a rocking chair, holding a baby. The baby is wearing a bib and appears to have a pacifier in its mouth.

The porch has exposed brick on its walls. There is writing in chalk on the brick. The image suggests a sense of family life and possibly a moment of relaxation or leisure. The style of clothing suggests the photo was taken in the early to mid-20th century.

Created by gemini-2.0-flash on 2025-06-16

Here is a description of the image:

The black and white photograph features a family on a porch, divided roughly in half by a pillar. On the left side of the pillar, a man sits in a rocking chair with his elbow resting on the armrest and his hand supporting his chin. Next to him, a woman stands with her hands on her hips. In front of her, a small child is seated in a rocking chair.

On the right side of the pillar, a woman sits in a chair holding a baby in her lap. Both women wear dresses, and the baby has a pacifier in its mouth.

The porch is part of a brick building, and there are windows visible on either side of the porch. Some graffiti is visible on the brick wall behind the woman holding the baby.

Mistral Pixtral

Created by us.mistral.pixtral-large-2502-v1:0 on 2025-06-01

The black-and-white photograph depicts a group of four people on the porch of a house. The house appears to be made of wood and brick, with some visible wear and tear.

From left to right:

  1. A man is seated in a rocking chair. He is wearing a long-sleeved shirt, trousers, and boots. He has a relaxed posture, with one leg crossed over the other.
  2. A woman stands beside the man, with one hand on her hip and the other resting on the back of the chair. She is wearing a dress with a pattern and a belt around her waist.
  3. A child is seated on the rocking chair next to the woman. The child is wearing a light-colored outfit and appears to be looking off to the side.
  4. Another woman is seated on the right side of the porch, holding a baby in her lap. She is wearing a dress and sandals. The baby is dressed in a light-colored outfit and appears to be looking in the direction of the camera.

The porch has a railing, and the background includes a door and windows with curtains. The overall setting suggests a casual, possibly rural environment, and the photograph seems to capture a candid moment of family life.

Qwen

No captions written

Text analysis

Amazon

PM

Google

E.C
E.C