Human Generated Data

Title

Social Settlements: United States. New York. New York City. "Jacob A. Riis Neighborhood Settlement": The Jacob A. Riis Neighborhood Settlement, New York City.: Fresh Air Housed at Twin Island. Where mothers and children are sent for a stay of two weeks.

Date

c. 1903

People

Artist: Unidentified Artist,

Classification

Photographs

Credit Line

Harvard Art Museums/Fogg Museum, Transfer from the Carpenter Center for the Visual Arts, Social Museum Collection, 3.2002.3194.2

Human Generated Data

Title

Social Settlements: United States. New York. New York City. "Jacob A. Riis Neighborhood Settlement": The Jacob A. Riis Neighborhood Settlement, New York City.: Fresh Air Housed at Twin Island. Where mothers and children are sent for a stay of two weeks.

People

Artist: Unidentified Artist,

Date

c. 1903

Classification

Photographs

Credit Line

Harvard Art Museums/Fogg Museum, Transfer from the Carpenter Center for the Visual Arts, Social Museum Collection, 3.2002.3194.2

Machine Generated Data

Tags

Amazon
created on 2019-06-06

Nature 99
Building 98.1
Countryside 98.1
Rural 98.1
Shelter 98.1
Outdoors 98.1
Plant 79.8
Vegetation 79.8
Land 75.3
Urban 72.6
Housing 71.9
Soil 71.8
Human 71.7
Architecture 66.3
Automobile 66.1
Vehicle 66.1
Transportation 66.1
Tower 63.9
Tree 62
People 61.9
Ocean 58.8
Sea 58.8
Water 58.8
House 56.5
Hut 55.9
Villa 55.1

Clarifai
created on 2019-06-06

people 98.6
home 98.5
no person 96.7
house 95.6
military 95.2
group 94.6
tree 94.6
building 94.1
war 93.5
architecture 92.9
landscape 92.6
vehicle 92.5
art 91.2
farmhouse 91.1
town 91
print 90.6
cavalry 90.3
religion 87.5
skirmish 87.2
old 85.7

Imagga
created on 2019-06-06

grave 44.7
fortress 40
ancient 38.2
architecture 36.1
stone 35.4
building 31.9
travel 30.4
history 27.8
old 25.8
sky 25.7
tourism 25.6
wall 25.5
landscape 25.4
landmark 24.4
brick 24
rampart 23.2
ruins 22.4
tower 21.7
roof 20.6
temple 20.1
house 19.3
historic 19.3
castle 18.5
rock 18.3
construction 18
monument 17.8
thatch 17.3
desert 16.9
tourist 16.5
church 15.8
village 15.4
medieval 15.4
historical 15.1
building material 14.6
scenery 14.5
religion 14.4
rural 14.1
pyramid 13.8
clouds 13.6
structure 13.2
mountain 12.7
monastery 12.5
stones 12.3
hill 12.2
culture 12
pharaoh 11.9
archeology 11.8
city 11.7
protective covering 11.6
summer 11.6
great 11.5
scenic 11.4
famous 11.2
town 11.2
vacation 10.7
cemetery 10.4
grass 10.3
sand 10
trees 9.8
outdoors 9.7
fortification 9.2
countryside 9.2
tomb 8.9
ruin 8.8
antique 8.7
rocks 8.5
art 8.5
destination 8.4
traditional 8.3
exterior 8.3
river 8
civilization 7.9
center 7.9
religious residence 7.7
middle 7.7
covering 7.7
sculpture 7.7
religious 7.5
palace 7.5
east 7.5
coast 7.2
farm 7.2
country 7

Google
created on 2019-06-06

Microsoft
created on 2019-06-06

outdoor 99.7
house 96.3
sky 85.4
tree 85.1
old 84.1
white 68.7
building 57

Color Analysis

Feature analysis

Amazon

Person
Car

Clarifai

No features identified

AWS Rekognition

Person 71.7%

AWS Rekognition

Person 63.4%

AWS Rekognition

Car 66.1%

Categories

Imagga
created on 2019-06-06

nature landscape 91.8%
streetview architecture 7.2%

Captions

Salesforce

Created by general-english-image-caption-blip on 2025-05-17

a photograph of a large house with a large stone building

OpenAI GPT

Created by gpt-4o-2024-11-20 on 2025-06-10

This image shows a large, rustic house situated within a picturesque rural setting. The building has a combination of stone and timber construction, with multiple roof peaks and a distinctive architectural style. A large tree stands nearby, partially shading part of the house. Two figures are visible on the grassy lawn in front of the house, suggesting human presence or activity. The foreground features uneven terrain with rocks and wild grass, contributing to the natural and peaceful ambiance of the scene. The photograph is old and sepia-toned, giving it a vintage appearance.

Created by gpt-4o-2024-08-06 on 2025-06-10

The image depicts a large, old house surrounded by trees and set on a slight hill above a rocky area. The house appears to be a multi-story structure with a pitched roof and several architectural details characteristic of turn-of-the-century homes. The exterior seems to be made of stone or brick. In front of the house, there are two people, dressed in period-style clothing, standing together near the base of the trees. The scene has a rustic, serene quality, and the photograph is in sepia tones, giving it an antique feel. The number "508" is written in the upper left corner of the image.

Anthropic Claude

Created by anthropic.claude-3-haiku-20240307-v1:0 on 2025-06-19

The image shows a large, multi-story building with a distinctive architectural style, surrounded by trees and set on a hillside. The building appears to be an old, historic structure, with a complex roofline and multiple chimneys. In the foreground, there is a rocky stream or creek flowing through the landscape. The overall scene has a vintage, sepia-toned quality, suggesting this is an old photograph.

Created by us.anthropic.claude-3-opus-20240229-v1:0 on 2025-06-19

The sepia-toned image depicts a large, stately house or mansion situated on a grassy hill. The house has multiple stories and appears to be made of stone or brick, with a pitched roof and several chimneys. There are trees surrounding the property, giving it a secluded, rural feel. In the foreground, there is an area of rocky terrain or boulders, contrasting with the manicured lawn directly in front of the house.

Created by us.anthropic.claude-3-5-sonnet-20241022-v2:0 on 2025-06-19

This is a historical sepia-toned photograph showing a large Victorian-style mansion or estate house situated on top of a rocky hill. The house appears to be made of stone and features multiple levels, dormer windows, and a complex roofline typical of Victorian architecture. The building is quite substantial in size and has several chimneys visible. The foreground shows a rocky outcropping or ledge, and there are trees visible around the property. The photograph has the soft, slightly faded quality characteristic of early photography, likely from the late 19th or early 20th century. There appears to be a small figure in light-colored clothing visible near the house, though the image quality makes details difficult to discern.

Meta Llama

Created by us.meta.llama3-2-11b-instruct-v1:0 on 2025-06-10

The image is a sepia-toned photograph of a large house with a stone foundation and a grassy yard.

  • The house is three stories tall, with a gable roof and multiple windows on each floor.
    • The house has a stone foundation and a wooden frame.
    • There are several chimneys visible on the roof.
  • In the foreground, there is a grassy yard with rocks and trees.
    • The grass is tall and unkempt.
    • There are several large rocks scattered throughout the yard.
    • There are several trees in the background, including a large oak tree to the left of the house.
  • In the background, there are more trees and a clear sky.
    • The sky is light gray and overcast.
    • There are several trees visible in the distance, including a large tree to the right of the house.

The overall atmosphere of the image is one of nostalgia and simplicity. The sepia tone gives the image a warm, vintage feel, and the lack of modern features such as cars or power lines suggests that the photo was taken many years ago. The image appears to be a family home, possibly from the early 20th century.

Created by us.meta.llama3-2-90b-instruct-v1:0 on 2025-06-10

The image depicts a large, two-story house with a stone foundation and a gable roof. The house features multiple windows on both floors, as well as a chimney. In the foreground, there is a grassy area with trees and rocks. Two people are standing in front of the house, one of whom appears to be wearing a white dress. The sky above the house is overcast.

The image has a sepia tone, suggesting that it may be an old photograph. There is some handwritten text in the top-left corner of the image, but it is not legible. Overall, the image presents a serene and peaceful scene, with the large house and surrounding landscape evoking a sense of tranquility.

Amazon Nova

Created by amazon.nova-lite-v1:0 on 2025-06-10

The image shows an old photograph of a large house with a sloping roof, surrounded by trees and grass. The house has a chimney and several windows, and it appears to be situated on a hill. In front of the house, there are two people standing, and there are rocks and a small stream of water in the foreground. The image has a vintage look, with a sepia tone and a slightly blurry quality.

Created by amazon.nova-pro-v1:0 on 2025-06-10

The image shows a black-and-white photograph of a large house with a porch and chimneys. The house is situated on a hill, surrounded by trees and grass. Two people are standing on the porch, and they seem to be looking at something. The house has many windows, and the roof is sloped. The image has a watermark with the number "508" on the top left corner.

Google Gemini

Created by gemini-2.0-flash on 2025-05-25

The photograph shows a large house, likely a mansion, set on a small hill with overgrown grass. In the foreground, there is a rocky area, possibly a stream bed or a decorative rock garden. The house has multiple levels and distinct architectural features, including a rounded bay window section and a prominent dormer window in the roof. There are trees partially obscuring the view of the house, one of which is quite large and centered in front of the main building. A few figures are standing near the house, but they are small and not clearly distinguishable. The image is in sepia tones, which suggests it is an old photograph. The overall impression is of an old, stately home in a rural or suburban setting. There is handwriting in the top-left corner.

Created by gemini-2.0-flash-lite on 2025-05-25

Here's a description of the image:

Overall Impression: The image is a sepia-toned photograph of a large, stately house, likely taken in the early 20th century. The composition is a landscape view, emphasizing the house's integration with its natural surroundings.

The House: The house is the central focus, a grand structure with several distinct features. It appears to be made of stone or brick. The main portion of the house has a gabled roof and a dormer, with multiple windows, suggesting a multi-story building. Attached to the main structure is a lower, more elongated section that likely encompasses a porch or additional rooms. There are several chimneys visible on the roofs.

Setting: The house is situated on a slight rise or hillside. A grassy area is in front of the house, leading down to what appears to be a rocky stream bed or a dry, rocky area. Mature trees are present on either side of the house and partially obscure the house. The general setting suggests a rural or suburban location.

People: Two people are visible, standing on the grass in front of the house. The fact that they are in the forefront helps establish scale.

Style & Tone: The sepia tone and the formality of the architecture point to a vintage aesthetic, indicative of photography from the early 1900s. The overall impression is one of prosperity, serenity, and a close connection between the house and nature.

Mistral Pixtral

Created by us.mistral.pixtral-large-2502-v1:0 on 2025-06-05

The image depicts a large, historic-looking house situated in a natural, somewhat rugged environment. The house appears to be constructed from stone and wood, with multiple stories and a complex roofline that includes several gables and dormers. The architectural style suggests it might be from the late 19th or early 20th century.

In front of the house, there is a person dressed in light-colored clothing, possibly a dress, standing near the entrance. The surrounding area is overgrown with grass and shrubs, and there are large rocks and boulders scattered around the foreground, indicating a rocky terrain. Trees are visible on either side of the house, adding to the rustic and secluded atmosphere of the setting. The sky is clear, suggesting it might be a sunny day. The overall scene conveys a sense of tranquility and isolation.

Qwen

Text analysis

Google

So
So