Human Generated Data

Title

Becky and Ian, Newton Center, Maine

Date

1980

People

Artist: Charles Melcher, American 20th century

Classification

Photographs

Credit Line

Harvard Art Museums/Fogg Museum, Transfer from the Carpenter Center for the Visual Arts, 2.2002.1429

Human Generated Data

Title

Becky and Ian, Newton Center, Maine

People

Artist: Charles Melcher, American 20th century

Date

1980

Classification

Photographs

Credit Line

Harvard Art Museums/Fogg Museum, Transfer from the Carpenter Center for the Visual Arts, 2.2002.1429

Machine Generated Data

Tags

Amazon
created on 2022-01-22

Human 99.8
Clothing 85.5
Apparel 85.5
Meal 67.4
Food 67.4
Finger 65.1
Plant 61.6
Kid 58.9
Child 58.9
Furniture 57

Clarifai
created on 2023-10-26

child 99.9
people 99.6
son 99.4
monochrome 99.3
boy 98.6
group 97
family 95.1
sibling 93.6
two 93.6
offspring 92.7
portrait 92.6
three 91
baby 90.7
group together 88.8
interaction 88
facial expression 86.7
wear 86.5
girl 85.9
recreation 84.6
adult 84.4

Imagga
created on 2022-01-22

child 36.7
people 28.4
person 24.5
male 24.1
happy 23.8
cute 23.7
boy 22.6
kid 22.1
family 19.6
man 19.5
portrait 19.4
adult 18.8
face 17.7
happiness 17.2
fun 17.2
childhood 17
smiling 16.6
youth 15.3
little 15
brother 14.3
women 14.2
joy 14.2
children 13.7
parent 13.4
play 12.9
adorable 12.9
love 12.6
black 12.6
attractive 12.6
couple 12.2
smile 12.1
human 12
mother 11.9
hair 11.9
casual 11.9
two 11.9
lifestyle 11.6
together 11.4
father 11.4
baby 11.4
playing 10.9
leisure 10.8
enjoying 10.4
education 10.4
school 10.1
looking 9.6
men 9.4
park 9.4
son 9.3
holding 9.1
fashion 9
outdoors 9
home 8.8
sitting 8.6
outside 8.6
pretty 8.4
photographer 8.2
cheerful 8.1
daughter 8
dad 7.9
blond 7.8
toddler 7.6
friends 7.5
one 7.5
care 7.4
device 7.3
handsome 7.1
work 7.1
working 7.1
day 7.1

Google
created on 2022-01-22

Microsoft
created on 2022-01-22

person 99.8
outdoor 98.7
text 98.6
black and white 97.3
clothing 93.8
toddler 86.8
human face 85.3
monochrome 79.5
baby 75
boy 73.4
street 62.3
crowd 1.3

Color Analysis

Face analysis

Amazon

Google

AWS Rekognition

Age 1-7
Gender Female, 92.8%
Confused 56%
Sad 36.1%
Angry 2.7%
Calm 2.5%
Disgusted 1.1%
Fear 0.8%
Surprised 0.5%
Happy 0.4%

AWS Rekognition

Age 0-6
Gender Female, 94.6%
Sad 96.6%
Calm 1.2%
Angry 0.7%
Fear 0.7%
Confused 0.4%
Disgusted 0.2%
Happy 0.1%
Surprised 0.1%

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Unlikely
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Feature analysis

Amazon

Person

Clarifai

Clothing
Human face
Tree
Footwear
Human arm
Human nose
Human hair
Human leg
Human mouth
Human eye

AWS Rekognition

Person 99.8%

AWS Rekognition

Person 99.5%

Clarifai

Clothing 96.5%

Clarifai

Clothing 89.5%

Clarifai

Clothing 55.4%

Clarifai

Human face 95.6%

Clarifai

Human face 84.7%

Clarifai

Tree 86.5%

Clarifai

Tree 75.2%

Clarifai

Tree 44.9%

Clarifai

Footwear 48%

Clarifai

Footwear 44.4%

Clarifai

Footwear 42.5%

Clarifai

Footwear 42%

Clarifai

Human arm 41.6%

Clarifai

Human arm 39.5%

Clarifai

Human nose 40.5%

Clarifai

Human hair 39.3%

Clarifai

Human leg 34.4%

Clarifai

Human mouth 33.5%

Clarifai

Human eye 33.4%

Categories

Imagga
created on 2022-01-22

people portraits 82.1%
paintings art 8.6%
beaches seaside 3.7%
nature landscape 2.9%
streetview architecture 1.4%

Captions

Clarifai

Created by general-english-image-caption-clip on 2025-07-13

person, left, and person, both, have their hands full with a pair of scissors.

Salesforce

Created by general-english-image-caption-blip on 2025-05-22

a photograph of a boy and girl sitting on a bench in a garden

OpenAI GPT

Created by gpt-4o-2024-11-20 on 2025-06-13

The image depicts two children sitting outdoors, surrounded by leafy branches. The child on the left is wearing a long-sleeve shirt and pants, holding some of the branches, while the child on the right is dressed in a dark sweater and appears to be playing or inspecting something with their hands. The background shows natural greenery and sunlight, providing a peaceful and playful outdoor setting.

Created by gpt-4o-2024-08-06 on 2025-06-13

The image shows two children sitting on the ground, surrounded by leaves and branches. The child on the left has curly hair and is wearing a long-sleeve shirt with a visible graphic, and they are holding leaves in their hands. The child on the right has straight hair and is wearing a dark long-sleeve shirt, and they are looking down at their hands while sitting next to the other child. There is a natural outdoor setting with trees visible in the background. The image is in black and white.

Anthropic Claude

Created by anthropic.claude-3-haiku-20240307-v1:0 on 2025-06-20

The image shows two young children, a boy and a girl, sitting together in a natural setting surrounded by foliage. The boy appears to be focused on something in his hands, while the girl is looking off to the side with a thoughtful expression. The black and white photograph captures a quiet, contemplative moment between the two children.

Created by us.anthropic.claude-3-5-sonnet-20241022-v2:0 on 2025-06-20

This is a black and white photograph showing two people sitting or reclining outdoors among some foliage or branches. One appears to be wearing glasses and has light curly hair, while the other is wearing dark clothing. The image has a casual, relaxed feel to it, with natural lighting filtering through what appears to be leaves or branches around them. The setting seems to be in a garden or outdoor space, and the composition has an intimate, candid quality to it.

Meta Llama

Created by us.meta.llama3-2-11b-instruct-v1:0 on 2025-06-10

The image is a black and white photograph of two children sitting in a garden. The child on the left has blonde, curly hair and is wearing a white t-shirt with a graphic design on it, dark pants, and dark shoes. They are sitting behind a bush and appear to be holding a leaf. The child on the right has dark hair and is wearing a dark sweater and dark pants. They are sitting in front of the bush and appear to be holding something in their hands.

The background of the image is blurry but appears to be a garden with trees and bushes. The overall atmosphere of the image is one of innocence and playfulness, as the two children seem to be enjoying each other's company in the garden.

Created by us.meta.llama3-2-90b-instruct-v1:0 on 2025-06-10

This image is a black and white photograph of two children sitting on the ground, surrounded by branches and leaves. The child on the left has blonde hair and is wearing a white shirt with a logo on it, along with dark pants. They are looking at the other child and touching a leaf with their right hand.

The child on the right has dark hair and is wearing a dark shirt and pants. They are looking down at something in their hands. In the background, there are trees and a path or road. The overall atmosphere of the image suggests that the children are playing or exploring in a natural setting.

Amazon Nova

Created by amazon.nova-lite-v1:0 on 2025-06-06

The black-and-white photo shows two children sitting on the ground, surrounded by leaves and branches. The girl on the left is wearing glasses and is holding a branch with leaves. The boy on the right is wearing a sweater and is holding a leaf in his hands. They are both looking down, perhaps examining the leaf or branch. The photo has a white border, and the background is blurry, with trees and a car parked in the distance.

Created by amazon.nova-pro-v1:0 on 2025-06-06

The black and white image shows two kids sitting on the ground. The kid on the left is wearing glasses and is holding a branch with leaves. The kid on the right is wearing a sweater and is holding something in his hands. The kids are surrounded by plants and trees. In the distance, there is a blurry image of a chair, and a car is parked.

Google Gemini

Created by gemini-2.0-flash-lite on 2025-05-17

Here's a description of the image:

This is a black and white photograph of two young children playing outdoors. They are both sitting amongst leafy bushes and branches.

  • Left Child: The child on the left is a little girl with curly hair, wearing large-framed glasses. She's wearing a light-colored long-sleeved shirt with a design on it and dark pants. She is reaching out with her hands, seemingly looking at leaves and branches.
  • Right Child: The child on the right is a boy with short, dark hair, dressed in a dark sweater and pants. He's looking down and focusing on his hands, possibly playing with something small.

The background shows foliage, indicating they are outside in a garden or park. The black and white tones create a nostalgic and intimate feeling. The overall impression is one of childhood exploration and quiet play.

Created by gemini-2.0-flash on 2025-05-17

Here is a description of the image:

This is a black and white photograph of two young children sitting amongst foliage. The child on the left has curly light-colored hair and is wearing glasses. They are wearing a light-colored sweater with a graphic design on the front. Their dark pants and shoes are visible. The child is holding onto a branch with large leaves.

The child on the right has dark, short hair with a blunt fringe. They are wearing a dark long-sleeved sweater and dark pants. The child is looking down and manipulating a small object in their hands. The foliage surrounds both children, with some greenery and what appears to be a bench visible in the background. The photo has a border around it.

Mistral Pixtral

Created by us.mistral.pixtral-large-2502-v1:0 on 2025-06-05

The image is a black and white photograph featuring two young children outdoors. They appear to be in a garden or park setting, surrounded by foliage and plants.

The child on the left is a girl with curly, light-colored hair. She is wearing glasses and a light-colored shirt with dark pants. She is holding onto a large leaf with both hands and looking down, seemingly focused on something in front of her.

The child on the right is a boy with short, dark hair. He is wearing a dark-colored sweater and pants. He is looking down and appears to be holding or examining something in his hands, possibly a small object or a part of the plant.

The background shows more greenery and some trees, suggesting a natural, outdoor environment. The children seem to be engaged in an activity related to the plants around them, possibly exploring or learning about nature.

Qwen

Created by Qwen/Qwen2.5-VL-7B-Instruct on 2025-07-05

The image is a black and white photograph of two children sitting on the ground amidst some foliage, possibly in a garden or park. The child on the left has wavy hair and is wearing glasses, while the child on the right has short hair and is wearing a dark long-sleeved shirt. Both children appear to be engaged in an activity, with the child on the left holding a branch and the child on the right focusing on something in their hands. The background shows more greenery and foliage, and the overall atmosphere of the image suggests a calm and natural setting.

Created by Qwen/Qwen2.5-VL-72B-Instruct on 2025-07-05

The black-and-white image shows two children sitting outdoors among fallen leaves and branches. The child on the left has light, curly hair and is wearing glasses and a light-colored sweatshirt. This child appears to be holding a leaf and looking at it closely. The child on the right has short, dark hair and is wearing a dark long-sleeve shirt. This child is focused on something in their hands, possibly a small object or leaf. The background includes more leaves, branches, and some trees, suggesting they are in a natural or garden setting. The overall mood of the image is calm and contemplative, capturing a moment of the children engaging with nature.