Human Generated Data

Title

Colonel Ernst August von Hugo and Lt. Colonel von Schlepegrell

Date

1787

People

Artist: John Singleton Copley, American 1738 - 1815

Classification

Paintings

Credit Line

Harvard Art Museums/Fogg Museum, Gift of Mrs. Gordon Dexter, 1942.180

Human Generated Data

Title

Colonel Ernst August von Hugo and Lt. Colonel von Schlepegrell

People

Artist: John Singleton Copley, American 1738 - 1815

Date

1787

Classification

Paintings

Credit Line

Harvard Art Museums/Fogg Museum, Gift of Mrs. Gordon Dexter, 1942.180

Machine Generated Data

Tags

Amazon
created on 2019-04-08

Art 96.4
Painting 96.4
Person 94.1
Human 94.1
Apparel 82.1
Clothing 82.1
Hat 62.4
Home Decor 55.7
Linen 55.7
Photography 55.6
Photo 55.6
Face 55.6
Portrait 55.6

Clarifai
created on 2018-03-16

people 99.9
adult 99.2
lid 98.5
two 98.2
portrait 97.1
man 96.5
one 96
veil 95.4
wear 95.2
woman 93.8
painting 93.5
three 92.2
facial expression 89.9
group 87.4
art 86.9
son 86.8
jacket 85.3
child 84.4
famous 82.8
retro 80.1

Imagga
created on 2018-03-16

hat 68.5
portrait 33
person 30.1
headdress 28
face 27.7
cowboy hat 27.3
man 24.2
clothing 23.3
model 22.5
people 21.7
hair 19
fashion 18.1
happy 17.5
sombrero 17.5
male 17
attractive 16.1
looking 16
pretty 15.4
old 15.3
smile 15
adult 14.9
senior 14.1
lady 13.8
smiling 13.7
sexy 13.6
black 13.2
cute 12.9
make 12.7
covering 12.7
posing 12.4
eyes 12
costume 12
style 11.9
head 11.8
fun 11.2
outdoors 11.2
consumer goods 10.7
retro 10.6
vintage 9.9
eye 9.8
mysterious 9.7
close 9.7
beard 9.6
art 9.6
mystery 9.6
brunette 9.6
culture 9.4
expression 9.4
sensual 9.1
human 9
currency 9
cowboy 8.8
look 8.8
straw 8.7
hairstyle 8.6
wearing 8.6
makeup 8.2
one 8.2
gorgeous 8.2
dress 8.1
handsome 8
romantic 8
happiness 7.8
western 7.7
elegant 7.7
elderly 7.7
money 7.7
guy 7.6
casual 7.6
skin 7.6
hand 7.6
shirt 7.5
lips 7.4
nice 7.3
girls 7.3

Google
created on 2018-03-16

painting 86.9
portrait 79.8
art 73.8
headgear 61.9
gentleman 59.7
modern art 55.4
artwork 53.9
hat 51.7
illustration 50.3

Microsoft
created on 2018-03-16

person 97.2
old 72.9

Color Analysis

Face analysis

Amazon

Microsoft

Google

AWS Rekognition

Age 49-69
Gender Male, 71%
Angry 1.2%
Surprised 1.8%
Happy 1.3%
Sad 1.1%
Calm 90.4%
Disgusted 3.6%
Confused 0.7%

AWS Rekognition

Age 23-38
Gender Male, 94.9%
Confused 3.6%
Sad 12%
Surprised 3.3%
Calm 70.8%
Happy 4.1%
Disgusted 2.7%
Angry 3.4%

AWS Rekognition

Age 26-43
Gender Female, 97%
Sad 38.4%
Disgusted 0.9%
Angry 2.7%
Happy 2.9%
Surprised 3.1%
Calm 50%
Confused 2.1%

Microsoft Cognitive Services

Age 71
Gender Male

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Possible
Blurred Very unlikely

Google Vision

Surprise Very unlikely
Anger Very unlikely
Sorrow Very unlikely
Joy Very unlikely
Headwear Very unlikely
Blurred Very unlikely

Feature analysis

Amazon

Painting
Person
Painting 96.4%
Person 94.1%

Captions

OpenAI GPT

Created by gpt-4 on 2024-01-27

The image showcases a figure dressed in a red coat with gold-colored buttons and a black high-collared clothing piece underneath. The attire includes a white fabric, suggesting a scarf or cravat, draped around the neck and tucked into the coat. The subject appears to be of high rank or status due to the ornate clothing. To the figure's left, there's a hint of a drawn or etched profile of another individual with pronounced facial features and wearing a hat. The background is minimal, with mostly muted and unfinished brushstrokes or sketches. The artwork gives an impression of being partially complete, blending detailed areas with sections that appear to be in progress.

Anthropic Claude

Created by claude-3-haiku-48k-20240307 on 2024-03-29

The image depicts two elderly men wearing military uniforms with ornate hats and capes. They appear to be in a contemplative or thoughtful pose, with one man looking slightly upwards and the other man gazing ahead intently. The painting has a warm, reddish-brown color scheme, and the background is slightly blurred, drawing the viewer's focus to the detailed facial features and expressions of the two men.

Meta Llama

Created by us.meta.llama3-2-90b-instruct-v1:0 on 2024-12-26

This image is a painting of two men in military uniforms, with one man wearing a black hat and the other wearing a red coat. The painting appears to be unfinished, with visible brushstrokes and areas of bare canvas. The man on the left wears a black hat with a white feather tucked into the band, and a red coat with gold buttons and a white collar. He has a serious expression on his face and looks to the right. The man on the right wears a red coat with gold buttons and a white collar, and a black hat with a white feather tucked into the band. He has a serious expression on his face and looks up and to the right. In the background, there are some faint outlines of other figures, but they are not fully developed. The overall effect of the painting is one of grandeur and dignity, with the two men depicted in a formal and dignified manner.

Created by us.meta.llama3-2-11b-instruct-v1:0 on 2024-11-24

The image is a painting of two men in military uniforms, with one man's face partially obscured by a hat. The man on the left wears a black hat with a white band and a red coat with gold buttons and a white collar. He has white hair and looks to the right. The man on the right wears a black hat with a white band and a red coat with gold buttons and a white collar. He has white hair and looks up. The background is a beige color with brown and black splotches. The overall atmosphere of the painting is one of dignity and respect, as the two men are depicted in formal military attire. The use of muted colors and the serious expressions on their faces add to the sense of solemnity and gravity.

Amazon Nova

Created by amazon.nova-pro-v1:0 on 2025-01-04

The image is a painting of two men in military uniforms, likely from the 18th century. The men are depicted in profile, facing the same direction. They are wearing red coats with white collars and black buttons. They are also wearing black hats with gold trim. The man on the left has a white beard and mustache, while the man on the right has a white beard. The background of the painting is blurry and appears to be a cloudy sky.

Created by amazon.nova-lite-v1:0 on 2025-01-04

The image is a painting of two men dressed in military uniforms. The man on the left has a serious expression and is wearing a black hat with a silver buckle. The man on the right has a more relaxed expression and is wearing a red coat with a white collar and a black bow tie. Both men are looking up, possibly at something or someone above them. The background of the painting is a dark brown color, and there are some white lines and shapes that appear to be part of the painting's composition.