How AI Guesses Ethnicity From a Face: Eigenfaces, Landmarks, and Neural Networks

The Prehistory: Galton's Photographic Plates

Machine face-reading begins with a Victorian polymath and a stack of portraits. In the late 1870s, Francis Galton started exposing multiple portraits — criminals, soldiers, the residents of a particular asylum — onto a single photographic plate, each for a fraction of the normal exposure time. The result was a 'composite portrait': a soft, ghostly face in which individual peculiarities canceled out and, Galton believed, the underlying type emerged.

Galton was chasing essences, and they were the essences of his era; the project was frankly typological and ultimately eugenic. But the mechanical idea underneath was durable: averaging many faces reveals what they share and erases what they do not. That idea is the direct ancestor of the composite faces in this site's quiz — and, less obviously, of the algorithms that turned faces into numbers.

Eigenfaces: A Face as a Weighted Sum

The first real algorithm arrived in 1991, when Matthew Turk and Alex Pentland at MIT published 'Eigenfaces for Recognition.' Take a large set of aligned face images, apply principal component analysis, and the leading components — ghostly, face-like basis images they named eigenfaces — define a coordinate space. Any new face can then be approximated as a weighted sum of those basis faces: a short list of numbers instead of a picture.

Recognition becomes geometry. To decide whether two photographs show the same person, you compare coordinates; to search a database, you look for the nearest neighbor. It was crude by modern standards and revolutionary in effect: for the first time, a face was formally a point in a space, and everything since has been a refinement of that move. Every face system in production today, from phone unlock to airport gates, is a descendant of that diagram.

The Modern Pipeline

Contemporary systems follow the same skeleton with far stronger muscles, in four steps:

• Detection: find the face in the image and crop it. What once required hand-engineered features is now a deep network in its own right.
• Alignment: locate landmarks — pupils, nose tip, mouth corners — and warp the face onto a common grid. This is precisely the alignment step that composite photography depends on; Galton managed it with an adjustable camera mount.
• Feature extraction: a deep convolutional network maps the aligned face to an embedding — a vector of hundreds of numbers — trained so that photographs of the same person land close together. The deep embedding systems of the mid-2010s, like Facebook's DeepFace and Google's FaceNet, made these representations dramatically powerful.
• Comparison: whatever the task — verifying a match, identifying a person, or estimating a category — happens in embedding space, by nearest-neighbor lookup or a small classifier trained on top.

What an 'Ethnicity Estimate' Actually Is

Strip away the marketing and an ethnicity estimate is a statement of this form: of the categories this system was trained on, this image most resembles the training images labeled X. It is a pattern matcher reporting on its own training distribution — not an ancestry reader. The labels themselves are conventions, chosen by dataset builders and carrying the demographics and assumptions of whoever assembled the data.

That is why the output is a probability over buckets, not a fact about a person. If the training set underrepresents some region, faces from that region get confidently sorted into whichever adjacent bucket is better stocked. The model does not know the world; it knows the sample. When the sample is skewed, the skew is reported back as truth.

The Bias Is the Dataset

Every limitation of the data becomes a learned behavior of the model: lighting conventions, camera hardware, pose norms, demographic skews, labeling habits. A system trained on passport photographs from one country and era will underperform on casual selfies from another — not because faces changed, but because photograph-making changed. The audits that exist are not reassuring: the well-known Gender Shades study of 2018 found commercial face-analysis systems whose error rates for darker-skinned women were dozens of times higher than for lighter-skinned men, a distortion inherited entirely from the training data.

The uncomfortable conclusion is that this is structural, not a bug to be patched. A model's universe is its dataset, nothing more. Galton's composites had the same limitation — his 'criminal type' was an average of whoever he had photographed — which is why the history of this technology doubles as a history of its biases.

Where This Site Draws the Line

There is also a purely statistical reason to refuse individual classification: group-level correlations say nothing reliable about an individual. On average, populations from colder regions have narrower noses; a particular person's nose is evidence of almost nothing, because within-group variation dwarfs between-group differences. Applying population statements to a single case is a textbook statistical error — the ecological fallacy — and it is committed every time a selfie is sorted into an ancestry bucket.

That is why Ethnicity Guesser deliberately does not do this. The faces here are historical composites, and they are quiz clues — never subjects of analysis, never mirrors held up to a user. There are no photo uploads, and there never will be. It is the same line the site draws when it says a phenotype is not a race: population-level appearance patterns are real enough to be learnable as geography, and exactly the wrong instrument for sorting individuals.

Frequently Asked Questions

Can AI recognize faces better than humans?

On verification benchmarks — deciding whether two photographs show the same person — modern systems can outperform typical human observers. But recognizing identity and classifying ethnicity are different tasks. The first compares a face to other faces; the second compares a face to a training set's labels, inheriting every skew that set contains.

What are eigenfaces, in plain language?

Ghostly, average-like images produced by principal component analysis over a set of aligned faces. Any individual face can be approximated as a weighted blend of them, which reduces a photograph to a short list of numbers — coordinates in a 'face space.'

Does this site use AI to analyze my face?

No. The site never accepts photo uploads and never analyzes any visitor's image. Composite faces appear as quiz questions about historical geography; whatever analysis happens stays in your head.

Related Phenotypes

Faces from the encyclopedia that appear in this article. Open any entry for its full description, distribution, and references.

• Amur-Sakhalin

  Siberia

  Distinctive Tungid variety, probably split from neighbouring types at the end of the Paleolithic. Native to parts of the Sakhalin-Amur regio...

• Baykal

  Siberia

  Northern Tungid variety, native to the vast, sparsely populated Taiga and Tundra regions of East Siberia, adapted to survive in the coldest ...