Based on historical distributions from ~1500 years ago. Estimates appearance, not DNA ancestry or personal identity.

AI & TechnologyNovember 19, 20267 min read

AI Race Detector: Built to Audit Models, Sold to Label Faces

The race classifier inside face tools was built to audit other systems in batches, where errors cancel; pointed at one person, they no longer do.

Illustration of the Chukiangid entry, a South Sinid variety recorded from the subtropical forests of southern China

Two pipelines share one classifier

The weights do not change between an audit and a consumer product. What changes is where the label lands: on a batch of a thousand images whose proportions get read, or on one photograph whose owner gets a name. A model cannot tell the difference, because the difference is not in the model. It sits in the question the pipeline asks. Asked for a share of a sample, an ordinary classifier is a measuring instrument with a stated error rate. Asked for a category for this person, the same classifier is answering something its training set never asked it to answer.

That split explains why the component exists at all. You cannot measure whether a face recognition system treats groups evenly without knowing which faces belong to which group, and those labels have to come from somewhere. The demographic evaluations published by the National Institute of Standards and Technology, which reported false positive rates differing by factors of ten and more across groups in one-to-many searches, needed exactly this metadata before any differential could be reported. The classifier is the ruler that measures the other system.

Why an audit needs a race label

The audit question is narrow. Does this system fail more often for one partition of the test set than for another? Answering it requires a partition, and the only partition available in bulk is the label. Used this way, the label describes the behaviour of the system rather than the person photographed. If a vendor's matcher is twice as likely to return a false match for one group, that sentence is about the vendor, and the label is the denominator of the sentence. It can be coarse, contested, and still do that job.

FairFace, assembled by Kärkkäinen and Joo and first posted in 2019, is the clearest published case of a collection built for that purpose. Public face sets skewed toward one group, so the authors sampled for balance and reported that training on the balanced set made accuracy more consistent across groups. The repair was in the sampling. The vocabulary was left as it stood, a handful of names chosen by two researchers for one paper.

The arithmetic that separates a batch from a person

Take a classifier that is right on seventy per cent of faces, with errors falling roughly independently. Over a thousand faces the estimated share of any one group carries a standard error near 1.5 percentage points, because the variance of a proportion is p times one minus p over n, and 0.7 times 0.3 over a thousand, square-rooted, comes to 0.0145. That is a usable figure for auditing a system. The same seventy per cent applied to a single face is a three in ten chance of a wrong answer, with no interval printed anywhere on the screen.

Independence is the weak assumption, and it fails in the direction you would expect. Errors cluster by lighting, pose, lens and compression, so a real test set has an effective sample size well below its file count. That makes an audit less precise than the binomial arithmetic suggests. It does not convert a batch statistic into a fact about an individual, because clustering widens an interval while a single call has no interval to widen.

Where a short alphabet loses contact types

The output alphabet is short and the record is not. Fijid is described as a Polynesid-Melanesid contact type, dominant in Fiji, with both elements mixed in that zone for several millennia. No slot in a short list corresponds to a contact type, because a contact type is defined by the meeting rather than by either side of it. Chukiangid, named for the Chu Khang river of southern China, is recorded as dispersed across Indochina and the Sunda Islands by trade and colonisation, which places one entry across a boundary that a short alphabet draws only once.

Central Ethiopid, in its purest form among Oromos, sits where the Horn meets the Sahara, astride the line between two large boxes in any short taxonomy. None of this is a defect in the classifier. It is the cost of a list short enough to be usable, and the catalogue pays the opposite cost: 209 entries, several of them defined by what they sit next to rather than by an extreme of their own.

What a documented entry carries instead

Each of those entries arrives with a paper trail. Chukiangid traces to Liu C. H.'s 1937 classification of the races of China in Zeitschrift für Rassenkunde, to Eickstedt's fieldwork in South and East Asia between 1938 and 1940, and to a coordinate in Guangdong. Fijid rests on Howells and Moss's 1933 anthropometry of Fiji and the Solomon Islands, printed in the American Museum papers, and on Gabel's 1958 study of Fijian measurements. Central Ethiopid goes back to Eickstedt's 1934 handbook and Baumann's 1952 survey of African variation.

A reader can find those papers, read the measurement, disagree with the author, and check the coordinate against a map. A score offers none of that: a name, a number, and often no published statement of what the names were. The distinction is not accuracy. It is whether a claim can be traced to something specific enough to be wrong in a locatable way, instead of merely being wrong.

The alphabet ages faster than the faces

The modern complication is a matter of timing. A label list written in one decade describes a population whose parents may have been born on three continents, and the boxes do not move while the people do. The catalogue has the same defect pointing the other way. Its baseline sits roughly 1500 years ago, and several entries admit it in their own text, noting Nilotid admixture among the western Ethiopian tribes or describing an entire type as the product of contact.

Two dated records, failing in opposite directions. The classifier is too recent to see a long history, and the catalogue is too old to describe a present face. Both remain honest while they print their date, and both turn misleading the moment the date is dropped and the label is presented as a property of the person in front of the camera.

Questions that survive a short alphabet

If the goal is to know whether a system treats groups evenly, use an aggregate, publish the interval, and name the test set that produced it. If the goal is a description of a documented pattern, read an entry and its references, and accept that the entry describes a historical sample rather than a living person. If the goal is your own ancestry, use a genotype panel with a published reference set, because no photograph carries that information.

None of these returns a race. A classifier asked for one is being asked to repeat a category it was handed, and repeating it with more confidence does not make that category older than the dataset that wrote it down. The smaller and more useful request is to say which list was used, when it was written, and what the returned figure is a share of.

Frequently Asked Questions

Can an AI detector tell what race someone is from a photo?
It can return one name from a list it was trained on, and the list is the limit. A model taught seven groups can only ever answer with one of those seven, so an entry that describes a contact type between two of them has no correct reply available. The output is also a ranking over that list rather than a measurement, and consumer tools rarely publish the list or the threshold. What you get is the closest match among labels someone else chose, presented as if it were a property of the person.
Why do race classifiers exist if they are unreliable for one person?
Because their main industrial use is auditing other systems rather than labelling people. To show that a face recognition system fails more often for one group, you need demographic labels across a test set, and a classifier can supply them at the scale a test set requires. Errors that would be decisive for a single face partly cancel across a thousand, so the group proportion stays usable. The same component pointed at one photograph loses that averaging and keeps only the error.
What would a fair test of a race detector look like?
It would need four published items: the label list, the annotation instructions, a held-out test set that resembles real photographs, and a threshold chosen before the test ran. Without the threshold there is no precision or recall, only a claim. Without the instructions the labels cannot be reproduced. And a fair test would have to include faces that belong to none of the listed groups, since a detector that has never been shown an out-of-list face has no measured false positive rate, which is the number that matters when the tool is pointed at a stranger.

Related Phenotypes

Faces from the encyclopedia that appear in this article. Open any entry for its full description, distribution, and references.

  • Chukiangid

    East Asia

    South Sinid proper subvariety, named after the Chu Khang (Xi) river in South China. Developed in the fishing and gatherer populations of sou...

  • Fijid

    Melanesia

    Polynesid-Melanesid contact type. The dominant type in the population of the Fiji Islands. Both elements have probably mixed in the zone for...

  • Central Ethiopid

    East Africa

    Ethiopid subtype of the central Ethiopian highlands and their surroundings. Found in its purest form among Oromos. Also in other Cushitic sp...

Test what you learned

Put your eye for regional faces to work in the daily quiz — a composite face, a world map, and your best guess.