LLMs Develop New Hiring Biases From Scratch, Princeton and Chicago Study Finds
Researchers show that large language models can invent social stereotypes in hiring scenarios even when no pre-existing bias exists—and at higher rates than humans.
What matters
- Princeton and University of Chicago researchers ran LLMs through a hiring game previously used with human participants.
- Candidates belonged to four fabricated ethnic groups (Tufa, Aima, Reku, Weki) and were equally likely to succeed in any role.
- Both humans and LLMs developed biases from feedback, but LLMs were more likely to form stereotypes than humans.
- Because the group names are invented, any bias that emerged was generated during the task, not inherited from training data.
- The finding suggests that debiasing training data alone may not prevent AI hiring tools from developing new discriminatory patterns.
What happened
Researchers from Princeton University and the University of Chicago ran a set of large language models through a hiring game that had previously been used with human participants. In the task, participants assign candidates to specific roles and then receive feedback on whether the hire was successful. All candidates were equally likely to succeed in any given job, but each belonged to one of four fabricated ethnic groups: the Tufa, Aima, Reku, or Weki.
When human participants played this game, the feedback they received led them to form biases against the made-up groups. For example, if a participant hired a Tufa as a doctor and got negative feedback, they became reluctant to hire another Tufa as a doctor again. Those biases persisted even after the game ended.
When the researchers had LLMs complete the same task, the models also developed stereotypes based on the feedback they received—and, according to the study, did so more readily than the human participants. The finding is notable because the fabricated group names carry no real-world cultural or historical associations, meaning any bias that emerged was generated entirely from the model's own learning during the task rather than inherited from training data.
Why it matters
AI is already widely used in hiring and recruitment, from résumé screening to interview analysis. A common defense of automated hiring tools has been that, if designed carefully, they can avoid the human prejudices that creep into manual review. This study challenges that assumption by suggesting that bias in AI hiring systems is not only an inheritance problem—models can also create new prejudices on their own through feedback loops.
The implication is that simply debiasing training data or removing known protected attributes may not be sufficient. If models can form stereotypes around arbitrary group labels after just a few rounds of feedback, then any deployment that involves iterative learning or reinforcement from outcomes could generate novel discriminatory patterns. That raises the stakes for auditing, monitoring, and human oversight in AI-assisted hiring pipelines.
The study also underscores a broader concern: as LLMs are increasingly embedded in decision-making workflows, their tendency to generalize from limited feedback could produce unintended consequences that are harder to detect because they involve categories or patterns that did not previously exist.
What to watch
- Whether regulators and standards bodies cite this study in upcoming guidance on AI in employment, particularly in jurisdictions with existing algorithmic accountability laws.
- How vendors of AI hiring tools respond—whether they add bias-monitoring for emergent stereotypes rather than only known protected classes.
- Follow-up research testing whether different model families, sizes, or prompting strategies reduce or amplify the tendency to form new biases.
- Whether enterprises using LLM-based screening begin requiring independent audits that specifically test for emergent, not just inherited, bias.
What to do next
Developers
Audit any LLM-based ranking or classification pipeline for emergent bias by testing with arbitrary group labels and measuring outcome disparities after feedback loops.
The study shows models can form new stereotypes from feedback alone, so standard debiasing of known attributes may miss novel discriminatory patterns.
Founders
If your product uses AI for hiring or candidate screening, commission an independent audit that specifically tests for emergent—not just inherited—bias.
Investors and enterprise customers will increasingly ask for evidence that your tool does not generate new discriminatory categories during use.
PMs
Add monitoring dashboards that track selection rates across arbitrary user-defined cohorts over time, not just protected classes.
Emergent bias can appear around labels you did not anticipate, so limiting monitoring to known demographic categories leaves blind spots.
Investors
Assess whether AI hiring startups in your portfolio have plans to test for emergent bias, and flag the regulatory and reputational risk if they do not.
This study provides ammunition for regulators and plaintiffs; companies without mitigation strategies face growing liability.
Operators
Where AI assists hiring decisions, keep a human reviewer in the loop and log the feedback the model receives so you can detect when it starts generalizing from limited outcomes.
The study's hiring game shows that even a few negative feedback events can trigger stereotype formation, so visibility into the feedback signal is critical.
Testing notes
Caveats
- This is a research finding, not a released product or tool. The study methodology could theoretically be replicated, but the full paper details are not available in the provided sources, and no code or dataset has been publicly linked.