GPT Image 2
Leads worldwide at 1,022, from 331,112 blind votes.
Click any country to explore its preferences
Bradley–Terry scores adjusted for presentation order. Higher is better; 1,000 is the population baseline.
Leads worldwide at 1,022, from 331,112 blind votes.
Scores come from blind A/B choices by people; 1,000 is the population baseline. Presentation order alone is worth +10 points in this modality, so every score is adjusted for it.
A demographic view appears only after it reaches at least 150 votes and 25 participants.
Our results show that different populations can rank the same models differently. Aggregate results broadly align with established human-preference evaluations, but population views reveal differences hidden by a single global ranking. Retaining anonymized information about who provided each judgment lets us examine variation across geography, profession, age, gender, and other characteristics.
We present our human-preference evaluations of generative AI models. The dataset includes more than 450,000 blind pairwise comparisons from participants across countries, professions, and demographic groups, after quality filtering.
Human preference is not a single global quantity. A pooled leaderboard reflects the particular distribution of people, prompts, and contexts that generated its votes. It does not imply that the same ordering will hold for every population or deployment.
Real AI systems are rarely deployed to an average user. A company may care about customers in a particular market, a professional tool about people with specific expertise, or a public institution about the population it serves.
Participants compare two model outputs without seeing which models produced them. We estimate relative strength with a Bradley–Terry model and include a position term to account for systematic differences associated with where an output appears.
Worldwide confidence intervals come from a bootstrap that resamples participants rather than individual votes, so heavy voters do not make results look more certain than they are. Population-specific rankings are point estimates from the same model fitted within each group.
Before estimation we remove 248,734 votes from participants whose voting pattern did not reflect the content shown, such as choosing the same side almost every time, and traffic from sources that failed our quality checks.
Most evaluation asks which model is best. Deployment decisions are narrower: which model is best for the users and task we care about? The answer can change with market, language, age, profession, and modality.
Preference is one dimension of quality. Accuracy, safety, latency, cost, reliability, and policy requirements can be equally important.
Model selection should be a conditional query, not a global number.
The goal is not only to identify the model with the highest aggregate score. It is to understand which models perform best for the specific populations and use cases an organization actually serves.
We want to map where different AI models work best across combinations of population, language, task, domain, market, modality, and deployment conditions. These results reflect the participants represented in the dataset and should not be read as census-representative estimates of entire countries.
We welcome collaboration with AI labs, researchers, companies, and public institutions on model evaluation, population-specific studies, additional modalities, and new model releases.
Collaborate with us