Announcing Human Behavior Bench with Google DeepMind and Kaggle. See how Rehearsals reached 94.1% distribution agreement in the evaluation.Explore the benchmark
Get started

IntroducingHuman Behavior Bench

In collaboration with Google DeepMind and Kaggle, Rehearsals is building a large-scale benchmark of human behavior.

Rehearsals
Scroll to learn more
1,244U.S. adults
95,553Individual responses
1,199Evaluation tasks

Can AI predict what
people will choose?

By the Rehearsals and DeepMind team

AI knows how humans talk. Does it know how humans behave?

Language models are convincing conversationalists. They can reason through preferences, explain motives and answer from almost any point of view. But sounding human is not the same as representing humans accurately.

Ask different groups of 100 people the same question and you get wildly different answers: specific, uneven and often surprising. Ask an LLM to answer for each group and it spreads a smooth distribution across every reasonable option, or settles on whatever feels agreeable. Much of what makes people human gets trained out of AI models to keep them reasonable and useful.

Human Behavior Bench measures that gap: how accurately can AI simulate human responses?

The questions in this benchmark span health, finance, civic life, religion and everyday decision making, as well as foundational psychographic information. We show how this knowledge translates to concrete improvements in domains like pricing or product preference, and what not having this is costing companies every day.

Special thanks to the teams at Kaggle and Google DeepMind for their close collaboration in designing this benchmark and for sponsoring the compute behind it. You can see the real-time leaderboard on Kaggle.

How the benchmark
is built.

01

Ground truth

The ground truth data comes from 1,244 adults based in the US in Rehearsals’ first party panel surveyed across June to September 2026 via two survey modes: voice interviews and form surveys.

Learn about the Rehearsals panel
02

Audiences

People are grouped into audiences, which are defined by shared demographic information or behavior. Some cohorts use structural filters that are commonly found in census demographic stratification such as age, income, and geographic location. Other filters are behavioral, for example people who take GLP-1s, people who bought a car in the last 12 months, etc.

03

Evaluation tasks

Each task is a question asked to a specific cohort with associated ground truth. We selected a variety of questions across finance, health, recreation, values, and consumption behaviors. We specifically selected tasks where the ground truth data of a cohort differs from general population data, where base language models fail most.

Question

“Self-checkout fails to scan a $4 item and you notice at the exit. Realistically, you…”

Audience

35-44 year olds with household income over $150k

Ground truth

How that audience really answered, option by option.

Every model predicts this distribution.
Human responsesModel 1 · 92.9Model 2 · 68.1More models …

Go back and pay for it

47%
42%
21%

Tell an employee

17%
23%
11%

Keep walking

22%
23%
44%

Honestly, it depends on my day

14%
12%
24%
Each prediction system is scored on agreement with the real distribution: 100 × (1 − total variation distance).

Public and private tasks.

239 public960 private

In this benchmark, there are 1,199 tasks total, 239 tasks public, and 960 private (20/80 holdout, stratified by category). Every single question is closed-form, with a pre-defined set of options.

Anyone can see the public tasks, with their questions, audiences and answers.

What we are evaluating

Base LLMs are evaluated through a harness that runs in Kaggle. The harness takes in the case, prompts the model to generate a distribution over the option set given the audience, and then calculates the metric against the ground truth.

We also introduce a new way of predicting human distributions called Engram. Engram was evaluated through Rehearsals’ inhouse evaluation system that has the same case and metric set up as the Kaggle harness. However, instead of having a model elicit a distribution, we create a simulation with AI twins of real people and then tally their individual responses into a final distribution.

Performance.

V1.0 · United States · Fieldwork June–September 2026

Kaggle Methodology Limitations Cite this benchmark
Human Behavior Benchmark V1. Rehearsals uses its own evaluation harness; base models are evaluated on Kaggle. Higher agreement is better.
Prediction systemDistribution agreement0–100
01Rehearsals Engram 1.0
94.1
02Gemini 4 Argon
80.6
03GPT-5.6 Sol
79.5
04Claude Opus 5.5
79.3
05Gemini 3.8 Flash
79.1
06GPT-6 Astra
79.0
07Grok 4.6
76.2
Agreement = 100 × (1 − TVD). Higher is better.

Four findings.

LLMs are weakest on product and marketing decisions.

Teams do user research to decide what to build, what to charge and what to say. Those are the questions LLMs get wrong most often. When it comes to picking the top answer for an audience, LLMs get it right about four times in ten on product features, messaging and pricing, against seven in ten on brand perception and daily routines. Rehearsals Engram 1.0 gets the top answer right nine times in ten.

LLMs get the top answer right 41% of the time on product questions, and 74% on brand perception.

Highlight an LLM
Average of six LLMsGemini 4 ArgonRehearsals Engram 1.0
Six decisions that a team would get wrong with models today.

Each is a question a pricing, product or marketing team would act on. All six LLMs miss in similar ways.

1 / 6
Is our rural renewal base sticky?

Audience: adults who live in a rural area
Asked: When your car insurance last came up for renewal, you…

Highlight an LLM

Share who renewed without checking other prices

Human responses
24%
Rehearsals Engram 1.0
24%
Gemini 4 Argon
51%
All LLMs
50–56%

LLMs say about half of rural drivers renew on autopilot. Really, about a quarter do.

Share of tasks where the prediction ranks the audience’s real top answer first, averaged over the questions of each type. Bars: the average of the six LLMs.

LLMs predict people as apathetic.

On average, LLMs underpredict the most strongly felt option by 13 points. They also predict about 10 points too high on “never”, “none” and “don’t” answers.

LLMs mellow out strong feelings.

Highlight an LLM
Human responsesRehearsals Engram 1.0Gemini 4 ArgonOther LLMs
Four tasks where people feel strongly.
1 / 4
I wish I could get all the content I want in one place

Audience: household income $25k-$50k, men

Highlight an LLM

Share who answered “Strongly agree”

Human responses
70%
Rehearsals Engram 1.0
66%
Gemini 4 Argon
30%
All LLMs
23–31%
Four tasks where LLMs over predict “never” answers.

Across the 435 tasks that offer a “never”, “none” or “don’t” answer, people pick it 14% of the time. The LLMs predict 22–25%.

1 / 4
Outdoor activity sessions of an hour or more, last 30 days

Audience: aged 45-54, in the Northeast

Highlight an LLM

Share who answered “None”

Human responses
7%
Rehearsals Engram 1.0
4%
Gemini 4 Argon
34%
All LLMs
22–40%

LLMs don’t understand how age affects preference.

LLMs usually know which way an age group leans, but not by how much. For audiences under 60 they typically predict about three quarters of the real difference from other audiences. For audiences 60 and over they overshoot it. That also happens to be the age group they predict worst.

Highlight an LLM

One in five people over 65 skipped short-form video last week, but LLMs guess more than half.

Share who did not use short-form video (TikTok, Reels, Shorts) in the last 7 days

Ages 65+, earning $50-100kOther audiences (13)
Under 60, LLMs play the difference down. At 60 and over, they exaggerate it.

Each line is one LLM. At 100%, it gets the size of an age group’s difference from other audiences exactly right. The typical LLM lands at 73–78% for audiences under 60 and 114% for audiences 60 and over.

“Other audiences” averages every other audience asked the same question. The trend shows each LLM’s typical ratio of predicted to real difference. LLMs get the direction right for 63–85% of audiences.

LLMs are all the same species, and it’s still not human.

The six LLMs give answers that are about two times closer to each other than to the answers of real people. On about one in four tasks, all six LLMs mispredict the top option. Even Gemini 4 Argon, which has the best score, is the best on only about one task in four. The best LLM changes with the question type.

The LLMs are about twice as close to each other as to real people.

Highlight an LLM

How far apart the answers are, in points. Lower is better. The top bar compares two LLMs with each other. The rest compare each LLM with real people.

Distance is how far apart two sets of answer shares are, averaged over all tasks.

Why small errors
can add up.

ILLUSTRATIVE EXAMPLE
Human Prediction
3 percentage points
Perfect matchLarger mismatch
1 − MAE97/100

Average error per option

1 − TVD82/100

Agreement across the distribution

TVD adds up the error across all 12 answers. MAE averages it, which makes the same prediction look more accurate.

Score = 100 × (1 − ½ Σ |pᵢ − qᵢ|)

For normalized answer distributions, 100 means a perfect match.

Questions and limits.

What does V1 measure?

V1 compares predicted and observed answers for groups of US adults. It does not test predictions for individuals, consistency across one person’s answers, or performance in other countries. The tasks include audiences whose answers differ from the general population.

What counts as human behavior here?

The benchmark uses survey and interview responses about people’s behavior and choices in hypothetical situations. Matching these responses does not show that a model can predict future purchases or other actions.

How much do human samples vary?

A different sample of people can give different answers. The study reports a median of 176 people per task and examines this variation through resampling and split-half comparisons. A difference between model scores may reflect sampling noise; the scores alone do not establish statistical significance.

How is Rehearsals Engram 1.0 evaluated?

Kaggle evaluates the base models. Rehearsals used its own evaluation harness to run the same tasks.

Can anyone submit a model today?

Public submissions are planned for a later release. Kaggle will maintain the benchmark and add new models.

IN COLLABORATION WITH

Google DeepMindKaggle

Google DeepMind and Kaggle contributed benchmark design advice, independent base-model evaluations, and sponsored compute. Kaggle will maintain the benchmark as new models are released.

View on Kaggle

The roadmap ahead.

OCTOBER 2026

V1 · United States

Adults
1,244
Tasks
1,199
LATE OCT — EARLY NOV 2026

V1.5

Adults
3,500
Tasks
1,200
DEC 2026 — JAN 2027

V2 · Worldwide

  • United States
  • Canada
  • United Kingdom
  • France
  • Switzerland
  • Italy
  • Japan
  • Korea
  • Taiwan
  • Singapore

The model
in practice.

Understanding human decision-making is the core problem we work on at Rehearsals. Rehearsals is a self-serve simulation platform for predicting marketing outcomes. We build AI twins of real humans, each grounded in a series of multimodal, behavioral interviews so that marketing and product teams can see how their customers really think or act.

Explore the Rehearsals platform

Contribute
your story.

Share your experiences in a Rehearsals research interview. Help us build AI twins grounded in real people.

Take the interview

Benchmark your model.

View on Kaggle