New hires meet their hardest client before their first real one.

The problem

Before a new hire could face a client, they read the procedure docs, wrote tests on the company's practices, and sat through multiple rounds of mock interviews with a senior member of staff. The green light depended on senior people finding the hours, and every interviewer ran the interview a little differently.

The client's question to us was direct: could AI simplify this, and could it produce an honest measure of a person's strong and weak points instead of a gut feel.

What we built

A roleplay bot that plays the client, and not a polite one. It carries multiple personas, works from the company's own scoring rubric (we spent real time with their team turning that rubric into something a machine can apply), and runs a full practice interview.

Every session ends in a report: a readiness score, focus items, wins and fails, and flagged moments where the trainee fell short of the standard, or where they'd have delighted a real client.

If a trainee gets stuck, they can break character and ask the bot for guidance, then step back into the scene. Practice, with a coach standing just offstage.

The part worth noticing

In our other study, the right answer was no AI at all. Here it's the opposite, for one reason: no human can be ten different difficult clients, on demand, at 8am and 8pm, without ever getting tired or going easy on someone they like. This is work only AI can scale.

Two things made it actually work. First, we had to make the bot less helpful. Every AI's instinct is to be an eager assistant; a realistic client is nothing of the sort, and training against a pushover teaches nothing. Second, we test the bot like software: every change gets replayed against a standard set of interviews and judged against the rubric before it goes anywhere near a trainee. The bot runs on Google's stack; the tool we use to stress-test it is Anthropic's Claude. We use whatever does the job, which is easy when you resell nothing.

The results

Three months in, the bot is part of the company's QA process for new hires. It serves as the early warning that someone isn't ready to face clients yet and needs more training, and every candidate now gets measured against the same rubric instead of whichever interviewer had time that day.

A person still reviews every report, holds their own conversation with the candidate, and makes the final call. At current volume, the bot conducts what would otherwise be three to five hours of a senior person's interviewing every week; that figure is derived from session counts, not measured, so take it as arithmetic rather than a benchmark. It's early for catch-rate stories, and we don't publish what we can't stand behind.

What didn't go smoothly

A roleplay bot with a company's whole rulebook behind it runs long conversations, and long conversations plus a heavy rulebook is exactly where AI systems hit their memory limits. The first versions proved it. The fix took a few tries: we consolidated the bot's reference knowledge into NotebookLM, kept the standing instructions lean so the depth gets pulled in only when needed, and wired the whole flow into Google Workspace. None of this is exotic, but nobody's demo mentions it, and it's the difference between a bot that works in week one and one that still works in month three.

The pattern you can borrow

Use AI where scale is the point and humans keep the judgment. The bot does the hundred hours of patient roleplay nobody has; the human makes every readiness call. If your business has a "shadow a senior person until they say you're ready" process, this pattern applies to you, and a smaller version of it costs less than you think.