Reading the model’s mind

Detecting the gap between what a traveler needs and what an AI assistant provides.

When we talk to someone, we hear what they say. We never see everything happening in their mind. With AI, the answer on the screen also reveals only part of the picture. Beneath those words is a rich pattern of internal activity, and researchers are learning how to read it.

In 2024, Anthropic found a feature inside Claude associated with the Golden Gate Bridge. When researchers turned up its activation, the bridge started appearing in unrelated answers. Ask how to spend $10, and Claude suggested paying the bridge toll. Ask for a love story, and it wrote about a car longing to cross the bridge. An internal signal could be identified, amplified, and used to change the model’s behavior.

Golden Gate Claude: Turning up a feature changed the model’s answers. Conceptual illustration.

At Navan, we are exploring a practical possibility: could signals inside Ava help us spot when it is missing a traveler’s intent, and change course while there is still time to help?

A traveler tries to book a flight, but the booking button is disabled. Ava, Navan’s AI assistant, repeats the instruction to click on the button to book it. The traveler’s goal is clear, but Ava’s response is not helpful. Ava’s language model, its reasoning engine, brings together the conversation and its context to decide what to do next. We are exploring whether signals inside the model can help detect when issues in conversations unfold in real-time. A real-time indicator could give Ava a chance to try a better next step or bring in a person.

Looking inside an AI model, termed interpretability, is used to understand how it arrives at an answer. Advances in the study of a model’s internal workings are presenting opportunities to improve the intelligence of AI assistants such as Ava. Across the field, there is growing excitement about the potential to turn a deeper understanding of models into more reliable and useful AI systems.

At Navan, we are exploring this approach, and are putting it into practice to directly improve Ava. Our starting point is that the richest representation of the traveler’s wishes available to Ava resides inside the reasoning language model. Internal model signals can reflect the traveler’s goal. Looking inside the model gives us deeper observability of both the traveler’s intent and whether Ava is successfully fulfilling it. Broadly, this type of approach is termed actionable interpretability, and we are using it to provide intent telemetry: using that understanding to help Ava understand and act better.

Intent telemetry turns internal language model activations, what its neurons are computing, into measurements of how the conversation is progressing. It can flag a conversation that is not going well, where Ava is missing something about what the user conveyed or when an ask was not entirely clear.

The conversation without steering

Illustrative signals; exchange paraphrased from the video. The early warning appears before a live-agent request.

Here, the traveler wants an earlier flight. Ava finds an option and explains how to book it, but the booking button is disabled. The gap is clear: the traveler needs help completing the booking, while the system repeats offering a step that is not working.

Our intent telemetry interpretability tools use trained detectors that flag the growing frustration. Internal activity patterns also provide recognizable concepts, such as failing booking controls or an unresolved request. Together, those signals help the system identify what it may be missing.

The intent signal rises while the booking remains blocked, then eases after the system changes its response and resolves the need. Illustrative sequence.

The warning prompts the system to revisit what it would usually do. It brings more deliberate reasoning to its own thinking and the conversation, with context about the likely cause and what has already failed. In this illustrated flow, Ava recognizes the obstacle and completes the booking directly.

Detecting the gap early and rapidly gives the system a chance to promptly resolve it. Nonetheless, when Ava cannot meet the traveler’s need, or the traveler wants a person, the same understanding supports a faster transfer to a live agent.

The conversation with steering

Alternative continuation from the early warning: steering resolves the obstacle, and the traveler confirms the need is met.

At Navan, we are turning model understanding into a way to improve assistance while a conversation is still unfolding. When Ava’s internal signals suggest that its response is missing the traveler’s goal, intent telemetry gives the system a reason to slow down, understand the obstacle, and change course. Travelers should not have to repeat themselves until frustration builds. Ava should resolve the need sooner, or connect them promptly to a person who can.