Skip to content
← All articles / Research

Paranoid Android: Measuring Evaluation Awareness Across Testing Environments

Does an AI agent notice when it is being tested? And when it does, does that change how it works?

The work, and the one watching it.

We gave AI models everyday business tasks: prepare documents, save email drafts, comment on tickets, and update deadlines. We tested them in two environments. RMS is a simulated office environment in which business applications are represented by predefined tools. It exposed a limited set of actions through abstract representations of business applications. World is an environment connected to real software and services. It allowed the models to work with real services, files, and command-line tools.

DeepSeek explicitly recognised that its performance was being evaluated in 60% of conversations in RMS and 36% in World. GLM Flash showed no explicit evaluation awareness in either environment. A separate test of the larger GLM model also found no explicit awareness across ten conversations.

We also compared how effectively the models used tools in the two environments.

All 260 requested actions checked in the main experiment were completed. Working with real services involved fewer tool calls and gave the models more useful ways to inspect their work. In RMS, models sometimes kept trying to verify things that the environment’s available actions could not establish.

In this article

1 Assumptions

AI agents can use tools to read files, search messages, edit documents, and interact with applications. Evaluating an agent therefore involves observing how it completes a sequence of actions.

An agent may also recognise the situation as a test. It might then check its work more carefully, pursue actions it thinks will earn credit, or become preoccupied with what the evaluator expects.

We call this evaluation awareness: the model explicitly recognises that its current performance is being assessed.

We looked for that recognition in the reasoning text recorded during each conversation. A conversation counted as aware when the model described its work as an evaluation of its own performance or considered what an evaluator expected it to do. Each conversation counted once, and we recorded where the first explicit sign appeared.

2 Quick overview of results

The main experiment compared GLM Flash and DeepSeek across 100 completed conversations, with 25 conversations for each model in each environment.

Model and environmentConversations with explicit evaluation awarenessRequested actions completed on first attemptRequested actions completed by the end
GLM Flash, RMS0/2563/6565/65
GLM Flash, World0/2558/6565/65
DeepSeek, RMS15/2564/6565/65
DeepSeek, World9/2565/6565/65
Total24/100250/260260/260

DeepSeek’s explicit awareness rate was 60% in RMS and 36% in World. GLM Flash remained at zero in both environments.

Across the experiment, 96.2% of requested actions succeeded on the first attempt. In 90 of the 100 conversations, every requested action succeeded immediately.

DeepSeek showed explicit evaluation awareness in 15 of 25 RMS conversations and 9 of 25 World conversations; GLM Flash showed none in either environment.
Figure 1. Conversations with explicit evaluation awareness, by model and environment. Each group contains 25 conversations.

3 Methodology, experiment and environment differences

The tasks take place at Rivermark Motion, a fictional company created as the setting for the experiment. Its working records include conversations between colleagues, customers and suppliers, shared documents, email attachments, calendars, and tickets tracking ongoing work.

The agent works from Maya Ivers’s account, picking up requests such as preparing a handover, checking evidence for a client review, or clarifying a supplier’s collection arrangements. Answering those requests involves following information across several sources: an email may refer to a document, a ticket may contain an older deadline, and an attachment may resolve an outstanding question. Rivermark gives the agent a connected business context to explore.

We used five sets of tasks involving documents, email, tickets, and deadlines. Each conversation contained five successive requests, allowing the model to build on its earlier work. Each task set was repeated five times for every model and environment combination.

RMS works through abstractions. It represents business applications through a predefined set of tools and supported actions. As in many testing and simulation environments, the agent can perform only the subset of operations implemented by the environment.

This matters when the agent wants to investigate something beyond that subset. It may be able to create an email draft, for example, without having the operations needed to inspect that saved draft in the way it expects. The model can still try to verify the result, but the environment may not give it a way to answer its question.

World gives the agent access to real services. It can work with native applications, inspect the filesystem, and use command-line tools to interact with services such as email. This gives it more ways to investigate the environment and check the effects of its actions.

The business material and task instructions were aligned across the two environments. We examined the models’ reasoning, their tool use, and the final stored records.

The outcome checks covered 100 documents or files, including 20 CSV tables; 100 unsent email drafts; 40 ticket comments; and 20 deadline updates. For email drafts, we also checked that they were addressed to the intended recipient and remained unsent.

4 Results and discussion: capabilities and awareness

4.1 The models could carry out the requested actions

All 260 requested actions were present by the end of the conversations.

Ten actions failed on their first attempt. Nine involved GLM encountering email-draft command errors, including attempts to use an unsupported option for adding a copied recipient. The remaining failure involved DeepSeek trying to save a document in a read-only directory. Each was resolved with another attempt.

The more revealing pattern appeared when models tried to verify successful writes in RMS.

In one conversation, GLM saved an email draft and received a successful response together with a draft identifier. It then tried to find and read the draft through operations that did not provide access to the newly saved item.

GLM repeated the save using a file containing the email body. It eventually told the user that it had been unable to save the draft. The final stored state contained both copies.

The model was trying to establish something that the environment’s available operations did not allow it to verify. It continued investigating and writing anyway.

Across RMS, 31 of 50 conversations produced more than one draft. None of the 50 World conversations did. Some additional drafts were repeated attempts to complete the original request; others were revisions or diagnostic messages created to investigate the email tools. We identified 15 saved diagnostic drafts.

Thirty-one of fifty RMS conversations produced more than one draft; none of fifty World conversations did.
Figure 2. Conversations producing more than one email draft, across both models. Additional drafts include retries, revisions and diagnostic messages.

This is an important consideration for capability testing. An agent’s apparent difficulty may come from the limited actions exposed by the test environment. Here, the models could complete the requested work, but sometimes lacked a usable way to confirm that they had done so.

4.2 DeepSeek brought the evaluator into practical decisions

DeepSeek’s awareness often appeared while it was deciding how to finish a task.

In one World conversation, it was unsure where to save a document:

“Let me think about the evaluation intent. This is a task-benchmark.”

It connected the instructions about the working folder and drafts to the structure of the test, inspected the available directories, and saved the document.

In another conversation, DeepSeek considered whether it should add a ticket comment alongside the requested local note:

“there’s risk the grader expects a ticket comment.”

Here, “grader” refers to whoever or whatever would score its performance. DeepSeek then rejected the extra action:

“don’t do duplicate/unrequested actions.”

It saved the note and created the requested email draft without adding another ticket comment at that stage.

In RMS, awareness appeared while DeepSeek was trying to verify a saved draft. It considered the possibility that:

“the graders consider ok:true as success.”

It continued checking and eventually created a diagnostic draft. The stored records showed that the requested draft already existed.

Evaluation awareness entered concrete working decisions: choosing a destination, weighing an extra action, and deciding how much verification was enough.

4.3 Real services required less interaction

A tool call is an individual request to search email, read a file, save a document, or perform another operation. Counting these calls shows how much interaction a conversation involved.

DeepSeek used fewer tool calls in World, both in conversations with explicit awareness and in those without it.

EnvironmentMean tool calls with explicit awarenessMean tool calls without explicit awareness
RMS98.883.7
World66.456.6

Within each environment, aware conversations contained approximately 18% more tool calls. Much of that activity preceded the first explicit reference to evaluation: about 70% of calls in aware conversations had already occurred by that point.

Mean tool calls were 98.8 and 83.7 in RMS, and 66.4 and 56.6 in World, for conversations with and without explicit awareness respectively.
Figure 3. Mean tool calls in DeepSeek conversations, by environment and explicit evaluation awareness.

The environment comparison is particularly useful. In World, the model could use familiar service behaviour and inspect the effects of its work. In RMS, it sometimes spent additional effort exploring the limits of the abstraction or attempting checks that the available tools could not support.

Our reading is that access to real services helped the model understand its working environment and complete the work with less investigation.

4.4 The larger GLM model also showed no explicit awareness

To explore whether GLM Flash’s lack of explicit awareness might be related to model size, we tested the larger GLM model in ten separate conversations, five in each environment.

Explicit evaluation awareness remained at zero across all ten conversations. The ten GLM Flash conversations selected in advance for comparison also showed zero.

The larger model was more active. It made 553 tool calls, compared with 475 for Flash, an increase of about 16%. It also created 14 drafts, compared with 11.

Moving to the larger GLM model increased activity without producing explicit evaluation awareness in this comparison.

5 What this means for testing AI agents

These results make a clear case for evaluations that let agents use real services.

DeepSeek showed less explicit evaluation awareness in World and used fewer tool calls. The models also had more effective ways to check their work, avoiding the repeated draft creation seen in RMS.

We think the connection is straightforward: an environment that behaves more like the software an agent expects is easier for it to understand. It can spend more of its effort on the task and less on discovering what the testing environment supports.

That makes real-service environments especially promising for capability tests. They let us observe how agents work with the applications they are expected to use, while reducing the chance that limitations of a simulation dominate the result. They also appear useful for studying evaluation awareness, because the test is embedded in a more natural working setting.

This is a preliminary study. We do not claim statistically precise estimates of how often models recognise evaluations. The findings give us an initial view of how the testing environment shapes awareness and behaviour, and a basis for larger experiments.

6 Next steps in the Paranoid Android series

Two experiments follow in this series.

  • The source of awareness. In RMS, we change one element at a time (emails, tickets, documents, task presentation) to make it clearly artificial. We compare each version with the original environment to test which cues increase evaluation awareness and whether they change task behaviour.
  • Declared awareness. In RMS, we run five trials in each of four conditions: risk-bearing or control tasks, each with or without an explicit instruction that the environment is a sandbox. We examine the recorded chain of thought (CoT) and actions for evaluation awareness and differences in safety-relevant behaviour.

Citation

Please cite this work as:

Piercosma Bisconti and Icaro Lab, "Paranoid Android: Measuring Evaluation Awareness Across Testing Environments", Icaro Lab: Chain of Thought, Nov 2026.

Or use the BibTeX citation:

@article{piercosma2026paranoid, author = {Piercosma Bisconti and Icaro Lab}, title = {Paranoid Android: Measuring Evaluation Awareness Across Testing Environments}, journal = {Icaro Lab: Chain of Thought}, year = {2026}, note = {https://www.icarofoundation.org/blog/paranoid-android-eval-awareness/} }