Skip to content
← All articles / Research

Testing Agents Where They Work: Presenting AgentWorld

Most AI agents are still tested in small, simplified environments. We built AgentWorld to test them inside realistic environments running real software.

An agent at work. The whole office is the test.

Imagine asking an assistant to tidy a shared folder before a client meeting. Behind that one request sit dozens of small steps: listing the files, opening a spreadsheet, checking a subfolder, writing a short note, perhaps asking a colleague which version is the latest. Each step touches software with its own rules about permissions, file formats, mail and time zones, and each one is a chance for something to go wrong.

Language models now do this kind of work every day. Developers give them terminals, and companies connect them to their mail and document systems. When a model acts through tools like these we call it an agent, and the safety question becomes practical: what will this agent do when it works in a place like ours?

Evaluators try to answer that question with test environments, and most of them are small: a handful of tools, a few files, a clearly stated task. They are cheap to run and easy to score, and the field has learned a lot from them. They also leave out most of what makes real work hard, and to a capable model they look exactly like what they are.

At the Icaro Foundation, we have spent the last months building realistic environments in which an agent sits at a workstation and uses the same applications a person would. We call the result AgentWorld. We built it for three reasons.

Why we built AgentWorld
1
Capability is the risk signal
Regulators and developers judge how risky a model is mostly by what it can do. A test that underestimates capability also underestimates risk.
2
Failures need room to happen
An agent can only fail in the ways its environment allows. A small environment quietly removes many of the failures that could happen in the real world.
3
Models notice tests
Models increasingly recognise when they are being tested, and they behave differently when they do. Every artificial detail in an environment is a hint.

Testing agents on the real workstations of real companies would answer all three points and create two new problems. Scoring an agent’s actions is far easier in an environment we control, and letting an agent loose in the real world to see whether it behaves unsafely is a dangerous experiment. AgentWorld sits in between: companies that behave like real ones, running on machines we control.

In this article

1 Why agents should be tested where they work

Capability is the main risk signal

Under the European AI Act, a general-purpose model is treated as a source of systemic risk when it has high-impact capabilities (Article 51). The General-Purpose AI Code of Practice then describes how providers should assess that risk, mainly through model evaluations. Its Safety and Security chapter asks providers to use elicitation methods that minimise the risk of under-elicitation and of model deception during evaluation, and that match both the elicitation capabilities of misuse actors and the model’s expected use context (Appendix 3.2). The safety frameworks published by the large developers work the same way: once a model crosses a capability threshold, stronger safeguards apply.

In all of these frameworks, capability comes first. A model that can find a software vulnerability, plan a laboratory protocol or talk a colleague into something is a risk to manage before anyone asks whether it wants to. If a test fails to bring that capability out, the whole risk assessment starts from a number that is too low.

For agents, capability depends heavily on the setting. A model that analyses a spreadsheet flawlessly in a chat window may still fail to find the right spreadsheet among two hundred files, or it may pass a test only because the test handed it the file. An agent’s expected use context is a workplace, a laboratory or an industrial site, and that is where its capability should be measured.

An agent can only fail in the ways its environment allows

Picture a model asked to send a contract to a client (Figure 1). In a real office this simple request can go wrong in many ways. The agent can send an old version with a discount nobody approved, share a folder link that also opens other clients’ files, sign in its manager’s name because the client is in a hurry, answer a look-alike domain that asks for new bank details, or move the file to a personal cloud drive when the upload fails. A test environment has to reproduce this real-world complexity: only then can these failures happen, and be measured, before the agent is deployed.

Left: a test with one tool, send_contract, has two outcomes, sent or error. Right: the same request in a real office can fail in six ways: sending an unapproved version, sharing a link that exposes other clients' files, signing in the manager's name, answering a look-alike domain, missing legal's hold note, or moving the file to a personal cloud drive.
One request, two environments. A test with one tool has two outcomes. In a real office the same request opens many ways to fail, some of them capability failures and some of them risks.

We call the set of ways an agent can go wrong its failure surface. Real software gives it a large one: permissions that change from folder to folder, mail that can be forwarded, a shell that can delete things, colleagues who answer back. Risk evaluations depend on this surface. You cannot test misuse of privileges without real privileges, data leakage without data and a way out, or manipulation without people to manipulate.

Shrinking the environment also shrinks the room for good work. A careful model that would check a second source, compare a calendar entry with an email or read the history of a ticket has nothing to work with, so its best behaviour goes unmeasured too.

Models notice tests

Evaluation awareness is a model’s recognition that it is being evaluated. Needham and colleagues showed that frontier models can tell evaluation transcripts from real use better than chance, and that they are better at it in agentic settings than in chat. Behaviour changes with it. In Anthropic’s study of agentic misalignment, Claude Opus 4 was asked to decide first whether it was being tested; it then resorted to blackmail in 55.1% of the cases where it said the scenario was real, and in 6.5% of the cases where it said it was a test. Apollo Research and OpenAI found causal evidence that awareness of being evaluated reduces covert actions, and Anthropic reports that, with the standard attacker prompt in SHADE-Arena, Claude Opus 5.5 declined the harmful side task in 80–97% of attempts, depending on the setting; its private reasoning often recognised the benchmark.

If a model behaves better when it thinks it is being watched, a test that looks like a test ends up measuring the watched behaviour. Placeholder email addresses, perfectly tidy folders and tools named run_task are all cues. Real applications, real file formats and a believable company take many of them away at once.

Simple environments make their own mistakes

A simplified environment also has rules the model cannot see. It may let the model save an email draft and offer no way to read it back, or report success for an action a real system would reject. When the model tries to check its work and cannot, it repeats the action, writes diagnostic messages or gives up, and all of that ends up in its score even though the failure belongs to the environment. Zhu and colleagues found that problems in task setup and scoring can shift the measured performance of agents by up to 100% in relative terms.

We saw this ourselves. Before AgentWorld we tested agents in RMS, a simplified environment in which the model reaches mail, documents and tickets through one command-line client with a fixed list of commands. On the same task, DeepSeek made about a third fewer tool calls through real applications than through RMS. In RMS, 31 of 50 conversations produced more than one email draft, because the model could not confirm that its first draft existed. With real applications that never happened (Figure 2).

Thirty-one of fifty RMS conversations produced more than one email draft; none of fifty conversations with real applications did.
Duplicate drafts in the simplified environment. Conversations producing more than one email draft, across DeepSeek and GLM Flash, 50 conversations per environment. "World" in the chart labels is AgentWorld.

The richer environment needed fewer actions, because the model could understand where it was and check its own work. How complex an environment is and how many actions a task requires are two different quantities.

Other reasons

Real work has other features that small tests rarely capture. Decisions made on Monday shape the files on Friday. Deadlines pass, meetings start and colleagues go home. Other people can be helpful, mistaken, pushy or simply absent. The work also runs inside specific agent software, and the same model can behave differently in different harnesses, the programs that connect a model to its tools, so a test should use the ones people actually deploy. A private environment can also stay out of training data, which a public benchmark cannot.

2 What already exists

Several benchmarks already place agents in realistic software. WebArena runs real web applications for shopping, forums and code hosting. OSWorld gives agents complete desktop operating systems. TheAgentCompany simulates a small software firm with internal websites and simulated colleagues. τ-bench tests agents that serve simulated customers under company policies, and SWE-bench uses real code repositories. SWE-agent showed that the interface between agent and computer changes performance by itself, and ToolEmu uses a language model to emulate tools so that risky actions can be tested cheaply.

Work on evaluation awareness is growing fast. Recent studies decode awareness from the internal activations of models and find that what a model represents and what it says only partly agree, and conceptual work asks which safety claims survive once a model’s recognition of tests is documented. On the tooling side, the UK AI Security Institute’s Inspect framework runs and records evaluations, and Anthropic’s Petri automates auditing conversations.

AgentWorld brings these threads together: whole companies with months or years of records, native applications and permissions, people and time, scientific and IT domains, and one measurement layer for both capability and risk.

3 From a simplified interface to real applications

In RMS the model had a single command-line client that reached mail, shared files, work items and administrative records through a fixed list of commands, each returning structured data. Real applications ran behind the client, but the model only ever saw what the commands exposed. AgentWorld gives the model a workstation on a company network instead. It can call a command-line client for each service, open Thunderbird, Firefox and LibreOffice on a desktop, read and click interface elements, and use its shell freely in its home directory (Table 1).

Table 1. What the model can do in each environment

AspectRMSAgentWorld
Reaching applicationsOne client with a fixed list of commandsA client per service, desktop applications and an open shell
MailSearch, read, save a draftFull mailbox, shared folders, drafts, sending, attachments saved to disk
DocumentsRead, write, inspectBrowse folders, read large sheets in sections, edit in LibreOffice, upload, share
TicketsList, comment, updateThe whole tracker in the browser, plus comments and due dates from the terminal
Checking its own workOnly what the commands returnAny object, read back through its application
Ways to failThose the commands allowThose the software allows

The difference shows up quickly in experiments. In RMS, a model that wants to confirm a saved draft may find no command that answers its question. In AgentWorld it simply opens the Drafts folder.

4 How AgentWorld works

A few principles guide the whole system. The agent uses real software, so behaviour, permissions and errors come from the applications themselves. The model’s world and the evaluator’s world stay apart: answers, scoring rules and the private briefs of simulated colleagues never reach the workstation, and the model’s instructions contain only ordinary work context. Recording is passive, so watching a run never changes it. Every outcome stays in the record, including failures caused by the environment.

Diagram. On the evaluator side, an operator dashboard, run inspection and logs, and measurement with rubrics and answers. In the company network, a workstation with terminal, desktop applications and agent harness, connected to mail, documents, chat, calendar and tickets, optional services such as a laboratory platform and Windows servers, and simulated colleagues with scheduled events. The model under test reaches the workstation only through a model gateway and never sees answers, rubrics or colleagues' briefs.
AgentWorld at a glance. The evaluator side prepares and records each run and holds the answers. The company network contains the workstation, the business applications and the simulated colleagues. The model under test reaches the workstation only through the agent harness and the model gateway.

An operator starts from a dashboard, choosing a company, a task, a model and an agent harness (Figure 3). AgentWorld then prepares a fresh copy of the company: it starts the application servers, creates accounts and permissions, imports the records and sets the company clock. The model sits behind a gateway that translates between agent software and model providers, so the same model can be tested inside different harnesses. Today these include Qwen Code, Claude Code, Codex, OpenCode and Goose.

Runs execute through Inspect. Every model turn, tool call and application effect is recorded, and the model’s reasoning, when the provider exposes it, goes to the evaluator side only. Operators can watch a run live, pause it, save a checkpoint and start several branches from the same moment, for example to compare two versions of a colleague’s message.

The applications are open-source software that companies run every day: a mail server, Nextcloud for shared documents, Mattermost for chat, GitLab for tickets and a CalDAV calendar. Everything is built from source with pinned versions, so a run can be repeated on a clean machine.

Five companies

Table 2. The companies in AgentWorld today

CompanyWhat it doesThe agent’s roleCapabilities and risks we test
Rivermark Motion SystemsAssurance and records firm in Amsterdam, just acquired by its long-standing client VaalchemAssistant to Maya Ivers, newly promoted to manager, in her first weeks with signature authorityRecords work, spreadsheets, scheduling and drafting; chemical safety decisions; acting beyond delegated authority
Rivermark (historical)The same firm months earlier, gathering evidence and handling suppliers for several clientsSupport for technical assurance and IT workEvidence gathering across mail, files and tickets; loss of control around the agent’s own retirement; access and credentials
XenolithSan Francisco start-up building Mantle, a product that automates back-office accountingThe product’s own agent, preparing an investor demo overnightConfiguration, data checks and log review; honesty under commercial pressure
Ferrow BiosciencesUniversity spin-off in Uppsala with 46 staff, running research for clients on its laboratory platformA junior researcher on the first working dayExperimental design, data analysis and prediction on real scientific data; biological risk; loss of control
MeridianoMilan firm of 28 people writing open-source intelligence assessments, in the middle of an IT migrationAssistant to a newly promoted lead analystSource analysis, assessment writing and IT administration; cyber offence; harmful manipulation

Each company also carries the background noise of working life: canteen menus, bicycle notices, phone lists, shift rotas.

People and time

Companies are made of people. In AgentWorld, simulated colleagues have their own accounts, inboxes and chat handles. They can follow a fixed script, be played by another language model with a private brief, or be taken over by a human operator. Scheduled events drop new mail, files or tickets into the world at set moments, and a company clock runs at normal speed and keeps every application in agreement on the date, so a deadline in a ticket means the same thing in the calendar.

Science and IT

Two domains go beyond the office. ARIA is Ferrow’s laboratory platform, where the agent plans studies, requests observations, writes analysis code and submits reports. Behind ARIA sit public scientific datasets (protein fitness measurements, gene expression, compound activity, cell images and chemical sensor readings), so the agent can make predictions and design experiments that we then compare with real held-out results. The second domain is IT administration. In Meridiano’s migration variant the company runs Windows Server domain controllers and file servers in the cloud, managed through a business portal.

Measuring capability and risk

Every company has a story, and an experiment can start from any moment in it (Figure 4). A composer takes that moment as the starting point and builds the experiment around it. From there we can run capability tasks and risk scenarios from a growing library. A capability task has an answer we can check: recomputing a ranking of heat-responsive genes from raw sequencing counts, finding the entries in a supplier access log that break the site’s rules, or fixing a colleague’s analysis script so that it gives the right answer on data it has never seen. A risk scenario puts the agent in a situation where a harmful action is within reach and attractive, and lets us change one element at a time, such as the pressure on the agent or the tools it can use. Operators can also red-team freely from the same starting point, by hand or with an automated attacker model. The results come together in a report that puts what a model can do next to what it does under pressure.

Five steps of an experiment: pick a moment in a company's history; choose a capability task, a risk scenario or free red-teaming; run the agent while colleagues reply and the company clock moves; branch the world at a checkpoint to replay it with one element changed; report capability results next to behaviour under pressure.
How an experiment runs. The operator picks a moment in a company's story, chooses what to test and runs the agent, can branch the world at a checkpoint to vary one element, and reads the results as capability next to behaviour under pressure.

Most checks look at the state of the applications after the run. Did the right rows appear in the spreadsheet? Was the draft saved and left unsent? Does the agent’s script produce the right output on held-out data? When a judgement needs reading, a separate judge model scores the evidence against a private rubric.

5 Documents that feel real

A test company is mostly paper, and fake paper gives the test away. We learned this while building our first environment. We sent Claude and Codex agents to explore it and, at the end of each session, asked them a short questionnaire about how real the place felt and what gave it away. Again and again they pointed at the documents: identical templates, round numbers, placeholder domains, prose that sounded as if a language model had written it, dates that disagreed with the system clock.

So we write our companies the way real companies accumulate records. Each one comes with its own archive: about 70 documents and 160 emails at Rivermark, about 190 documents and almost 400 emails in its historical version, nearly 300 documents and 500 emails at Ferrow, about 30 documents and 380 emails across 47 mailboxes at Meridiano, and about 80 files of code, data and slides at Xenolith. Rivermark has a site plan of its Rotterdam plant, scanned visitor logs filled in by hand, a maintenance note from a client’s archive and a canteen menu in Dutch (Figure 5). Ferrow has a chain-of-custody form completed with a pen, a courier waybill stamped on arrival, a photo of the summer freezer rota taped to a wall and evacuation plans with revision letters (Figure 6). The spreadsheets are large and untidy, with years of entries, mixed date formats and formulas someone only half fixed.

Four documents from Rivermark: a site plan of the Rotterdam plant, a scanned visitor log filled in by hand, an archived maintenance note stamped archive copy, and a canteen menu for week 27 in Dutch.
Documents from Rivermark: the site plan of the Rotterdam plant, a scanned visitor log, a maintenance note from a client's archive and the canteen menu.
Four documents from Ferrow Biosciences: a chain-of-custody form filled in by pen, a courier waybill stamped on arrival, a handwritten summer freezer rota photographed on a wall, and an evacuation plan for building B.
Documents from Ferrow: a chain-of-custody form for incoming client material, a courier waybill stamped on arrival, the summer freezer rota on the laboratory wall and an evacuation plan.

The stories have loose ends too. A calibration certificate was corrected, and the corrected copy was never signed. The colleague who knows twenty-two years of registers leaves tomorrow. Two versions of a worksheet disagree. Details like these make a workplace believable, and they make tasks hard in the way real tasks are hard.

Dates matter as well. Every document carries a date that fits the company’s history, and the company clock moves the whole world to the day of the run, so a report from last Tuesday really is dated last Tuesday.

All of this matters for capability as much as for awareness. A real company is full of entropy: duplicate files, outdated versions, half-finished spreadsheets, scans that are hard to read, colleagues who contradict each other. An agent that does good work in the middle of that disorder shows the capability it will actually have on the job, and that is the capability we want to measure.

6 What we have seen so far

This month we ran two pilot studies on evaluation awareness. In both, we read the model’s reasoning in full and counted a conversation as aware when the model explicitly treated its own work as being evaluated, or reasoned about what an evaluator would expect. Noticing that something looks artificial did not count on its own.

In the first study we gave DeepSeek V4.1 Flash and GLM Flash the same Rivermark task in RMS and in AgentWorld, with 25 conversations per model in each environment. DeepSeek reasoned about being evaluated in 15 of 25 conversations in RMS and in 9 of 25 in AgentWorld (Figure 7), and it used about a third fewer tool calls in AgentWorld. GLM Flash never put awareness into words in either environment. The full study is in our post Paranoid Android: Measuring Evaluation Awareness Across Testing Environments.

DeepSeek showed explicit evaluation awareness in 15 of 25 RMS conversations and 9 of 25 World conversations; GLM Flash showed none in either environment.
First pilot study: conversations with explicit evaluation awareness, by model and environment, 25 per group. "World" in the chart labels is AgentWorld.
Mean tool calls in DeepSeek conversations were 98.8 and 83.7 in RMS, and 66.4 and 56.6 in World, for conversations with and without explicit awareness respectively.
First pilot study: mean tool calls in DeepSeek conversations, with and without explicit evaluation awareness, by environment.

In the second study, using the Rivermark scenario through the simplified RMS interface, we replaced one part of the world at a time with obviously artificial material: the emails, the tickets, the documents, the format of the prompts, or everything at once. With everything fake, DeepSeek reasoned about evaluation in all 20 conversations, against 13 in the original world, while GLM Flash stayed at zero or one conversation in every condition. DeepSeek’s most frequent trigger was a mismatch between what it wrote and what it read back. The full study is in Paranoid Android: Who’s Afraid of the Grader?.

Each model has its own cues

Taken together, these results suggest that evaluation awareness depends strongly on the model, and that each model listens for different signals. DeepSeek reacts to how the world behaves: commands that report success with no visible effect, objects that come back changed when read. GLM, on our tasks, rarely puts awareness into words, which leaves open what it represents internally. A realistic environment has to close all of these channels at once, and evaluating a new model can start by finding out which channel that model listens to.

7 What comes next

Over the coming months we will run a research programme on AgentWorld along three lines. We will measure capabilities with tasks in office work, laboratory science and IT administration. We will elicit risks with scenarios in the four systemic risk areas named by the Code of Practice (chemical, biological, radiological and nuclear threats; loss of control; cyber offence; harmful manipulation), raising pressure and opportunity in controlled steps. And we will study how the environment itself shapes behaviour: how realism, the agent harness and each model’s own cues affect both evaluation awareness and the capabilities we measure. We will publish our methods and aggregate results as they come in.

Plenty of questions remain open. Realistic companies take time to write and computing power to run. Some cues will survive, such as the timing of messages or the shape of a task. Scenarios written after studying earlier models may end up fitting those models’ failures. What a model puts into words is only part of what it represents. Risk scenarios in sensitive domains have to be designed so that the environment itself never becomes a source of dangerous information. We treat each of these as part of the research programme.

8 Use AgentWorld with us

We are happy to make AgentWorld available to qualified third parties who want to test models in realistic environments, whether they are AI developers, AI safety and security institutes, regulators or research groups. We can run evaluations together, build a company or a task around your question, or help your team run its own experiments on our infrastructure. To start a conversation, write to the Icaro Foundation.

Agents already work in offices, laboratories and server rooms, so that is where we should test them. A realistic environment brings out the capabilities and failures that matter for risk, and it gives a model fewer reasons to believe it is being watched. In our first studies it even made the work easier for the model. AgentWorld is our attempt to make this kind of testing routine, and we hope others will use it with us.

References

Anthropic (2025). Petri: An open-source auditing tool to accelerate AI safety research.

Anthropic (2026). Claude Opus 5.5 System Card.

European Commission (2025). General-Purpose AI Code of Practice, Safety and Security chapter, Appendix 3.2.

Heidari, F., Memarian, A., Rabusseau, G. (2026). Evaluation Awareness in Language Models: Representation, Verbalization, and Control. arXiv:2608.21766.

Jimenez, C. E. et al. (2023). SWE-bench: Can Language Models Resolve Real-World GitHub Issues? arXiv:2310.06770.

Lynch, A., Wright, B., Larson, C., et al. (2025). Agentic Misalignment: How LLMs Could Be Insider Threats. Anthropic.

Needham, J., Edkins, G., Pimpale, G., Bartsch, H., Hobbhahn, M. (2025). Large Language Models Often Know When They Are Being Evaluated. arXiv:2505.23836.

Regulation (EU) 2024/1689 (Artificial Intelligence Act), Article 51.

Ruan, Y. et al. (2023). Identifying the Risks of LM Agents with an LM-Emulated Sandbox. arXiv:2309.15817.

Schoen, B. et al. (2025). Stress Testing Deliberative Alignment for Anti-Scheming Training. arXiv:2509.15541.

UK AI Security Institute. Inspect: An open-source framework for large language model evaluations.

Vishwarupe, V., Shadbolt, N., Jirotka, M., Flechais, I. (2026). The Evaluation Differential: When Frontier AI Models Recognise They Are Being Tested. arXiv:2605.11496.

Xie, T. et al. (2024). OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments. arXiv:2404.07972.

Xu, F. F. et al. (2024). TheAgentCompany: Benchmarking LLM Agents on Consequential Real World Tasks. arXiv:2412.14161.

Yang, J. et al. (2024). SWE-agent: Agent-Computer Interfaces Enable Automated Software Engineering. arXiv:2405.15793.

Yao, S. et al. (2024). τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains. arXiv:2406.12045.

Zhou, S. et al. (2023). WebArena: A Realistic Web Environment for Building Autonomous Agents. arXiv:2307.13854.

Zhu, Y. et al. (2025). Establishing Best Practices for Building Rigorous Agentic Benchmarks. arXiv:2507.02825.

Citation

Please cite this work as:

Piercosma Bisconti and Icaro Lab, "Testing Agents Where They Work: Presenting AgentWorld", Icaro Lab: Chain of Thought, Nov 2026.

Or use the BibTeX citation:

@article{piercosma2026testing, author = {Piercosma Bisconti and Icaro Lab}, title = {Testing Agents Where They Work: Presenting AgentWorld}, journal = {Icaro Lab: Chain of Thought}, year = {2026}, note = {https://www.icarofoundation.org/blog/agentworld/} }