Nana Wu
Many introductory tutorials for building AI agents follow a deceptively simple pattern: provide a language model with a set of tools, initiate a loop, and let it run until an answer emerges. While this approach effectively demonstrates the concept of tool calling, it glosses over the critical engineering challenges that arise in production environments. What happens when a tool fails? How do you validate arguments from an untrusted model? How do you prevent an agent from exhausting its context window or executing unsafe commands?
These questions define the responsibility of the agent harness—the essential software layer sitting between a large language model (LLM) and the environment it manipulates. By building a provider-independent harness in TypeScript, we can move beyond basic demos to create robust, observable, and testable agent systems.
The Architecture of Control: Main Facts
The fundamental architectural principle of a secure harness is clear: The model proposes actions, but the harness executes them.
When a model requests a tool call, it has not yet performed any action; it has merely generated a structured request containing a name and arguments. The harness acts as the gatekeeper, interpreting that request, validating it against strict schemas, executing the implementation, and feeding the result back into the conversation. This separation of concerns ensures that the model remains a reasoning engine, while the harness serves as the hardened, deterministic control layer.
Chronology of Development
To understand how to build this system, we can follow a modular development path, moving from basic execution loops to a production-grade harness.
1. Building the Basic Agent Loop
The initial phase involves creating a contract that defines how the model and tools interact. By creating a Model interface that is decoupled from specific providers like OpenAI or Anthropic, we ensure the harness remains portable. The core loop follows a simple sequence: send the conversation history to the model, parse the response, execute requested tools, and append the results. Importantly, tool failures are treated as data—the harness records an error associated with the specific call ID, allowing the model to recover in subsequent turns.
2. Implementing Tool Dispatch and Validation
The next milestone introduces a ToolRegistry to manage execution. A common security risk is trusting model-generated arguments. In this stage, we implement a JSON Schema validator that checks every argument before a tool is invoked. If validation fails, the tool is never executed; instead, the harness reports the error to the model. This creates a hard security boundary that prevents the agent from passing malicious payloads into the system.
3. Workspace Isolation and Command Restrictions
For a coding agent, the ability to read and write files is essential. However, exposing the entire filesystem is a critical vulnerability. By implementing a Workspace class, we restrict file operations to a specific root directory. The harness resolves canonical paths, rejects attempts to escape the root (e.g., ../../secrets.txt), and forbids symbolic links. Similarly, command execution is restricted to a set of predefined presets, preventing the model from inventing its own arbitrary shell commands.
4. Enforcing Execution Limits
A runaway agent can quickly exhaust resources. By implementing maxSteps, maxToolCalls, and timeoutMs, we ensure the harness retains ultimate authority over execution. We use the AbortController API to propagate cancellation signals throughout the entire stack, ensuring that if a process exceeds its time limit, it is cleanly halted.
5. Managing Context and Observability
As the conversation grows, the context window can become a bottleneck. We implement a selectContext utility that manages the transcript, retaining only the most relevant turns when the context budget is reached. By treating assistant turns and their subsequent tool results as an "indivisible group," we ensure the model never receives an incomplete or misleading conversation state. Finally, integrating an event-based observability system allows developers to track the lifecycle of a request—from model_start to run_end—without tightly coupling the telemetry to the agent’s logic.
Supporting Data and Testing
The most significant hurdle in AI agent development is the non-deterministic nature of models. To solve this, we employ deterministic testing. By using a "scripted fake model" that returns predefined responses, we can verify that the harness handles failure paths—such as malformed tool calls, budget exhaustion, or invalid argument schemas—exactly as expected.
For instance, testing the budget limits involves running 150 seeded scenarios where the model requests varying batch sizes. These tests confirm that when a batch exceeds the budget, the harness rejects the entire request rather than executing a partial, potentially inconsistent set of actions. This creates a verifiable foundation for system reliability.
Official Design Philosophy
The design philosophy behind this harness is rooted in cooperative safety. The harness does not attempt to "fix" the model; instead, it constrains the environment so that the model’s potential failure modes (like hallucinated parameters or circular reasoning) result in safe, observable errors rather than system crashes or security breaches.
By isolating the "intelligence" (the model) from the "mechanics" (the harness), we follow the principle of least privilege. The model should never have direct access to the filesystem or the network; it should only have access to the narrow, validated tools that the harness explicitly exposes.
Implications for Production AI
Building an agent harness from scratch has profound implications for developers:
- Portability: By defining a universal
Modelinterface, swapping from a local Llama 3 instance to a cloud-hosted Claude 3.5 model becomes a simple task of updating the adapter, with no changes required to the core agent logic. - Security: The move from "trusting the model" to "validating the model" is the difference between a prototype and a secure enterprise tool. By treating LLM output as untrusted user input, we eliminate entire classes of prompt-injection vulnerabilities.
- Observability: Without an event-based harness, an agent’s internal reasoning is a black box. A structured event stream allows teams to perform post-mortem analysis on failed tasks, identifying exactly where a model went astray or where a tool bottlenecked.
- Reliability: The use of cooperative cancellation and execution budgets ensures that the system is predictable. In a production environment, the ability to bound the cost and time of an agent run is non-negotiable.
Ultimately, the agent harness is not just an optional utility—it is the essential infrastructure required to bring AI agents into professional software ecosystems. As models become more capable, the complexity of the harness must increase, but the core principles remain constant: validate inputs, bound execution, and maintain observability at every turn. By building these systems from the ground up, engineers gain the control necessary to turn experimental AI demos into reliable, scalable software agents.
Dwi Wanna
Laily UPN
Ammar Sabilarrohman
Asep Darmawan
Asro
Layla Zulfa
Nila Kartika Wati
Muslim