Trusting the Machine: Sophos AI Unveils New Methodology to Audit Autonomous SOC Agents

For the fourth consecutive year, Sophos AI, the research arm of the global cybersecurity leader, is set to make a significant contribution to the academic and practitioner discourse at the Conference on Applied Machine Learning in Information Security (CAMLIS). On October 23, Senior AI/ML Engineer Tamás Nyiri will present a pioneering poster session titled “Show, Don’t Tell: Calibrating Agentic SOC Analysts via Hierarchical Evidence Graphs.”

As Security Operations Centers (SOCs) increasingly pivot toward autonomous agents to triage the deluge of daily alerts, the industry faces a critical paradox: while these agents drastically reduce the time-to-resolution, they also introduce a "black box" risk. Sophos’s latest research seeks to solve this by forcing AI agents to expose the internal logic of their investigations, allowing human analysts to identify potential errors before they lead to catastrophic security lapses.

The Rise of the Agentic SOC

In the modern cybersecurity landscape, the sheer volume of telemetry—logs from endpoints, networks, firewalls, and cloud environments—has far outpaced the capacity of human analysts. To bridge this gap, SOCs have turned to agentic AI. These systems do more than just flag alerts; they act as autonomous digital investigators. They extract entities, correlate context, plan investigation steps, query disparate tools, and ultimately generate a final verdict on whether a threat is malicious or benign.

While these agents are efficient, they are not infallible. They suffer from systemic blind spots. An agent might interpret a missing log as evidence of a clean bill of health rather than a sign of a compromised host. They may erroneously link two events simply due to temporal proximity, or they might struggle to distinguish between a legitimate administrative tool and a piece of malware performing similar actions. Because these agents are trained to present their findings in a confident, narrative format, these subtle, high-stakes errors are often buried under layers of technical jargon, making it difficult for human reviewers to catch them.

A New Framework: The “Fact Graph”

Recognizing that current reporting structures hide the "how" behind the "what," Tamás Nyiri and his team at Sophos AI developed a methodology to force AI agents to show their work.

The core of this approach is the creation of a "fact graph." Instead of relying on the agent’s final report, the system parses the structured trace fields of the investigation and uses a Large Language Model (LLM) to extract meaning from free-text outputs under a rigid, fixed schema. This process strips away the agent’s persuasive narrative, leaving behind a cold, structural map of the investigation.

How the Fact Graph Functions

The fact graph functions as a repository of reality. It captures:

Observed Evidence: A granular record of what the agent actually saw during its investigation.
Tool Failure Metadata: A log of which queries failed or timed out, providing context on where the agent may have lacked sufficient data.
Entity Relationships: A map of how different assets, users, and processes interact within the scope of the alert.
Inferred Missingness: Perhaps the most innovative feature, the graph explicitly identifies what is not there—such as alternative hypotheses the agent failed to consider or crucial data sources that remained unqueried.

By withholding the agent’s ultimate verdict during the creation of this graph, the researchers ensure that the evaluation remains objective. The graph acts as a building block for three distinct "evidence views," which are then fed into a traditional supervised machine learning model. This model serves as a calibration layer, designed to estimate the probability that a senior human reviewer would disagree with the agent’s initial conclusion.

Chronology of the Research Development

The project began as an internal effort to improve the quality of automated triaging within Sophos’s own threat-hunting ecosystem.

Phase 1: Defining the Failure Modes (Early 2024): The team analyzed historical cases where AI agents reached a consensus that was later overturned by human experts. They identified common patterns of failure, such as confirmation bias and over-reliance on limited telemetry.
Phase 2: Developing the Fact Graph (Spring 2024): Engineers built the parsing architecture to convert unstructured investigation logs into the graph format.
Phase 3: The Human-in-the-Loop Labeling (Summer 2024): To train the model, the team implemented a two-stage review process. A primary analyst reviewed the agent’s investigation, followed by a senior analyst who established the "ground truth."
Phase 4: Modeling and Validation (Q3 2024): Using 314 distinct cases, the team applied a regularized logistic regression model to score each investigation. They utilized leave-one-out cross-validation (LOOCV) to ensure that no specific case biased its own scoring, maintaining the integrity of the data.

Supporting Data and Performance Metrics

The performance of this proof-of-concept is, by all accounts, highly encouraging. The researchers sought to answer three critical questions for every investigation:

Is the verdict correct?
Did the agent investigate the right entities?
Was the conclusion supported by the evidence found?

By assigning a "disagreement score" based on the model’s predictions, the researchers were able to prioritize cases for human review. In a standard workflow, a reviewer might look at alerts at random. Using the Sophos AI approach, they look at the alerts the model deems most likely to be wrong.

The data speaks for itself: while the senior reviewer overturned the agent’s verdict in 25% of all cases, that number jumped to 65% among the 20 highest-scoring cases. This represents a 2.6x increase in the efficiency of finding errors. By effectively "triaging the triagers," the system allows human experts to focus their limited time on the investigations most likely to require a correction.

Official Perspectives and Industry Implications

The implications for the cybersecurity industry are profound. As enterprises move toward "autonomous security operations," the ability to audit an agent’s reasoning becomes as important as the agent’s capability itself.

"The goal is not to replace the analyst," says Nyiri. "The goal is to calibrate the trust we place in the agent. If we can provide a mechanism that highlights why an agent might be wrong, we empower human experts to supervise these systems effectively rather than blindly trusting them."

This research signals a broader trend in AI safety within security: the move toward "explainable agentic workflows." As AI systems take on more responsibility in defending critical infrastructure, the necessity for transparency will likely become a regulatory and operational requirement. Sophos AI’s work provides a blueprint for how this transparency can be achieved without sacrificing the speed that makes AI agents valuable in the first place.

Future Outlook: Beyond the Proof-of-Concept

While the results are strong, the research team acknowledges that this is still in the early stages. Future iterations of the model will likely incorporate more complex graph neural networks to better understand the relationships between entities. There is also the potential to expand the "fact graph" to include external threat intelligence feeds, providing even richer context for the calibration model.

For those attending CAMLIS, the poster session on October 23 offers a unique opportunity to see these fact graphs in action. Nyiri will demonstrate a real-world case study, showing how a seemingly straightforward investigation was flagged by the model and subsequently corrected by a human, preventing a potential false negative.

As the industry stands on the precipice of a new era of AI-driven security, Sophos AI’s contribution serves as a reminder that in cybersecurity, trust is not something to be granted to an algorithm—it is something that must be earned through verifiable evidence. By making the "black box" transparent, the researchers are ensuring that as our digital defenses become more automated, they also become more reliable.

For security professionals, the message is clear: the future of the SOC isn’t just about faster agents; it’s about smarter, more accountable ones. The "Show, Don’t Tell" methodology is a significant step toward making that future a reality.