What is it about?

AI agents are different from traditional software because they can reason about goals, make decisions, use tools and APIs, access data, retain context, and take actions with limited human involvement. These capabilities create security risks that conventional threat-modeling approaches may not fully capture. This article explains a practical approach to threat modeling agentic AI systems. It shows security teams how to examine an agent’s goals, identity, permissions, tools, memory, data access, trust boundaries, and runtime behavior. It also considers how risks can emerge when an agent’s plans, permissions, context, or actions change during execution. The goal is to help architects, developers, security practitioners, and organizations identify agent-specific threats earlier and design security controls that remain effective while AI agents are operating.

Featured Image

Why is it important?

AI agents are moving from systems that simply generate answers to systems that can take real actions. They may call APIs, execute tools, access enterprise data, interact with other agents, and make decisions across multi-step workflows. This changes the security problem. Security reviews cannot focus only on prompts, models, or application boundaries. They must also consider what an agent is authorized to do, how its authority changes during execution, what tools and resources it can reach, and whether its actual actions remain aligned with the original intent. This article provides practitioners with a structured way to reason about these risks through agentic threat modeling. The approach can help organizations identify dangerous trust boundaries, excessive authority, unsafe tool interactions, and runtime risks before they become security incidents. As autonomous AI becomes more deeply integrated into enterprise systems, threat modeling that accounts for agent behavior and runtime authority can become an important part of secure AI engineering.

Perspectives

I wrote this article because I believe AI agents require us to rethink some assumptions behind traditional application security. When software primarily responds to requests, we can often reason about security around relatively stable application flows and trust boundaries. An AI agent may instead interpret a goal, create a plan, select tools, access resources, and take a sequence of actions that was not explicitly defined in advance. For me, the important security question is therefore not only “Can this agent perform this action?” but also “Should this agent be allowed to perform this action, in this context, at this point in the workflow, for this purpose?” My goal with this work is to make agentic AI security practical for architects, developers, threat modelers, and security teams. I hope it encourages practitioners to examine identity, authority, tools, memory, trust boundaries, and runtime behavior together rather than treating AI security as only a model or prompt-security problem.

Ravindra Annam

Read the Original

This page is a summary of: Running an Agentic Threat Model, Queue, September 2026, ACM (Association for Computing Machinery),
DOI: 10.1145/3848521.
You can read the full text:

Read

Resources

Contributors

The following have contributed to this page