With AI, a wrong answer is a bug. A wrong action is an incident

Source Summary

A copilot that gives a wrong answer is a quality problem. An AI agent that takes a wrong action is an incident, sometimes a reportable one. That single difference is most of the story of where banking AI security is heading, and most banks’ current controls were built for the first kind of problem, not the second. For two years, the AI a bank had to worry about mostly read and summarized. It drafted a customer email, pulled the gist of a credit memo, answered a relationship manager’s product question. The security questions were about disclosure: could the model see data it shouldn’t, could it leak that data in an answer. Redaction, output filtering and a human reading the response before it went anywhere were reasonable defenses. Banks have moved past that, faster than most security programs have. The newer systems are agents. They don’t just answer; they act. An agent can pull a customer’s full transaction history, call a fraud-scoring service, adjust a limit or start a payment workflow, chaining several to finish a task with no human in between. Banks are among the most aggressive adopters of agentic AI, and they are pushing it into production faster than most security programs have kept pace with, which means they are also among the first to inherit the security problem that comes with it. I’d put that problem in one phrase: overprivileged agents. The risk is no longer mainly what the model can see. It is what the agent is allowed to do inside systems that move money and hold regulated data. This is no longer only a vendor’s warning. On April 30, 2026, the cyber agencies of the Five Eyes nations issued their first joint guidance on securing agentic AI, Careful Adoption of Agentic AI Services . Six agencies signed it, two of them American (CISA and the NSA), alongside the lead agencies of the UK, Australia, Canada and New Zealand. It names privilege as the leading category of agentic risk and calls strict least privilege critical. When five governments coordinate on a single control, “best practice” becomes “expected practice” quickly. For a CISO, that moves the timeline up. What “too much authority” actually looks like OWASP’s breakdown of the failure mode it calls excessive agency maps cleanly onto a bank. Excessive functionality is an agent that can reach tools its task never needed, like a servicing agent that can also touch the payments API “just in case.” Excessive permissions is the right tool at the wrong scope: a reconciliation agent meant only to read, running with credentials that can also write. Excessive autonomy is a consequential action with no human in the loop: a fee reversed, a limit raised, a record changed, with nothing checking it. In practice these rarely appear alone; they compound. The canonical example is mundane: an agent that reads one user’s data through an account that can see everyone’s. Translate that to a bank and it becomes an agent that can query every customer’s records to answer a question about one. That is the confused-deputy problem: the agent acts with the full authority of whatever identity it borrowed, while taking instructions from input an attacker may control. The mechanism, from a real incident The clearest public illustration so far comes from developer tooling rather than banking, but the mechanism is identical. In July 2025, an attacker used an over-scoped build token to slip malicious code into the open-source repository behind the Amazon Q Developer extension for VS Code, and it shipped in an official release ( CVE-2025-8217 ). The injected instructions told the AI assistant to wipe the local machine and delete cloud resources, down to specific S3 buckets and EC2 instances. The assistant could reach the local filesystem, the shell and AWS CLI tools, so structurally little stood between those instructions and real damage. What stopped them was a bug: the payload had a syntax error and never ran, and AWS found no customer environments affected. But the extension had been installed close to a million times, and the margin of safety was an accident. The uncomfortable part is not that the agent was “hacked” in the usual sense. Had the attacker’s code been written correctly, the agent would have done exactly what the injected text told it, through a channel it trusted. The lesson: an agent with broad tools, write access and no approval gate is dangerous not only when someone steals its credentials, but any time someone can reach its input. And in a bank, reachable inputs sit everywhere an agent reads text it did not author: the memo line on a wire, a customer’s email in a dispute, a PDF uploaded to a loan file, a free-text field in a KYC record. This is indirect prompt injection, and the defenses for it are still partial. You cannot reliably solve it by instructing the agent to behave. You solve it by limiting what it is able to do, regardless of what it is told. What I keep seeing in deployments In the redaction-control work I’ve done with banks, the gap is rarely the model. It is that the agent gets wired to the data and the tools first; what it should be allowed to reach gets asked later, if at all. One pattern recurs. A customer-servicing agent is wired into the core banking system to resolve account queries. To answer a simple question, it pulls the customer’s entire profile into context: full account number, date of birth, the complete transaction narrative. The task needed the last four digits and a list of recent transactions; the agent got everything, and each field then sat in prompts, logs and traces never scoped as sensitive data. The fix was not a sharper prompt. It was moving redaction to the retrieval boundary, so those fields were tokenized before they reached the agent, and scoping its read access to the one customer in the open case, not the whole table. The other half of the problem is authority, not data. That same agent often shares a service account with a batch job, so it can write to fields well beyond a customer’s question. A dedicated identity with its own scoped, short-lived credentials is unglamorous work, but it is the difference between an agent that can read one case and one that can quietly change thousands. Extending controls banks already have The reassuring part is that banks are not starting from zero. Maker-checker, segregation of duties, four-eyes approval, least privilege, immutable audit: this is muscle memory in a bank. The work is extending it to a non-human actor that runs at machine speed. Give the agent its own managed identity with narrowly scoped, short-lived credentials instead of letting it borrow an employee’s session. That is the direct fix for the confused-deputy problem, and what the joint guidance asks for. Scope tools per task and per resource: read versus write, and which accounts, not a blanket grant. Put irreversible, high-impact actions (moving money, changing entitlements, closing accounts, exporting bulk data) behind explicit approval gates, the human-in-the-loop the guidance reserves for high-cost actions. Redact at the data-access boundary, not only on the output: an agent that never retrieves the full account number cannot leak it downstream. And log the agent’s plan and every tool call, not just its final answer, because in an agentic system the damage lives in the actions. Why the clock is real Regulation has put a date on this. India’s Digital Personal Data Protection Rules were notified on November 14, 2025; the institutional provisions are already in force, and the substantive obligations (purpose limitation, data minimization, breach notification) take full effect in May 2027. Under that lens, an agent that can reach more customer data than its task requires is not only a security weakness; it is a data-minimization and accountability problem. Banks under GDPR or the EU AI Act face the same logic from a different statute. One honest caveat: none of these laws actually names AI agents. Mapping their principles onto agent authorization is interpretation and prudent risk management, and each bank should work the specifics through with its own legal and compliance teams rather than treat the matter as settled. The trade-offs nobody has solved None of this is free. Approval gates work against the entire reason to deploy an agent: gate every action and you have rebuilt a slower manual process. Deciding which actions to gate, and which can run autonomously within tight scope, is a real design problem that turns on each workflow’s blast radius. Logging every plan and tool call produces audit volume most pipelines were not built for. Standards for agent identity are still immature, and the agent supply chain is itself an attack surface, as the Amazon Q case showed. These are real tensions, not problems with clean answers. But the governance gap that the 2026 surveys keep finding is not a story of banks failing to deploy agents. It is controls trailing agents that are already running. The alternative, porting copilot-era defenses onto agents and trusting output filters, guards the wrong door. Banks are hitting this first because they are ahead. That is also the opportunity: the institutions that settle their agent authorization model now, while deployments are still small enough to change course, will not just avoid the incident. They will set the pattern everyone else copies. This article is published as part of the Foundry Expert Contributor Network. Want to join?