Governing the AI bank

What’s the difference between a digital bank and an AI bank?

Emmanuel Daniel
Founder, The Asian Banker

Digital banking upgraded a traditional banking business model. It was aimed at doing what banks have done since the 1950s, but doing it faster and better.

AI represents a fundamental shift. It changes everything from how a bank is organized to how it makes decisions—even, or perhaps especially, how it communicates with customers.

In the past, banks controlled how and when customers interacted with them. Take the bank app, a creation of the digital banking era. Banks spent a lot of time making their apps easier to use and more visually appealing.

In the AI era, the app could become irrelevant. Customers will simply tell their agent what they want, and her agent will contact the bank’s agent. Banks will have to reduce friction for agents, not just human customers.

If autonomous agents begin making decisions and transacting with one another, what new governance problems does that create?

The key question right now is, “If something goes wrong, who is responsible?” That leads to a whole series of other questions about security, decision-making, and accountability. Even regulators are trying to determine whom to hold accountable, and who should have the authority to hit the “kill switch” that shuts down an AI that is causing harm.

If regulators move too slowly, can banks create their own AI governance systems?

I don’t think many banks around the world are thinking very deeply about this yet. They assume they can retain the institution in the form they are familiar with, and that decisions and responsibilities will basically stay the same.

That’s misguided at best. In an autonomous, agentic-AI-driven ecosystem, it’s not just processes that will change. The whole institutional framework will be transformed.

Right now, the different departments in most banks are thinking about how to use generative AI to make their employees more productive. They expect people to do more work every day.

But with agentic AI, many of those people may no longer be needed. Then whom do you hold responsible for a faulty process or failed transaction?

That sounds worrying for depositors as well as bankers. You’ve used the expression “manufactured belief” to describe one AI transmitting false information to other AIs. Could that turn a small error into a systemic problem?

When agents act in concert, an error in the process can be magnified. One agent carries an error to another agent, which amplifies and complicates it. Eventually, you may not even be able to identify where the error started.

Agents are learning platforms. In the old days, you programmed something and all the applications ran the program you had created. You were responsible for monitoring the program.

Today, an agent looks at what you created and improves on it, or interfaces with another agent and makes a decision that may be totally different from what you intended. It may also amplify what you intended—or what you didn’t.

That’s what we need to start thinking about. At what point do you raise a red flag? At what point do you shut down the process so you can take a closer look at what’s happening?

Is it harder to trace an accidental error than a malicious attack?

Malice may be easier to trace because there is intent—for example, the intent to commit fraud. You can audit the process and identify where someone deliberately introduced a factor that led to a fraudulent transaction.

An accidental AI error may have passed through multiple agents before anyone detects it. By then, it may be difficult to locate the original source.

You need good procedures to catch the problem early and correct it. You also need clear guidelines on when to use the kill switch if things look like they might spiral out of control.

No one has the answers right now. The whole world is on a learning curve.

Is this really a good idea? Are there some areas where we should use AI and others—perhaps banking—where we should leave well enough alone?

Banking is ripe for AI because it is a highly transactional industry. Millions of transactions take place daily around the world. Banking can benefit from using AI to make transactions smoother, faster, cheaper and safer.

But there are costs. The big question for AI in banking is: What happens when things go wrong?

Should we focus on whether AI makes banking safer overall, despite occasional failures?

It is still very early, and we don’t have the data. We don’t know how many bad transactions there have been, how much money has been lost or cheated out of the system, or how much fraud has occurred.

Once we develop a clearer idea of what AI failures are actually costing us, we can work backward and start solving the problems.

A system may be able to tolerate a 1% or 2% error rate, but perhaps not 5%, because at that point the cost begins to undermine the value of the transactions. We don’t have those numbers yet.

It may take another year before research around the world gives us a clearer picture of AI banking failures—the error rates, fraud losses and bad transactions. Only then will we know the scale and cost of the problem.

Can regulation play the same role in mitigating these risks that it plays in traditional banking?

Regulators are still learning how they want to deal with this. The issue is that an entire organization’s decision-making structure is called into question. How much responsibility is borne by the bank’s management? By its board of directors? Or by the people who run and monitor the system?

There are real risks to the bank’s balance sheet. If AI creates a liquidity crisis, how do you protect the bank and its depositors?

Bankers and regulators are struggling with this complex problem. They want to ensure checks and balances are in place without stifling innovation.

How can banks begin preparing for risks they don’t yet fully understand?

One useful approach is scenario planning. You imagine the potential scenarios that could arise and then work backward to determine where the kill switches should be.

For example, banks are already deploying a lot of AI in Know Your Customer (KYC) processes, especially those involving payments. Before a transaction is allowed to close, it has to pass certain screening parameters.

Large banks may process hundreds of millions of transactions. At that scale, the system will make marginal mistakes. But it will also learn. Every time it makes a mistake, you can feed the correction back into the system so that it doesn’t make the same error again.

Regulators must work with banks to improve these processes rather than looking only for opportunities to punish or fine a bank for a specific transaction.

When AI runs most banking operations, how can we prevent “the machine did it” from becoming a blanket excuse for failures?

We are no longer in the age of finding fault. When there is no single person to blame, you still have to fix the problem. That is the core governance consideration in AI.