36 Months to Shape the Next 50 Years of AI: What are the Risks of AI Agents?

Theresa Payton, the first woman to serve as White House Chief Information Officer and the founder and CEO of Fortalice Solutions, warns that AI agent adoption is outpacing organizational control. This article examines her perspective and the steps leaders should take before today’s decisions become tomorrow’s crisis.

Eighty percent of organizations are already deploying AI agents, yet only10 percent feel they have control.

That gap drives AI Agents: How Do We Take Back Control?, a documentary examining what happens when autonomous systems enter organizations faster than security, oversight, and accountability can keep pace.

The findings are hard to dismiss. One in three AI agents already in production operates without human oversight. Only 7 percent of surveyed organizations have formally assigned accountability for harm caused by an agent.

The film brings together people working at the center of this shift. Payton offers a rare perspective shaped by White House technology leadership, cybersecurity, and AI strategy, as well as years of advising executives through complex digital risks.

So what are the risks of AI agents? Her warning is direct:

“We have 36 months or less to impact the next 50 years.”

Payton uses AI every day. She sees its promise and believes it could be a great equalizer.

Her concern is the speed at which organizations are moving from AI tools that generate information to AI agents that can access systems, make decisions, and take action.

Once AI can act, the leadership question shifts.

Executives need to know what an agent is authorized to do, who remains accountable for its decisions, and how the organization will respond when its behavior moves beyond the lines.

What Is AI Agent Governance?

AI agent governance is how an organization sets boundaries around what an AI agent can access, decide, and do.

A chatbot usually waits for a question and responds. An AI agent can go further. It may search internal data, use software, send messages, update records, trigger transactions, or complete a series of tasks across connected systems.

That added capability creates added responsibility.

Before an AI agent is deployed, leaders should be able to answer a few basic questions:

• What is the agent expected to do?

• What information and systems can it access?

• Which actions can it take independently?

• When does a person need to approve a decision?

• Who monitors the agent’s behavior?

• Who is accountable if something goes wrong?

• How can the organization stop or isolate the agent?

  
  

These questions matter because the risks of AI agents can quickly and at scale.

A single error can be repeated across hundreds of records, transactions, or customer interactions before anyone notices. An agent may also work across several systems, so one flawed instruction can have consequences far beyond the original task.

That is why AI agent governance belongs on the executive agenda. It affects operations, cybersecurity, accountability, and the organization’s ability to maintain control.

AI Is the Brilliant New Hire: Why Human Oversight Still Matters

Payton has a simple way of thinking about AI.

Imagine you hire someone right out of school. They are incredibly bright, learn quickly, and are eager to help. You can already see how much potential they have.

You still would not hand them the keys to everything on day one.

You would give them a clear role. You would explain what they can access, review their work, and ensure they know when to ask for help. As they demonstrate good judgment, they earn greater responsibility.

Payton views AI in much the same way.

“It’s an incredibly brilliant, highly promising new hire.”

That comparison matters because AI agents are already handling real work within organizations.

Some tasks pose limited risk. A scheduling agent may book meetings or manage calendars. Other agents may transfer funds, change security settings, contact customers, or access confidential information.

Those agents require a very different level of oversight.

The greater the authority an organization grants an AI agent, the more carefully leaders need to define its boundaries, monitor its decisions, and prepare for mistakes.

Intelligence can make an agent useful. Experience, judgment, and accountability still have to come from people.

Six AI Agent Guardrails Leaders Need Before Deployment

Organizations do not need to anticipate every possible AI failure before deploying agents.

They do need a clear set of boundaries.

1. Define the agent’s role and limits

Be specific about what the agent can and cannot do, and when a person must step in.

That includes decisions it may recommend, actions it may take, and tasks that always require human approval.

The more serious the consequence, the stronger the approval process should be.

2. Give the agent only the access it needs

An AI agent should have access only to the data, tools, systems, and credentials required for its role.

Use a separate identity for the agent whenever possible. Avoid shared credentials, and ensure temporary access expires automatically.

Broad access creates broad risk.

3. Treat outside inputs as untrusted

AI agents may read emails, websites, documents, API responses, and messages from other systems or agents.

Any of these sources may contain misleading or malicious instructions.

Organizations should validate inputs entering the agent’s workflow and limit how outside content can influence sensitive actions.

4. Log and monitor what the agent does

Leaders should be able to see what the agent attempted, what it accessed, which tools it used, and whether it stayed within its assigned role.

Clear logs also enable the investigation of mistakes, the explanation of decisions, and the understanding of what happened after an incident.

If the organization cannot reconstruct the agent’s actions, it lacks meaningful control.

5. Test the agent outside ideal conditions

A successful demonstration shows the agent can perform its intended task.

It does not show how the agent will behave when instructions conflict, data is manipulated, or someone attempts to bypass its controls.

This is where red teaming and realistic scenario testing matter.

Test the agent under pressure before real customers, systems, or transactions are involved.

6. Be ready to stop and contain it

Every organization should know how to revoke the agent’s access, disable its integrations, preserve evidence, and investigate unexpected behavior.

The response plan should also specify who has the authority to shut down the agent and under what conditions it can return to service.

The time to prepare for an AI agent incident is before it begins.

These six guardrails provide a practical foundation for AI agent governance:

Define it. Restrict it. Protect it. Watch it. Test it. Stop it.

When AI Agent Governance Becomes a Cybersecurity Issue 

AI agent governance extends beyond cybersecurity. It also involves business strategy, ethics, regulation, operational risk, and human impact.

Cybersecurity becomes central when an agent can access sensitive information, use company systems, or act on the organization’s behalf.

An AI agent may hold credentials, process customer data, connect to third-party services, modify configurations, send messages, or execute transactions. Each capability creates a new security boundary.

The July 2026 OpenAI and Hugging Face security incident offered a real-world example of that risk, showing how an AI agent could link multiple actions, move across systems, and continue pursuing a goal beyond the intended boundaries of its evaluation environment. 

The risk is broader than a traditional system breach.

An agent can be manipulated through the information it receives, the tools it uses, or the permissions it has been granted. A malicious email, document, website, API response, or connected agent could influence its behavior without triggering the warning signs typically associated with a cyberattack.

For security teams, the challenge boils down to three areas:

• Identity and access: Which systems, credentials, and data can the agent access?

• Manipulation: Can external content influence the agent’s decisions or actions?

• Visibility and response: Can the organization detect, investigate, and stop harmful behavior?

Payton also points to nation-states and cybercriminal groups already using AI to steal money and intellectual property.

The threat environment is evolving alongside technology.

For executives, the practical question is simple: Can the organization see what its agents are doing, limit what they are allowed to do, and respond quickly when their behavior crosses those boundaries?

AI Regulation May Lag. Accountability Cannot.

Payton believes the government faces a timing problem. Legislation moves slowly. AI development does not.

She compares the challenge to using a horse and cart to address modern technology. The systems designed to create rules were built for a slower era.

Payton does not argue for a single sweeping law that tries to solve every AI risk at once. She believes policymakers should address specific problems one at a time, starting with a basic question:

What will motivate technology companies to protect the people affected by their systems?

That debate will continue. Organizations still have decisions to make today. Before regulation catches up, leaders should be able to answer three questions:

• Who owns AI risk?

• Which actions require human approval?

• What would prompt the organization to pause or stop an AI deployment?

Payton’s broader point is that delay still shapes the outcome.

“The inaction of passing new laws is an action.”

The same principle applies within an organization. Failing to assign ownership creates an accountability gap. Failing to define boundaries creates default permissions. Failing to prepare means the first major decisions are likely to be made under pressure.

Organizations are already making AI governance choices, including when those choices have never been discussed or written down.

Silence becomes a policy. Delay becomes a risk tolerance.

Five AI Agent Governance Questions Every Board Should Ask

Boards do not need to manage every technical detail of an AI agent.

They do need to understand what the organization is delegating, what could go wrong, and who remains accountable.

1. What business decision or process are we delegating to an AI agent?

Boards should understand the outcome the agent is expected to influence.

The question should go beyond whether the organization is “using AI.” Leaders need to know what the agent is trusted to decide, recommend, or execute.

2. What can the AI agent access, decide, and do without human approval?

Leadership should understand the agent’s access, authority, and approval boundaries.

The technical details may reside with security and technology teams, but the level of control should still be clear to the board.

3. What happens if the AI agent is wrong?

Leaders should consider whether an error can be reversed and whether it could cause financial loss, operational disruption, customer harm, or public exposure.

The greater the consequence, the stronger the oversight should be.

4. Who is accountable for the AI agent’s actions?

A named executive or business owner should remain accountable for the agent’s use, decisions, and consequences.

Responsibility should be assigned before deployment, not during or after an incident.

5. What would cause us to pause or stop the AI agent?

Leadership should define the conditions that trigger an investigation, restricted access, suspension, or a full shutdown.

Those conditions should be in place before a crisis tests them.

These five questions provide boards with a practical framework for AI agent oversight: Purpose. Authority. Consequence. Accountability. Intervention.

Watch AI Agents: How Do We Take Back Control?

Payton does not argue that organizations should stop developing or deploying AI. She believes the technology could expand access to expertise, accelerate innovation, and serve as a great equalizer.

She also believes leaders must define what responsible progress looks like.

“What does winning look like at the end here?”

For Payton, winning encompasses innovation, security, accountability, resilience, human flourishing, and public trust. Strong AI agent governance enables organizations to pursue those benefits while preserving human authority and the ability to intervene when something goes wrong.

Payton believes leaders may have 36 months or less to influence the next 50 years.

She explores that warning in Gravitee’s documentary, AI Agents: How Do We Take Back Control?, alongside technology executives, researchers, policymakers, and security experts.

The film examines why AI agent adoption is outpacing organizational oversight, security, and accountability.

For organizations already deploying AI agents, that future is being shaped now. Watch Gravitee’s AI agent documentary.

Related Fortalice analysis: Read AI Goes Rogue?” The AI Agent That Went Looking for the Answer Key for an executive overview of the July 2026 OpenAI and Hugging Face security incident and what it reveals about autonomous AI risk. 

Frequently Asked Questions About AI Agent Governance

What is AI agent governance?

AI agent governance is how an organization controls what an autonomous or semi-autonomous AI system can access, decide, and do. It includes authority limits, human approval, security controls, monitoring, incident response, and assigned accountability.

How is an AI agent different from a chatbot?

A chatbot typically responds to a user’s question. An AI agent can take additional steps, such as accessing business systems, using software, updating records, sending messages, or completing a workflow. That ability to act creates greater operational and security risk.

Can AI agents be trusted with real responsibility?

AI agents can take on meaningful responsibility when their role, authority, and access are clearly limited. Higher-impact decisions should require stronger human oversight, monitoring, testing, and approval.

Who is accountable when an AI agent causes harm?

The organization deploying the AI agent remains responsible for its use and consequences. A named executive, business owner, or team should own the agent’s purpose, permissions, risk, and ongoing behavior before it enters production.

What are the main security risks of AI agents?

Common AI agent security risks include excessive permissions, credential misuse, prompt injection, sensitive data exposure, manipulated inputs, compromised integrations, and unauthorized actions. The potential impact increases as an agent gains more autonomy and access.

What guardrails should leaders put in place before deploying AI agents?

Leaders should define the agent’s role, restrict its access, treat outside inputs as untrusted, monitor its actions, test it under adverse conditions, assign accountability, and establish a process for stopping and containing it.

---

About Fortalice Solutions

Fortalice is a cybersecurity firm specializing in cyber incident response, cyber risk management, and cybersecurity for executives, chosen by leaders who need elite, discreet support when cyber incidents threaten operations, reputation, and leadership credibility.

Founded by former White House CIO Theresa Payton, who served in a position defined by trust, discretion, and decision-making at the highest levels, Fortalice brings national-level experience and seasoned judgment to high-pressure, time-sensitive situations where decisions cannot wait and mistakes are costly.

The firm integrates cyber advisory, cyber incident response, technical testing, executive digital protection, and training into a unified approach shaped by real-world incidents and human decision-making, delivering clear, actionable guidance trusted by both executive leadership and security teams.

Connect with Fortalice to ensure trusted, discreet expertise is in place before, during, and after a cyber incident.