Open-Source AI Regulation: Why Risk Should Guide the Decision
A powerful new AI model built in China hits the open market, and the instinct is immediate: lock it down. But what if that instinct, however reasonable it sounds, is the wrong starting point for leaders trying to protect their organizations?
That’s the argument Theresa Payton, former White House Chief Information Officer and CEO of Fortalice Solutions, made in a recent interview with Akash Pasricha at The Information.
Their conversation began with national security concerns surrounding Kimi K3, a Chinese-developed open-weight AI model, but quickly broadened into a harder question about open-source AI regulation: how should leaders evaluate these models when cost, competition, national security, and business resilience all point in different directions?
Payton thinks the current debate is too blunt, lumping open-source models together or treating national origin as a simple litmus test. Where a model comes from can raise real security questions, but so can locking your organization into a closed vendor you can’t fully audit. The risks are different, not one-sided.
What’s missing, in Payton’s view, is evidence.
“Let’s do the work,” Payton said. “Let’s get it in the lab, see what the vulnerabilities are, and then let businesses decide for themselves based on the facts.”
That puts the executive decision in clearer terms: start with the risk, test the assumptions, and govern the model based on what it can actually do.
What Theresa Payton Told The Information About Open-Source AI Regulation
Payton’s position on open-source AI regulation rests on three ideas:
Build resilience: Avoid making one AI provider a mission-critical dependency.
Require evidence: Let a model’s origin shape the investigation, then test the concerns in a controlled environment.
Govern every model: Apply governance and guardrails across open and closed systems.
Together, these principles give leaders a defensible way to decide which models to approve, restrict, or reject.
AI Model Choice Is a Resilience Decision
Before debating which AI model is best, Payton argues leaders should ask a more uncomfortable question: what happens if the one you chose disappears?
As businesses weave AI into customer service, software development, research, security, and other essential operations, access to a specific model can become mission-critical.
Anything mission-critical that depends on a single provider creates a concentration risk. Leaders should know what happens if:
The provider raises prices or changes terms
The model becomes unavailable or restricted
The model no longer meets security requirements
The organization needs to move its data or workflows elsewhere
Payton advises executives to diversify for resilience, recoverability, security, and supply-chain continuity. A company that cannot operate without one AI provider has handed that vendor an extraordinary amount of control over its future.
That risk compounds as the model gains access to sensitive information, production systems, and critical business processes.
It also helps explain a shift Payton is seeing as businesses reconsider the cost of closed models. Newer versions can consume more tokens and cost more to run, pushing organizations toward lower-cost open-source alternatives that may be less advanced and still capable enough for the job.
“Good enough” can be a rational business standard.
An internal research assistant doesn’t need the most advanced model on the market. A security team may prefer to run a model inside its own infrastructure. A company may also value the ability to customize a model and keep sensitive data within a controlled environment.
Open-source models can offer lower costs, greater deployment control, better data isolation, and stronger continuity options, advantages that matter when operational independence is the priority.
Those advantages come with added responsibility.
The organization takes on hosting, configuration, access controls, updates, vulnerability monitoring, and long-term maintenance. Vendor dependence shrinks; internal obligation grows.
How Kimi K3 Changed the Open-Source AI Regulation Debate
Kimi K3 entered the conversation because its capabilities attracted attention and its Chinese origin raised national security concerns. Those concerns deserve examination.
The problem arose when the discussion moved from one model and one set of risks into a much broader argument about restricting open-source AI altogether.
Payton described that shift as a conflation of separate questions:
Does a specific Chinese-origin model create a security risk?
Do open-source models require governance?
Should open-source AI face broad restrictions as a category?
Each question requires analysis.
A model's country of origin can shape the risk analysis, including questions about supply chains, applicable laws, intellectual property, distribution channels, and potential government access. Hosting, deployment, licensing, and data flows determine how those risks apply in practice.
They do not reveal whether a particular vulnerability exists, how it could be exploited, or whether the organization can reduce the exposure. That requires testing.
Payton cautioned that sophisticated testing takes time and can still miss hidden weaknesses. That uncertainty should lead to stronger analysis. It should not lead to complacency.
What Risk-Based Open-Source AI Regulation Should Focus On
Governance and guardrails should begin with the technology itself, Payton said. Questions involving compute, chips, and data centers also require attention as AI adoption grows, but her recommendation is to begin with the software.
Her answer points toward a risk-based framework that considers:
The model’s capabilities
How and where it will be deployed
The systems and information it can access
The consequences of failure or misuse
Security testing and containment requirements
Monitoring and documentation
Human oversight
Clear accountability
Payton specifically emphasized rigorous testing, internal governance, and stronger containment around advanced AI evaluations. Her comments support a broader principle: higher-consequence deployments warrant stronger controls.
A model summarizing public information creates a different level of exposure than one connected to sensitive records, financial systems, security controls, or consequential decisions.
Internal enterprise policy should reflect those differences.
Open and Closed AI Models Create Different Risks
Open models place greater operational responsibility on the organization. Closed models place greater trust in the provider.
An organization deploying an open-source model may need to manage hosting, isolation, updates, monitoring, and vulnerability response. An organization using a closed model should examine data retention, provider access, contractual protections, service availability, model changes, and the ability to move to another platform.
Those differences do not change the core requirement: leaders need a consistent process for deciding which models to approve, restrict, or reject.
Seven Questions Every Executive Should Ask Before Approving an Open-Source AI Model
Executives need seven clear answers from the teams evaluating the model.
1. What business problem will the model solve?
Define the task, the intended users, and the expected outcome.
AI experiments can quietly spread across departments. A tool introduced for one limited purpose can become embedded in production workflows before leadership understands how widely it is being used.
A well-defined use case gives the organization a boundary for testing and enforcement.
2. What data will the model access?
Identify every category of information the model may receive, including customer records, employee information, credentials, regulated data, intellectual property, private communications, source code, and confidential business plans.
The sensitivity of the data should determine the strength of the controls.
3. Where will the model run?
Leadership should understand whether the model will operate within the company’s infrastructure, in a public cloud, via a third-party API, or across a hybrid environment.
The team should also explain what information leaves the organization, where that information travels, and how long it is retained.
4. How has the model been tested?
Performance benchmarks answer only part of the question.
Testing should also examine data leakage, prompt injection, unauthorized access, unsafe behavior, unexpected use of tools, privilege escalation, and the model’s response to adversarial instructions.
The test environment should reflect the way the organization plans to use the model.
5. Who owns AI model updates and vulnerability management?
Open-source models are continually evolving. New versions can enhance performance, introduce vulnerabilities, or disrupt existing systems.
Leadership should be aware of who reviews updates, tests compatibility, monitors vulnerability disclosures, approves new versions, and manages decisions regarding rollbacks.
6. What happens if the AI model fails or becomes unavailable?
Mission-critical AI requires a recovery path.
The organization should have a replacement option, a manual process, a rollback procedure, a data-portability plan, and a clear understanding of which operations would halt without the model.
No model should become an invisible single point of failure.
7. How will the organization monitor and stop the AI model?
Approval initiates the governance process.
Organizations need logging, access controls, human review, escalation paths, and clear intervention thresholds.
The executive approving the model should know who can stop it, how quickly that can happen, and what the organization will do next.
Unclear answers indicate the model is not ready for production.
Better AI Policy Starts With Better Questions
Open-source AI models can lower costs, improve resilience, and give organizations more control. They can also introduce security, maintenance, and governance risks.
That is why leaders should evaluate each model based on its capabilities, access, deployment, and potential impact. The country of origin can shape the investigation. Testing should inform the decision.
When an AI system exposes sensitive information, disrupts operations, or acts outside its assigned role, a technical issue becomes a leadership and reputation event.
Fortalice helps organizations identify those risks, strengthen governance, and make defensible AI decisions.
The safest AI strategy is one your organization can test, explain, govern, and defend.
Watch Theresa Payton’s full interview with Akash Pasricha at The Information for the complete conversation on open-source AI regulation, model testing, and governance.
Frequently Asked Questions
Is open-source AI safe for businesses?
Open-source or open-weight AI can be appropriate when the model has been tested in a controlled environment, identified vulnerabilities can be mitigated, updates can be managed, and the organization has strong internal governance and guardrails. The decision should be based on evidence from the specific model and the way the business plans to use it.
Should AI regulation depend on country of origin?
Country of origin can justify closer scrutiny. Theresa Payton argues that origin alone should not determine the conclusion. Organizations should test the specific model, identify vulnerabilities, review the results, and determine whether the risks can be mitigated.
What is risk-based AI regulation?
In this article, risk-based AI regulation means applying governance and guardrails according to the technology, its use, and its security and resilience implications. Systems used in more sensitive or consequential environments should receive stronger testing, oversight, and containment.
What is the difference between open-source and open-weight AI?
Open-weight models make their trained parameters available for use or modification. Open-source AI generally implies broader access to the components needed to study, modify, and redistribute the system under an open license. The exact level of openness varies by model.
What should executives evaluate before approving an open-source AI model?
Executives should evaluate the business purpose, data access, hosting environment, security testing, update process, monitoring, recovery plan, and ownership structure. They should also determine whether identified vulnerabilities can be mitigated and who has authority to restrict, replace, or stop the model.
Do closed AI models create security risks?
Yes. Payton says governance and guardrails should apply to open and closed models. Closed models can also create cost, vendor-dependence, resilience, recoverability, and supply-chain concerns, especially when one provider becomes essential to mission-critical operations.
Not sure whether an AI model is ready for your organization?
Fortalice can help you test the risks, put the right guardrails in place, and make a confident decision before it goes into production.
More From Theresa Payton and Fortalice on AI Risk
The OpenAI and Hugging Face Security IncidentSee what an AI evaluation breach revealed about containment, monitoring, and the risks inside advanced testing environments.
36 Months to Shape the Next 50 Years of AI: What Are the Risks of AI Agents?Read Theresa Payton’s perspective on autonomous AI, human approval, system access, and leadership accountability.
About Fortalice Solutions
Fortalice is a cybersecurity firm specializing in cyber incident response, cyber risk management, and cybersecurity for executives, chosen by leaders who need elite, discreet support when cyber incidents threaten operations, reputation, and leadership credibility.
Founded by former White House CIO Theresa Payton, who served in a position defined by trust, discretion, and decision-making at the highest levels, Fortalice brings national-level experience and seasoned judgment to high-pressure, time-sensitive situations where decisions cannot wait and mistakes are costly.
The firm integrates cyber advisory, cyber incident response, technical testing, executive digital protection, and training into a unified approach shaped by real-world incidents and human decision-making, delivering clear, actionable guidance trusted by both executive leadership and security teams.
Connect with Fortalice to ensure trusted, discreet expertise is in place before, during, and after a cyber incident.
