The Basics
- Generative Artificial Intelligence introduces new risks, such as inaccurate outputs, data exposure, bias, and rapid change, that traditional internal controls cannot fully address.
- COSO’s Internal Control – Integrated Framework offers a proven foundation for governing generative AI.
- This article provides practical steps organizations can use to minimize risk when using generative AI.
______________________________________________________________________________________________
Generative AI: A Solution That Brings New Challenges to Traditional Internal Controls
Generative AI is rapidly moving from experimentation to real business use. Your employees may already be using it to summarize contracts, support forecasting, automate workflows, draft communications, and analyze large sets of data.
That speed creates opportunity, but it also raises a serious question: Can your organization trust Generative AI’s outputs enough to rely on them?
Generative AI can improve efficiency and insight, but it can also introduce new forms of risk, including inaccurate outputs, unreliable or factually correct information, privacy concerns, bias, and changes that happen faster than governance processes can keep up. Organizations that use AI well need more than enthusiasm. They need structure.
That’s where internal control comes in.
In this article, you’ll learn:
- Why internal control matters for generative AI.
- What risks organizations need to address first.
- How COSO’s framework can be adapted for AI use cases.
- What practical steps leaders can take now to build confidence.
Why Generative AI Needs Stronger Internal Controls
Generative AI is different from traditional software. Most business systems are designed to produce the same result when given the same inputs. Generative AI does not work that way. It is “probabilistic,” which means it can produce variable outputs, even when the question appears similar.
That creates a different control challenge.
A generative AI tool may sound confident while producing inaccurate information. It may rely on incomplete source material. It may change behavior after an update to the tool. It may even be adopted by employees outside approved channels, creating “shadow AI” use that bypasses formal controls.
These are not small issues. If AI outputs influence business decisions, financial reporting, compliance activity, or customer communication, control gaps can quickly become serious problems.
Key Takeaway: If your organization is using generative AI in any meaningful way, internal controls should not be treated as a future-state issue. It is a current-state requirement.
The Top 5 Risks Generative AI Introduces
Before your organization can implement the right controls, you need to understand where AI risk shows up in practice.
While every use case is different, the primary issues increasing risk in organizations are:
1. Reliability and Accuracy
Generative AI can produce outputs that look polished and useful but are factually wrong. This is called a hallucination. In a business setting, hallucinations could result in an incorrect summary of a contract, a flawed forecast narrative, or an unsupported recommendation.
A finance team, for example, may use AI to draft management commentary for monthly reporting. If the tool introduces a false explanation for margin changes, and no one catches it, the issue can lead to an incorrect decision.
2. Data Quality and Source Integrity
AI systems are only as strong as the information they use. If source data is incomplete, inconsistent, outdated, or poorly governed, output quality will suffer. In many cases, organizations also struggle to understand and document the source AI pulled information from and how it used it.
That matters for accountability, transparency, and auditability.
3. Security and Privacy
Generative AI creates new entry points for data exposure. Employees may upload sensitive information into tools that are not approved for regulated or confidential data. Prompt injection attacks can also manipulate how tools behave or what they reveal.
For organizations operating in regulated environments, this risk is especially important. Internal controls help define what data can be used, where it can be used, and who has access.
4. Bias and Fairness
AI models can reflect bias from training data, source materials, or design choices. That can affect outputs in ways that are difficult to detect without deliberate testing and oversight.
Bias is often discussed as an ethical issue, but it is also a business issue. It can increase legal exposure, damage trust, and lead to poor operational decisions.
5. Third-party and Change risk
Many organizations rely on third-party AI vendors. That means key components of the system may change outside the organization’s direct control. A vendor update, new model version, or revised retrieval setting can shift output behavior quickly.
Without clear monitoring and change management, teams may continue relying on a tool that no longer performs as expected.
How COSO Can Help Protect Organizations that Use Generative AI
The good news is organizations do not need to invent an entirely new control model. The Committee of Sponsoring Organizations of the Treadway Commission (COSO), a globally recognized organization dedicated to helping entities reduce fraud while improving operations and oversight, has introduced an Internal Control – Integrated Framework.
COSO’s Internal Control – Integrated Framework provides a strong foundation comprising five components it deems crucial to effective internal controls in the age of generative AI:
- Control Environment
- Risk Assessment
- Control Activities
- Information and Communication
- Monitoring Activities
When adapted thoughtfully to your organization’s unique processes, these components can help your organization govern generative AI in a way that supports innovation and protects business objectives.
Best Practices for Implementing the COSO Components
To effectively implement each of the five COSO components, organizations must approach them with strategic planning and practical implementation.
Here’s an introduction to tackling each component, ensuring they align with modern business needs:
1. Control environment: Set the tone for responsible AI use
A strong control environment defines expectations before problems arise. With generative AI, that means leadership should make it clear that speed and experimentation do not replace accountability.
Organizations should establish:
- Clear acceptable use policies for AI tools.
- Defined ownership for each AI use case or platform.
- Role-based responsibilities for development, review, and approval.
- Training tailored to technical users, business users, and reviewers.
- Consequences for misuse or poor oversight.
This is where integrity becomes visible. If an organization says responsible use matters, its policies, oversight structure, and leadership behavior should reflect that.
A practical example: If a legal summary tool is used to review contracts, someone should own that tool, define approved document types, oversee changes, and confirm users understand its limits.
Key takeaway: Governance starts with clarity. People need to know what is allowed, who is responsible, and when human judgment must override automation.
2. Risk assessment:Identifywhere AI can go wrong
Risk assessment for generative AI should be active, specific, and ongoing. Annual review cycles alone are not enough. Models, data sources, prompts, and vendor settings can change too quickly.
Organizations should begin by asking a simple question: Is generative AI the right tool for this task?
In some cases, traditional automation or rules-based systems may be more reliable and easier to control. If AI is appropriate, the next step is to assess risk based on the use case.
That includes evaluating:
- The objective of the tool.
- The business impact of wrong outputs.
- The likelihood of hallucinations or “drift,” when the input data’s distribution shifts away from what the model initially encountered during training, leading to inaccuracies.
- Data sensitivity and privacy exposure.
- Vendor dependence.
- Fraud and manipulation scenarios.
- Regulatory or financial reporting implications.
For example, an AI tool that drafts internal brainstorming notes carries a different risk profile than one that supports reconciliations or compliance monitoring.
This is where business wisdom matters. Strong risk assessment is not about slowing everything down. It is about applying the right level of discipline to the right level of risk.
3. Control activities: Build safeguards into the process
Control activities are the practical actions that reduce risk. For generative AI, these controls should reflect how the tool is used, how much reliance is placed on it, and how quickly errors could spread.
Common control activities include:
- Human review for high-risk outputs.
- Confidence thresholds before automation is allowed.
- Approval workflows for prompts, rules, and model changes.
- Segregation of duties between those who configure and those who approve.
- Side-by-side testing before deployment.
- Source citation requirements for key outputs.
- Logging of prompts, outputs, and model versions.
- Rollback plans if performance declines.
Consider an AI-enabled reconciliation process. If the system is allowed to auto-post entries, there should be posting thresholds, clear exception routing, and multi-layer approval for any changes to those thresholds. If not, a small configuration issue could affect a large volume of transactions.
The goal is not to eliminate automation. It is to ensure automation remains trustworthy.
4. Information and communication: Make AI use traceable
Many AI-related failures become harder to manage because organizations cannot easily answer basic questions, such as:
- What data did the tool use?
- Which model version generated the output?
- Was the output reviewed?
- Were limitations communicated to users?
- Did anyone know the tool changed?
Strong information and communication practices help solve this.
Organizations should maintain records of prompts, inputs, outputs, model configurations, source references, and known limitations when appropriate to the use case. They should also communicate changes clearly to the people affected.
For example, if an AI summarization tool used by compliance teams receives a model update, reviewers should know what changed, when it changed, and whether performance was revalidated before continued use.
Internal communication matters just as much as documentation. Users, reviewers, managers, and governance teams all need the right level of information to make sound decisions.
Key takeaway: If AI supports an important process, its use should be understandable, traceable, and communicated clearly enough to support confidence and oversight.
5. Monitoring activities: Keep controls current as AI evolves
Generative AI cannot be controlled with a “set it and forget it” mindset. Even a well-designed control can weaken over time as the model changes, the data shifts, or users begin working around the process.
Monitoring should include both ongoing review and periodic assessment.
Ongoing monitoring may track:
- Accuracy rates
- Exception volumes
- Drift indicators
- Response quality
- Bias metrics
- Security incidents
- User behavior patterns
Separate evaluations may include:
- Periodic control testing
- Historical back-testing
- Independent challenge reviews
- Adversarial testing
- Validation after vendor updates
A useful example is an expense monitoring model that performs well at launch but slowly declines as employee spending patterns change. Without monitoring thresholds and formal retraining triggers, the decline may go unnoticed until reporting or compliance issues emerge.
This is where client success comes into focus. Monitoring is not just about catching failure. It is about sustaining value over time.
Common Gen AI Mistakes Organizations Make
As companies expand AI use, several control issues tend to appear early. Here’s what to look out for and avoid:
- Treating AI like standard software: Traditional IT controls still matter, but they are not enough on their own. Generative AI introduces issues like probabilistic outputs, prompt-based configuration, and source reliability that require added attention.
- Allowing adoption before ownership is clear: If no one owns the use case, no one owns the risk. Every meaningful AI tool should have a knowledgeable business owner that has authority and is accountable.
- Over-relying on outputs without review: Even useful tools can produce flawed results. High-impact outputs should be validated by qualified reviewers, especially when they affect reporting, compliance, or external communication.
- Failing to update controls as the tool changes: A control that worked at implementation may not work six months later. Monitoring, change control, and revalidation are essential.
A Practical Path Forward for Gen AI Governance
Organizations do not need to solve every AI governance issue at once. A more effective approach is to build control maturity in stages.
Here is a practical series of steps to start with:
- Inventory current AI use cases across departments, including unofficial tools.
- Classify each use case by purpose, data sensitivity, and business impact.
- Assign ownership for each approved tool or process.
- Assess risk using COSO’s five components as the foundation.
- Design and document controls based on the level of reliance and risk.
- Train users and reviewers on expectations, limitations, and escalation paths.
- Monitor performance and update controls as use cases evolve.
This kind of structure helps organizations move from reactive concern to proactive governance.
Next Steps
Achieving effective internal control over generative AI is not about resisting change. It is about making sure change creates value without compromising trust, compliance, or decision quality.
COSO’s framework gives organizations a principle-based approach to govern AI use with discipline and flexibility. When companies pair that structure with integrity, a focus on client success, and sound business judgment, they are better positioned to use generative AI confidently and responsibly.
If your organization is evaluating how to use generative AI, now is the time to apply the COSO principles to lay the foundation in the control environment. Start by identifying where AI is already influencing decisions, then assess whether your controls are keeping pace. That foundation can help you move forward with greater confidence and clarity.
Don’t let your controls lag behind your AI adoption
Evaluating where AI influences your business decisions is a critical first step — but you don’t have to navigate the complexities alone. At Rehmann, we believe effective governance should do more than reduce risk. It should help you move forward with confidence, integrity, and clear business purpose. Schedule a readiness assessment with our risk advisory team today to ensure your organization moves forward responsibly and securely.
Frequently Asked Questions
Q. What is “shadow AI” and why is it a risk?
A. Shadow AI refers to generative AI tools that employees adopt outside of approved channels, bypassing formal controls. It is a risk because unsanctioned use can expose sensitive data, introduce unreviewed outputs into business decisions, and leave organizations unable to track how AI is influencing operations.
Q. How often should organizations review their generative AI controls?
A. Generative AI controls should be reviewed continuously rather than on a fixed annual cycle, because models, data sources, prompts, and vendor settings can change quickly. Most organizations combine ongoing monitoring—such as tracking accuracy and drift—with periodic evaluations like control testing and validation after vendor updates.
Q. Can existing internal control frameworks govern generative AI, or is a new one needed?
A. Existing frameworks like COSO’s Internal Control – Integrated Framework can effectively govern generative AI without the need to create a new model. The key is adapting how each of the five COSO components is applied to account for AI-specific factors such as probabilistic outputs, source reliability, and third-party change risk.