Your ERP vendor just walked you through their AI roadmap: SAP Joule will automate bank reconciliations, Microsoft Copilot Finance will draft period-close commentary, and Odoo AI will handle customer follow-ups without human intervention. The demo was compelling. You said “interesting, we’ll look into it.”
That “we’ll look into it” deserves a structured answer before activating anything. Because the difference between an AI chatbot that suggests a phrasing and an AI agent that validates an invoice is not a difference of degree — it’s a difference in kind. In the first case, your user can ignore the suggestion. In the second, an agent error translates into a real payment, an accounting entry, a triggered purchase order.
This guide is for CIOs and CISOs at mid-market companies (100–1,000 employees) who are starting to receive AI activation proposals from their ERP vendors, and who want an operational framework to decide what to activate, with what guardrails, and how to audit what happens next.
Why AI Governance in ERP is Different from Classic IT Governance
AI Agents Act — They Don’t Just Advise
Classic IT governance deals with passive tools: software executes what humans command. An AI agent in 2026 is different. It has read-and-write access to ERP business objects, orchestrates sequences of actions, and makes decisions within the scope you’ve defined — sometimes without a human validating each step.
A concrete example: SAP Joule Studio allows building a production planning agent capable of validating and releasing manufacturing orders when stock and scheduling conditions are met (SAP, Production Planning and Operations Agent, 2026). This is a real capability, already generally available on S/4HANA Cloud. This isn’t a suggestion tool — it’s an actor in your value chain.
The governance question that follows isn’t “does this agent work?” (your vendor will show you it does). It’s “who is accountable when it makes a mistake, and how is that traced in the information system?”
The “Black Box” Risk: How Does an AI Agent Make Decisions in an ERP?
The language models powering ERP agents (OpenAI, Anthropic, Mistral depending on the vendor) don’t produce line-by-line explanation logs. SAP uses multi-model orchestration through its Generative AI Hub; Microsoft Copilot relies on OpenAI via Azure. In both cases, the model responds to a query — but the intermediate “reasoning” is not natively exposed in ERP audit logs.
For a CISO, this creates a blind spot: if the agent validates an unusual order, you have the outcome in the transaction log, but not the logical path that led the agent to that decision. This is precisely why AI governance cannot be limited to post-mortem analysis — it must integrate real-time monitoring and traceability by design.
What the EU AI Act Mandates vs. What Good Governance Recommends Beyond That
Regulation (EU) 2024/1689 on AI (the AI Act) imposes obligations on high-risk AI systems (Annex III) and transparency requirements for AI systems that interact with humans. For ERP modules, obligations vary depending on the nature of the processing: credit scoring on individuals, automated HR decisions, or predictive analysis on counterparty reliability.
But the AI Act sets a legal floor, not a ceiling of best practice. A customer follow-up agent that doesn’t fall under Annex III is not exempt from operational and reputational risks. The governance we describe here goes beyond minimum regulatory compliance — it addresses the operational maturity of an organisation integrating AI actors into its business processes.
For a deeper dive into the regulatory dimension, see our article EU AI Act and ERP: Compliance Guide for Businesses in 2026.
Mapping Your ERP’s AI Features by Risk Level
Before building a governance framework, you need to map what you have — or what you’re being asked to activate. AI features in modern ERPs are not homogeneous in terms of risk.
Level 1 — Assistive AI: Low Risk
These are features that suggest without deciding: rephrasing a product description, automatically summarising a supplier communication thread, proposing an accounting category on a scanned invoice that the user confirms with a click.
Characteristics: systematic human validation before any action, no automatic writes to the ERP, limited impact if an error occurs.
Minimum governance required: informing users that they are using AI (transparency obligation under Article 50 of the AI Act for chatbots), optional traceability.
Level 2 — Automation AI: Moderate Risk
These features execute recurring actions with periodic supervision, not real-time: automatic invoice matching in bank reconciliation, validation of a standard supplier invoice against predefined criteria (amount within threshold, known supplier, budget line not consumed), automated first-level dunning.
Characteristics: the agent writes to the ERP, but on bounded and repetitive transactions. Human supervision is non-real-time: batches are reviewed, not individual transactions.
Governance required: documented and audited triggering rules, consultable action logs, defined exception thresholds, weekly or monthly human review depending on volume.
Level 3 — Autonomous Agents: High Risk
These features operate on high-impact decisions: automated order intake based on commercial criteria, price adjustment based on real-time demand analysis, manufacturing order approval, recalculation of customer delivery timelines with automatic notification.
Characteristics: the agent acts on core business objects, with direct financial or customer impact. An error can propagate quickly before being detected.
Governance required: human validation above a defined threshold, comprehensive logging, rollback mechanism, mandatory sandbox testing before production.
Classification Matrix: Complete This Table for Your Current Features
| AI Feature | ERP Module | Risk Level | Current Supervision | Human Validation? |
|---|---|---|---|---|
| Period-close commentary draft | Finance | 1 | N/A | Yes, systematic |
| Automatic invoice categorisation | Accounting | 1-2 | Monthly log | Yes on exceptions |
| Automatic bank matching | Treasury | 2 | Weekly review | No (batch) |
| Automated customer dunning | CRM/Order Mgmt | 2-3 | Monthly review | No |
| Manufacturing order release | Production | 3 | Threshold alerts | To define |
| Real-time price adjustment | Sales | 3 | Not defined | No |
Reproduce this table for your features, and complete the “Current Supervision” and “Human Validation?” columns. Rows with Level 3 and “No” in the last column are your immediate governance priorities.
The 4 Pillars of the ERP AI Governance Framework
Pillar 1 — DECISION: Who Can Activate an AI Feature?
This question is not trivial. In many organisations, AI module activation in the ERP is done at the request of a business project manager or system administrator, without a formal validation process. Yet a Level 3 AI feature should go through a committee before any production activation.
Who sits on the AI activation committee? Minimum recommendation for a mid-market company: the CIO (responsible for IT system integrity), the CISO (security and data privacy risk assessment), the Business Owner of the domain concerned (finance, production, sales), and the DPO if personal data is involved.
Decision criteria to document for each feature:
- What is the scope of data accessed by the agent?
- What actions can the agent execute without human validation?
- What is the maximum impact of an error (financial, customer-facing, regulatory)?
- Does the vendor provide a native audit log?
- What is the unit deactivation mechanism?
Pillar 2 — SUPERVISION: Who Monitors AI Actions and How Frequently?
Supervision is not limited to “making sure it works.” It consists of verifying that the agent acts within the bounds of its triggering rules and does not produce unanticipated side effects.
Two supervision models exist, with different use cases:
Human in the loop: a human validates each action before it is executed. Appropriate for high-impact decisions (Level 3), during initial deployment phases, or for irregular processing.
Human on the loop: a human monitors current or past actions, with the ability to intervene or cancel. Appropriate for high-volume, low-unit-risk processing (Level 2 batches) where per-transaction supervision is not realistic.
The choice between the two must be explicit and documented in your ERP AI charter — not left to the discretion of the user or administrator.
Recommended supervision frequency by level:
- Level 1: quarterly review of usage statistics
- Level 2: monthly review of action logs, alert on anomaly above threshold
- Level 3: weekly review, real-time alert on threshold breach
Pillar 3 — AUDIT: How to Trace and Retain AI Decisions?
The obligation to maintain traceability comes from several sources simultaneously. The AI Act requires for high-risk systems (Annex III) automatic logging that allows reconstructing the context of each decision. Article 22 of the GDPR requires informing individuals of any fully automated decision affecting them and allowing them to contest its logic.
For CIOs, this translates into concrete requirements on log infrastructure:
- Who triggered the agent? (user identifier or automated process)
- Exactly when? (timestamp, not just the date)
- On which ERP objects? (references of impacted transactions)
- What decision was made? (validation, rejection, modification)
- With what outcome? (post-execution status, amount, state)
These 5 fields must be retained in line with your applicable retention policy (accounting data typically requires a minimum of 7 years under EU standards; verify local requirements). Verify that your vendor provides these logs natively and that they are exportable — do not rely on an online viewing interface that could disappear.
Pillar 4 — REMEDIATION: What to Do When AI Makes a Mistake?
Any governance policy that doesn’t include a remediation plan is incomplete. The question is not whether the AI agent will make a mistake — it’s when and how you’ll detect it.
Minimum remediation plan:
-
Detection: who receives the alert and within what timeframe? Define a channel (email, SMS, dashboard) and a maximum notification delay (e.g., any detected anomaly must be notified to the responsible person within 4 business hours).
-
Rapid assessment: is it an isolated error or a systemic one? A single transaction or an entire batch? Who decides to suspend the agent?
-
Suspension: each Level 2 and Level 3 AI feature must have a unit off-switch — not a global “disable all AI in the ERP” switch, but a mechanism to deactivate that specific module without impacting others.
-
Correction: who corrects the impacted transactions? Within what timeframe? With what audit trail to distinguish human corrections from the agent’s original actions?
-
Post-mortem: each significant AI incident (financial impact above threshold, potential GDPR incident) must be documented in a post-mortem within 5 business days.
Practical Implementation — The First 6 Weeks
Weeks 1-2: Inventory of Active and Upcoming AI Features
Start with a simple question: what is already active in your ERP? The answer will often surprise you. AI features are sometimes activated by default during vendor updates, without explicit notification.
Actions:
- Ask your vendor or integrator for a comprehensive list of AI features included in your licence, active or available
- Interview your business administrators about AI features they are using or have tested
- Document each feature in the classification table above (risk level, current supervision)
Week 3: Classification and Stakeholder Identification
Based on the inventory, classify each feature according to the 3 risk levels. For each Level 2 or Level 3 feature, identify:
- The business owner (who benefits from the feature)
- The supervision owner (who reviews the logs)
- Personal data potentially processed (to involve the DPO)
Weeks 4-5: Drafting the ERP AI Charter
The ERP AI charter is not a 50-page document. It is a 3-to-5-page operational document that defines:
- Activation rules (who decides, based on what criteria)
- Supervision matrix (who monitors what, at what frequency)
- Log and retention obligations
- Remediation plan (who does what in case of an incident)
- Explicit restrictions (prohibited features, data outside AI scope)
Have the charter validated by the CIO, CISO, DPO, and senior management. An unvalidated document is not a charter — it’s an intention.
Week 6: First AI Governance Committee + Review of Action Logs
Organise a first AI governance committee with the identified stakeholders. Agenda:
- Presentation of the inventory and risk matrix
- Validation or adjustment of the ERP AI charter
- Review of logs for already-active AI features (what has happened?)
- Decision on Level 3 features: conditional activation, deferral, or refusal
This first committee sets the cadence. In practice, a quarterly meeting is sufficient for organisations with only Level 1-2 features; a monthly meeting for those deploying Level 3 agents.
Common Mistakes When Deploying AI Agents in Your ERP
Activating an Autonomous Agent Without a Sandbox
A Level 3 agent is not tested in production. Before any production activation, the feature must run in a sandbox on representative data for a minimum of 4 weeks, with a manual comparison of its decisions against what a human would have done.
If your vendor does not have a sandbox environment for AI agents (this is sometimes the case with shared cloud offerings), this is a blocking point to resolve before any deployment.
Overlooking Data Access Rights: Does the AI Inherit the User’s Permissions?
In most current ERP architectures, an AI agent authenticates with a service account or with the permissions of the user who activated it. In the second case, if a sales director activates a quote management agent, the agent could potentially have access to the entire price catalogue, margins, and customer conditions of that director.
Explicitly verify your AI agent’s authorisation model: what ERP rights are associated with it, whether they can be restricted relative to the activating user, and whether the principle of least privilege is applied.
Not Planning a Kill Switch — AI Agents Must Be Deactivatable Individually
A global “disable all AI in the ERP” switch is insufficient and operationally dangerous (it also disables useful Level 1 features). Each autonomous AI agent or feature must be deactivatable independently, without a system restart, and this deactivation must be traceable (who did it, when, why).
Verify this capability during the pilot phase, before deploying to production.
Ignoring GDPR Traceability Obligations on Automated Decisions
Article 22 of the GDPR governs decisions based solely on automated processing that produce legal effects or significantly affect a natural person. In an ERP context, this covers: automated customer credit granting or refusal decisions, solvency scoring of individual business owners, or automatic triggering of a debt collection procedure.
If your AI agent makes such decisions, you must: inform the person concerned (transparency), provide a mechanism for contestation and human review, and document the decision logic in your data processing register (maintained by the DPO).
ERP AI Charter Template (Simplified Structure)
Here is the minimum structure for an ERP AI charter. Adapt it based on your organisation’s size and maturity.
ERP AI GOVERNANCE CHARTER — [Company Name] Version 1.0 — Approved by: [CIO] [CISO] [DPO] [CEO]
1. Purpose and Scope This charter defines the governance rules applicable to the activation, use, and supervision of artificial intelligence features embedded in [ERP Name]. It applies to all AI modules and agents integrated into the company’s management system.
2. Risk Levels and Activation Rules
- Level 1 (assistive): activation by the system administrator on business request.
- Level 2 (automation): activation by joint decision of CIO + Business Owner, with documentation of triggering rules.
- Level 3 (autonomous agents): activation by decision of the AI Governance Committee, after sandbox validation, with DPO involvement if personal data is involved.
3. Supervision Responsibilities [Table CIO/CISO/DPO/Business Owner vs. Features with review frequency]
4. Log and Retention Obligations Every action by a Level 2 or Level 3 agent is logged with the 5 fields defined in the audit policy. Logs are retained for [X years] in accordance with the company’s retention policy.
5. Remediation Plan In case of an AI incident: notification within [X hours], suspension decision within [X hours], correction within [X business days], post-mortem within 5 business days if impact exceeds [financial or GDPR threshold].
6. Explicit Restrictions The following AI features are not permitted on the company’s IT systems: [list to complete based on your context].
What You Should Do This Week
An ERP AI governance framework should not wait for your next ERP update or your next vendor presentation. Start with the inventory: ask your IT team which AI features are currently active in your ERP. The answer — whatever it is — is your starting point.
Organisations that address AI governance before deployment avoid costly incidents and compliance retrofits. Those who wait for a problem before structuring their governance end up doing the work twice — with urgency added.
For related topics, see our 2026 comparison of agentic AI in ERP and our guide on GRC integration in ERP.
