AI Governance Is an Inventory Problem: The Records Engineering Should Keep

When a legal or compliance team asks engineering about AI, the questions are almost always the same: What AI systems do we run? What can each one do? What data does it see? Who approved it, and how was it tested? What happens when it goes wrong? Very little of that is legal work. It’s records. Keeping those records makes the questions easier to investigate; the time required depends on the systems and evidence available.

This post is written for engineering leads, not lawyers. It is not legal advice. It provides an engineering evidence template. Use legal review to determine the obligations and dates for a specific system.

What changed in July 2026

The EU AI Act’s timeline moved this summer. The European Commission’s AI Act page, checked October 11, 2026, records that the AI Omnibus amendment entered into force on 27 July 2026 and pushed back the main high-risk obligations:

• High-risk systems listed in Annex III (employment, credit, education, essential services and similar): now 2 December 2027, previously 2 August 2026.
• High-risk systems that are safety components of products under Annex I: now 2 August 2028.
• Article 50 transparency obligations (telling people they’re interacting with an AI system, and labelling certain generated content) were not deferred and apply from August 2026; specific exceptions and transition arrangements require checking the enacted text for the use case.

Prohibited practices have applied since February 2025, and obligations for general-purpose model providers since August 2025. The prohibited-practice rules can affect deployers too; do not assume using a hosted model removes obligations.

For most teams building agents on top of hosted models, the practical position is: transparency applies now; high-risk obligations apply if your use case falls into an Annex III category, with more time to prepare than planned. Both need the same foundation. Outside the EU, the NIST AI Risk Management Framework with its Generative AI Profile and ISO/IEC 42001 are the common reference points. They overlap on governance evidence, but differ in scope and requirements. This inventory is not a compliance assessment for any of them.

The inventory comes first

You can’t govern systems you can’t list. An AI inventory should include everything that sends data to a model and uses the output. That covers the customer-facing agent, and it also covers the internal ticket summariser, the coding agents with repo and cloud access, and the vendor tool someone connected to the CRM.

Keep one record per system, in version control next to the code where possible, so it changes when the system changes. The fictional example below is a template, not an actual legal determination or AFL operating record:

# ai-systems/billing-investigator.yaml
name: billing-investigator
owner: payments-platform            # a team, not a person
purpose: >
  Investigates billing disputes and drafts a resolution for a human agent to send.
users: internal support staff
affected_people: customers with open disputes
eu_ai_act:
  annex_iii_category: pending legal classification
  article_50: pending applicability review
models:
  - id: anthropic.claude-sonnet-4-5-20250929-v1:0   # dated, not an alias
    provider: amazon-bedrock
    region: eu-west-1
data_in: [billing records, dispute messages, internal KB]
data_out: draft replies (not sent automatically)
actions:
  - tool: search_invoices      # read-only
  - tool: issue_refund         # irreversible; requires human approval > 50 EUR
permissions:
  role: arn:aws:iam::123456789012:role/agents/billing-investigator
  blast_radius: >
    If fully compromised: reads billing records for the EU tenant; can request refunds,
    above 50 EUR needing human approval. Cannot send email or modify accounts.
human_oversight: every refund over 50 EUR approved in the support console
evaluation:
  suite: evals/billing-investigator/
  last_run: 2026-08-14
  gate: blocks deploy below agreed thresholds
logging: agent_step events, 400-day retention; content logs 30 days
incidents: runbooks/billing-investigator.md

Every field here answers a question a governance review will ask, and nearly all of them are facts engineering already knows. The record just makes them findable and keeps them current.

Four records that carry most of the weight

Permissions and blast radius. What the system can do matters more to risk than what it’s meant to do. The IAM role, the tools, and a plain-language statement of the worst case under full compromise. If that statement is alarming, fix the permissions rather than the wording.

Evaluation evidence. A saved, repeatable eval suite with dated results tied to specific model and prompt versions. “We tested it” is not evidence. “This suite, run on this date against this version, with these results, gates deployment” is. See the evals pipeline post.

Human oversight, as built. Where people approve, review or can intervene, and what they see when they do. Frameworks ask about oversight. The honest answer is a description of the actual approval gate, not a policy statement.

Logs and incidents. Structured logs that can reconstruct what the system saw, decided and did, plus an incident process that adds a test case for every failure. This post covers the logging schema.

Make the record part of shipping

Governance documents written once for an audit are out of date within a month. Put the record where it can’t go stale quietly:

• It lives in the repository, and changes to models, tools or permissions update it in the same PR.
• CI checks that it exists, parses, and names a model ID that matches the deployed configuration.
• A new tool with side effects, or a change to the IAM role, requires a review from whoever owns AI risk. Use a CODEOWNERS entry with required code-owner review in the repository ruleset.

Where this stops

An inventory and good records make governance answerable. They don’t decide whether a use case is acceptable, which Annex III category it falls into, or what your obligations are in a specific jurisdiction. Those are legal and ethical judgments. The records make sure the people making those judgments are working from facts rather than guesses.