AI and Agentic Automation for Audit and Continuous Monitoring

AI audit automation is currently oversold in one direction and dismissed in the other. The vendor pitch implies an agent that audits your company. The sceptical response is that none of it is admissible as evidence. Both are wrong, and the useful territory in between is narrower and more specific than either.

What genuinely works today: testing full populations instead of samples, detecting anomalies across transaction volumes no team could review manually, monitoring controls continuously rather than annually, and drafting documentation that a human then corrects. What does not work yet: an AI conclusion standing as audit evidence without a controlled, reproducible process behind it.

AI audit automation in practice: from sampling to full population

Traditional audit testing samples because examining everything was impractical. That constraint has largely disappeared for structured data, and its disappearance changes what testing can conclude.

A sample of forty journal entries supports a statistical inference about the population. Testing every journal entry posted in the year, filtered for the characteristics that indicate risk: entries posted by unexpected users, entries to unusual account combinations, round-dollar amounts, entries posted outside business hours, entries at period end reversing shortly after, identifies specific items rather than inferring a rate.

This is not new in principle; data analytics in audit predates the current wave by decades. What language models change is the cost of building the analysis. Work that required a specialist and several weeks can now be specified, built and refined in days, which moves it from something reserved for large engagements to something available on ordinary ones.

Where language models genuinely add capability

The specific advantage is unstructured data, the material that resisted automation entirely.

• Contract review at scale. Extracting revenue recognition terms, change-of-control provisions, termination rights, liability caps and data-processing clauses across hundreds of agreements, producing a structured dataset from documents that were previously read one at a time or not at all.
• Free-text field analysis. Journal entry descriptions, expense narratives, support tickets and vendor names contain signal that keyword search misses because it depends on meaning rather than wording.
• Policy-to-practice comparison. Reading a written procedure and comparing it against evidence of what was actually done, at a volume that makes systematic comparison feasible.
• First-draft documentation. Process narratives, control descriptions and system descriptions drafted from interview transcripts and system evidence, then corrected by a person. This is straightforwardly a time saving, and it is where most firms should start.

Continuous controls monitoring

Annual controls testing tells you the control worked on the days you sampled. Continuous monitoring tells you whether it is working now, which is a different and more useful proposition, particularly for SOX programmes and SOC 2 observation windows, where a control failing in month two and discovered in month eleven is an exception in the report.

Controls that automate well: segregation of duties conflicts detected as they arise rather than in a quarterly review, new vendor creation matched against the payment file and against employee address data, access changes reconciled to approval records, journal entries flagged against defined risk criteria, and expense claims tested against policy thresholds.

The design constraint that determines whether this succeeds is alert volume. A monitoring system generating two hundred alerts a week gets ignored within a month, and an ignored control is worse than no control because it creates documented false assurance. Tuning to a volume the responsible person can genuinely review (and defining what happens to each alert) is most of the implementation work.

Making AI-assisted work defensible

The question every auditor and regulator will ask is how you know the output is right. The answer cannot be that the model is usually accurate.

What makes AI-assisted procedures defensible is the same thing that makes any procedure defensible, a controlled, documented, reproducible process:

• Deterministic where it matters. The extraction, filtering and matching logic is code, not a prompt. Language models are used for classification and extraction; conclusions come from rules that produce the same output every time.
• Validated against a known-answer set. Before relying on an automated procedure, it is run against a population where the answer was established manually, and the error rate in both directions is measured and documented.
• Human review of every exception. The system identifies; a person concludes. That boundary is not negotiable and should be visible in the workpapers.
• Version and change control over the automation itself. Which version of the logic produced which result, who changed it, when, and who approved it, the same change management discipline applied to any system relied upon.
• A retained audit trail. Inputs, logic version, outputs and dispositions preserved so the procedure can be re-performed.
• Data governance. What was sent to which model, under what terms, and whether the provider's agreement permits it. Sending client financial data to a consumer AI service is a confidentiality breach regardless of the quality of the analysis.

What is deliberately not claimed

An AI agent does not form an audit opinion, and any product suggesting otherwise is misrepresenting both the technology and the professional standards. Models produce confident output on questions they have handled poorly, which is precisely the failure mode that matters in assurance work. And a procedure that cannot be explained to an auditor in terms they can evaluate will not be relied upon, however well it performs.

The realistic position: AI reduces the cost of finding things by a large factor, and changes the conclusion process not at all.