Auditing Clients Who Use AI

The previous lessons dealt with AI in your hands. This one deals with AI in the client's, which is the larger and less discussed problem. Clients are deploying models into revenue recognition, credit loss estimation, inventory obsolescence, and fraud screening, and those models are producing numbers that land in the financial statements you sign.

The first mistake is treating this as a technology topic. It is an ordinary audit problem — an estimate or a process affecting an assertion — with an unfamiliar mechanism. The second mistake is the opposite: treating it as entirely ordinary and missing the two things genuinely new about it, which are drift and opacity.

Scoping: Does the Model Touch an Assertion?

Clients use AI for many things that have no bearing on the financial statements. A marketing team using generative AI for campaign copy is not an audit matter. The scoping question is whether the system affects the initiation, recording, processing, or reporting of transactions, or produces an input to an amount or disclosure in the financial statements.

Systems that commonly do:

• Expected credit loss models under IFRS 9. The most consequential case in financial services and increasingly in corporates with significant receivables.
• Inventory obsolescence and net realisable value models driven by demand forecasting.
• Revenue recognition where an AI system determines contract terms, performance obligation satisfaction, or variable consideration estimates.
• Fair value estimation for assets without observable market prices.
• Automated contract review feeding lease classification or revenue treatment.
• Fraud and credit screening that determines which transactions are accepted, affecting the population of recorded transactions.

The mapping to make with the client is straightforward and worth doing formally: list the AI systems in use, and for each, identify the financial statement line items and assertions affected. Most entries on that list resolve to "none," and the exercise is worth it for the few that do not. Do it as part of understanding the entity under ISA 315, not as a separate technology review.

ISA 540 Applied to a Model

Where a model produces an accounting estimate, ISA 540 (Revised) applies and provides more structure than most auditors expect. It requires evaluating the method, the assumptions, and the data — and those three map cleanly onto a machine learning model.

The method. What kind of model, and is it appropriate for the estimate? A model optimised for ranking is not necessarily appropriate for producing an unbiased expected value. The relevant question is whether the method suits the measurement objective in the applicable financial reporting framework, and management should be able to explain why this method was selected.

The assumptions. In a machine learning model the assumptions are less visible than in a discounted cash flow, but they exist and they are usually significant. The training period embeds an assumption that it is representative of the forecast period. The feature set embeds an assumption about what drives the outcome. The loss function embeds an assumption about the relative cost of over- and under-estimation, which directly affects whether the estimate is neutral or biased. Management bias is precisely what ISA 540 asks you to look for, and in a model it hides in these choices rather than in a rate that has been nudged.

The data. Training data completeness, accuracy, and relevance. A credit loss model trained exclusively on a benign period has not seen the conditions that make the estimate matter.

The standard also requires evaluating whether the estimate is reasonable in the context of the framework, and considering indicators of possible management bias. Two model-specific indicators are worth having in mind: a model retrained or reconfigured close to period end, and a post-model overlay or management adjustment applied to the output. Overlays are legitimate and common, particularly in expected credit loss, but they are also the easiest place to introduce bias, because they sit outside the model's discipline entirely. An overlay should receive the same scrutiny as any other significant management judgment, and often more.

Controls: Where Conventional ITGC Scoping Falls Short

Standard IT general controls cover access, change management, and operations. Applied to a model, that framework has gaps, because a model can change materially without any code changing.

The controls that matter:

• Model change control. Approval and testing for changes to model logic, features, or hyperparameters. Usually covered by existing change management if the model lives in a controlled codebase, and frequently not covered at all if it lives in a notebook or a spreadsheet.
• Retraining governance. This is the gap. If a model retrains automatically on a schedule, its behaviour changes without any change event that conventional change management would capture. Who approves a retrain, what validation runs before the new version goes live, and is there a record of which version produced the period's figures?
• Data pipeline controls. Completeness and accuracy of the data feeding the model, which is an ordinary ITGC concern but frequently unscoped because the pipeline is newer than the systems around it.
• Output review. A human control over the model's output. Its effectiveness depends entirely on whether the reviewer has the information and standing to challenge, which is the automation bias problem covered in the next lesson.
• Version records. Whether the client can tell you which model version produced the estimate in these financial statements. If they cannot, you have a documentation problem that limits what you can conclude.

Drift: Why Last Year Does Not Carry Forward

The genuinely novel risk is drift. A model's performance degrades as the world moves away from its training data, and this happens without any change to the model. Nothing breaks, no error is raised, and the outputs remain plausible. A demand forecasting model driving inventory provisioning becomes progressively less accurate as the product mix shifts, and the provision becomes progressively less reliable while the process appears entirely stable.

For audit, drift has a specific consequence: a model validated in a prior year provides limited comfort in the current year. The conclusion that the model was appropriate last year is a conclusion about a relationship between the model and last year's conditions.

What to look for each period: whether management monitors model performance against actual outcomes, what threshold triggers investigation or retraining, whether that monitoring was performed for the current period, and what it showed. Back-testing the model's prior-period predictions against actual outcomes is the most direct evidence available, and where management does not do it, the auditor often can, using the entity's own realised data. It is one of the more powerful procedures available in this area and it is underused.