Reading an AI Proposal Critically
Deliberate Academy Editorial Team
Reviewed for accuracy and professional relevance
Enjoying the course?
Sign up free to track your progress and earn a verified certificate when you pass.
- Identify the four categories of claim that appear in almost every AI vendor proposal or internal business case, and which ones require independent verification
- Distinguish a specific, checkable AI performance claim from a vague or unfalsifiable one
- Recognize the most common ways an ROI projection in an AI proposal understates the true cost or overstates the benefit
- Apply a structured reading pass to a proposal so a first review takes minutes, not hours, while still catching the claims that matter
An AI vendor proposal for a customer-service automation tool includes the line: "Our AI resolves 87% of customer inquiries without human intervention, based on results from our enterprise customers." It sounds specific — an exact percentage, an implied evidence base — and specific-sounding numbers are exactly what make weak claims pass unchallenged in a leadership meeting. Which enterprise customers? Measured how? Resolved, or merely responded to? Compared against what previous baseline? The number is not false. It is also not yet meaningful, and the gap between those two states is where AI proposals most often get approved on the strength of language rather than evidence.
The Four Categories of Claim in Every AI Proposal
Performance claims. Statements about how well the AI performs a task — accuracy, resolution rate, time saved. These are the claims most worth pressing on, because they are usually measured under conditions that flatter the vendor: a curated test set, a best-case customer, a metric that sounds like the one you care about but is not quite the same thing.
Integration claims. Statements about how easily the system connects to your existing tools and data. These tend to understate real integration effort, particularly around data quality and format mismatches that only surface once implementation begins.
Risk and safety claims. Statements that the system is "safe," "compliant," or "enterprise-grade." These are frequently the least specific claims in the entire proposal, and the ones a leader is most likely to accept without a follow-up question, because pressing on safety can feel adversarial in a sales conversation. It should not — a vendor confident in their safety posture will have specific evidence ready.
Cost and ROI claims. Projected savings or return figures, usually built on an implicit assumption about adoption rate, data readiness, and ongoing maintenance effort that the proposal does not make explicit.
Treat the phrase "based on results from our customers" as a prompt for a follow-up question, not as evidence in itself. Ask: which customers, what was actually measured, over what time period, and compared against what baseline? A vendor with a genuinely strong result will have this detail ready. One without it is asking you to take the headline number on faith.
A proposal states an AI tool 'reduces processing time by 60%.' What is the most important follow-up question before treating this as a reliable input to a business case?
Select one answer.
Where ROI Projections Usually Break
Most AI ROI projections share three quiet assumptions that a leader should make explicit before approving budget: full adoption from day one (real adoption ramps over months, and partial adoption delivers partial value); clean, ready data (data cleanup and integration work is routinely the largest hidden cost in an AI project, and it rarely appears as a line item in the vendor's proposal); and no ongoing human oversight cost (someone must review outputs, retrain or reconfigure the system as needs change, and handle the exceptions the system cannot — that cost is real, recurring, and frequently absent from the vendor's ROI model entirely).
Ask for the ROI model's underlying assumptions in writing, not just the headline number. If a vendor cannot or will not show you the adoption curve, data-readiness assumption, and ongoing oversight cost behind their projection, treat the headline figure as marketing, not a business case.
The ROI Model That Assumed Perfect Data — Mid-Size Insurance Broker
Context
A vendor proposed an AI claims-triage tool projecting a 40% reduction in manual triage time, based on an ROI model the sales team presented in a single summary slide with no supporting detail.
Action
Before approving the contract, the COO asked the vendor for the model's underlying assumptions in writing. The vendor's model assumed claims data arrived in a single standardized format; the broker's actual claims data arrived from eleven different insurer partners in inconsistent formats, a fact the vendor's discovery call had not surfaced. The COO required a data-readiness assessment as a contract condition before final sign-off.
Outcome
The assessment found that roughly 30% of incoming claims data required manual reformatting before the AI tool could process it reliably — a cost entirely absent from the original ROI projection. The broker negotiated a phased contract with a smaller initial scope covering only the standardized-format insurers, avoided a mismatched full-scale deployment, and used the first phase's real results to build an accurate ROI case for the remaining scope a year later.
According to this lesson, why is a vendor's ROI projection often unreliable even when the headline percentage is technically accurate?
Select one answer.
Exercise
Your Task
Take an AI proposal or business case your organization has recently reviewed (or a plausible one if you do not have direct access). Sort its claims into the four categories from this lesson: performance, integration, risk/safety, and cost/ROI. For each performance and cost claim, write the specific follow-up question you would need answered before treating the claim as reliable. Flag any claim in the risk/safety category that uses a word like 'safe,' 'compliant,' or 'enterprise-grade' without a specific supporting detail.
Success looks like
- Every performance and cost claim has a specific, answerable follow-up question attached to it, not a general note that it "needs more detail"
- At least one vague risk/safety claim is identified and flagged for a specific follow-up
Watch out for
- Accepting a specific-sounding percentage as evidence in itself, without asking what it was measured against
- Treating "enterprise-grade" or "compliant" as meaningful without asking which specific standard or certification is being referenced
- Every AI proposal makes four categories of claim — performance, integration, risk/safety, and cost/ROI — and each has a specific, predictable weak point worth pressing on.
- A specific-sounding percentage is not the same as a verified, comparable claim — always ask what baseline and task the figure was measured against.
- ROI projections routinely assume full day-one adoption, clean data, and no ongoing oversight cost — three assumptions worth making explicit before treating a projection as a reliable business case input.
- Risk and safety claims are the vaguest and least-challenged claims in most proposals — a vendor confident in their safety posture will have specific evidence ready when asked.
- A structured reading pass — sorting claims into the four categories and asking one pointed question per category — turns proposal review from a vague impression into a defensible evaluation.