Natural Language to SQL — Querying Data Without Writing It By Hand
Deliberate Academy Editorial Team
Reviewed for accuracy and professional relevance
Enjoying the course?
Sign up free to track your progress and earn a verified certificate when you pass.
- Identify the query types where natural language to SQL performs reliably in a BI context versus the query types where it fails silently
- Apply a three-step verification standard to any AI-generated SQL before it feeds a dashboard, metric, or self-service report
- Explain why AI SQL tools like ChatGPT, Claude, and platform-native copilots depend entirely on the schema context they are given, and what happens when that context is incomplete
- Distinguish low-risk, high-value uses of AI-generated SQL from high-risk uses that require full verification before shipping to production
Natural language to SQL, usually shortened to NL2SQL, lets a BI developer or business user describe what they want in plain English and get back a runnable query. General-purpose tools like ChatGPT and Claude can generate SQL from a pasted schema and a natural language request. Platform-native tools go further: Power BI Q&A and Tableau Ask Data translate a typed question directly against a connected semantic model. Both categories share the same underlying risk: the SQL that comes back can be syntactically perfect and answer a subtly different question than the one that was asked, and nothing in the output tells you that happened.
Where NL2SQL Performs Reliably in a BI Context
NL2SQL tools are strongest on well-defined, single-purpose queries against a data structure the model can see clearly. Simple aggregations with an unambiguous grouping field -- total revenue by region, count of active tickets by status -- resolve correctly the large majority of the time. Trend queries over a clearly named date field, and filter queries with explicit, stated conditions, are similarly reliable. These categories account for a large share of the ad hoc query volume a BI team handles day to day, which is exactly why NL2SQL adoption has moved so fast inside BI teams: the highest-frequency requests are also the lowest-risk ones to hand to AI.
Reliability drops sharply outside that zone. Multi-table joins where the join key is not obvious from column names, metrics with a business definition that is not encoded anywhere in the schema, and any query that depends on institutional knowledge not visible in the data model are the conditions under which NL2SQL tools produce fluent, wrong answers.
Where It Fails, and Why the Failure Is Silent
Business logic embedded in data, not in the schema. A "closed" support ticket might mean status = 4 in one system's convention, while a human agent knows that status = 7 (escalated-then-resolved) should also count as closed for reporting purposes. An AI tool sees the column and its literal values. It does not know the operational convention layered on top of them unless that convention is written down somewhere the tool can read.
Ambiguous metric names. When a semantic model or database has both "Revenue" and "Net Revenue" and "Total Revenue" columns representing three subtly different calculations, an NL2SQL tool resolves a plain-English question about "revenue" to whichever field its matching logic favors -- and a different phrasing of the same question can resolve to a different field. Two people asking what looks like the same question get two different, both-plausible numbers.
NULL handling. SQL's NULL semantics are unintuitive even for experienced analysts, and AI-generated SQL regularly mishandles them in WHERE clauses and aggregations. A WHERE clause built to exclude a specific status code will silently also exclude every row where that field is NULL, quietly dropping records a business user would expect to see included.
Join logic across three or more tables. The more tables a query touches, the more assumptions an AI tool has to make about which keys relate to which, and the more likely a plausible-looking but wrong join produces a subtly inflated or deflated result — one where row counts look reasonable but are not correct.
Before asking an AI tool -- ChatGPT, Claude, or a platform-native NL2SQL feature -- to generate a query, spend thirty seconds stating the business definition of any ambiguous term in your request explicitly: what counts as "active," how a specific status code maps to a business state, and how NULLs in the relevant columns should be treated. This single habit resolves the majority of the wrong-answer cases this lesson describes, because it removes the ambiguity the AI would otherwise have to guess at.
A revenue number that was correct SQL and the wrong metric
Context
An analytics engineer used ChatGPT to generate a query answering a stakeholder's question: what was total revenue by customer segment last quarter? The generated SQL ran without errors and returned figures that looked reasonable. The engineer shared the output directly in a Slack thread because the request seemed routine.
Action
A finance team member flagged that the numbers were roughly 12% higher than the finance system's own segment totals. The engineer traced the gap: the orders table contained a refunds_applied flag, and the AI-generated query had summed gross order value without excluding refunded orders, because nothing in the natural language request or the schema comment mentioned refunds. The engineer rewrote the request to explicitly state that revenue should be net of refunds and specify the exact column to check, then re-ran the query and validated the new total against the finance system's figure for one segment before sharing the corrected numbers.
Outcome
The corrected query became a saved, documented query in the team's shared query library with the refund-handling logic spelled out in a comment. The engineer added a standing check to the team's process: any AI-generated query touching a revenue or financial metric is validated against a known reference figure for at least one segment before being shared outside the analytics team, regardless of how routine the original request seemed.
An analytics engineer asks ChatGPT to generate a query for 'total revenue by customer segment last quarter.' The query runs without errors and returns a plausible-looking number that is later found to be 12% too high because it did not exclude refunded orders. What is the root cause of this error?
Select one answer.
The Three-Step Verification Standard for AI-Generated SQL
Every AI-generated query that will feed a dashboard, report, or shared metric should pass through the same three-step check before it is trusted. First, read the generated SQL and confirm the join logic and filter conditions match your understanding of the data model -- this catches a meaningful share of errors before the query even runs. Second, run the query against a small, known sample and compare the result to a value you can verify by hand or against a trusted reference. Third, once the sample checks out, run the full query and sense-check the aggregate against a benchmark you already trust, such as a prior period's reported figure or a number from an adjacent system.
This is not a one-time onboarding habit -- it is the standing discipline that separates AI-assisted SQL work from AI-generated SQL you have simply trusted. The time cost is small relative to the cost of a wrong number reaching a dashboard a director checks every Monday morning.
A query that runs without an error message has told you nothing about whether it answers the right question. SQL syntax errors announce themselves. Logic errors -- wrong joins, mishandled NULLs, an unstated business definition -- do not. Treat "it ran successfully" and "it is correct" as two entirely separate claims that require two entirely separate checks.
Low-Risk, High-Value Uses of AI-Generated SQL
Not every AI SQL use case carries the same risk. Asking AI to explain an existing, undocumented query written by a former team member in plain English is high-value and low-risk: there is no data retrieval risk because no new query is being executed against production data, and the output accelerates documentation and knowledge transfer for junior BI developers. Similarly, using AI to draft a first-pass query for your own exploratory analysis -- one you intend to inspect and refine before it touches anything stakeholder-facing -- carries low risk because the verification step is built into your own workflow by design.
The risk profile changes the moment AI-generated SQL feeds a metric, dashboard, or report that a business user will see and act on without independently checking it. That is the point at which the three-step standard becomes mandatory rather than optional.
A BI developer wants to use AI to accelerate two different tasks: (1) explaining an undocumented legacy query so a junior teammate can learn from it, and (2) generating a new query that will feed a live executive dashboard. How should the developer's verification approach differ between these two tasks?
Select one answer.
Exercise
Your Task
Take a natural language question you or a stakeholder asked recently that required a SQL query -- ideally one involving at least one join and one filter condition with business logic behind it (a status, a category, an active/inactive flag). Write the plain-English request exactly as it was originally asked, generate SQL from it using an AI tool, and then rewrite the request to explicitly state the business definition of every ambiguous term. Run both versions against a small sample and compare the results. Note any discrepancy and identify which of the four failure modes in this lesson caused it.
Success looks like
- You have both the original and the clarified natural language request, plus the SQL each produced
- You ran both queries against a small, verifiable sample and can state whether the results differed
- If a discrepancy exists, you can name the specific failure mode (embedded business logic, ambiguous metric naming, NULL handling, or join logic) that caused it
Watch out for
- Testing only on a large dataset where a small percentage error is not visible without a benchmark to compare against
- Accepting a query as correct because it "ran without errors" rather than validating the result against a known reference figure
Hint
Choose a request involving a status or category field if you can — ambiguous business logic in categorical fields is the single most common source of the silent errors this lesson describes.
- NL2SQL tools -- both general-purpose (ChatGPT, Claude) and platform-native (Power BI Q&A, Tableau Ask Data) -- perform reliably on simple aggregations, trend queries, and explicit filter queries, which cover a large share of routine BI request volume.
- The primary failure modes are business logic embedded in data but not the schema, ambiguous metric naming, mishandled NULLs, and multi-table join assumptions -- all of which produce fluent, wrong answers without any error message.
- The three-step verification standard -- read the logic, validate on a known sample, sense-check the aggregate -- is the minimum process before any AI-generated query feeds a dashboard, report, or shared metric.
- Stating the business definition of ambiguous terms explicitly in the natural language request resolves most of these failure modes before the query is even generated.
- Verification effort should scale with consequence: explaining existing SQL is low-risk, while generating SQL for a stakeholder-facing dashboard always requires the full standard.