Anthropic is right: Agentic Analytics is not text-to-SQL. Here's how we built the stack around it.

On June 3, 2026, Anthropic published one of the most honest production write-ups on agentic analytics I've read: How Anthropic enables self-service data analytics with Claude. Jose Parreño wrote an excellent breakdown of it, centered on a point I wish more teams would tattoo on their roadmaps: plugging an LLM into your database and asking it questions is not an analytics strategy.

Parreño calls the output of that shortcut "analytics slop" — confident, well-formatted, plausible-looking answers that nobody has verified. Anthropic's own numbers explain why. Without the surrounding system, Claude scored no better than 21% on their internal analytics evals. With it, accuracy moved above 95% in aggregate and around 99% in some domains. The model didn't change. The stack around it did.

That distinction matters to us because it's the exact bet we made when building InsightNarrator. We're a Voice-of-the-Customer / CX analytics platform, which means our "warehouse" is the messiest data there is: open-ended survey verbatims, reviews, support tickets, social mentions — unstructured text where ambiguity isn't an edge case, it's the raw material. Reading Anthropic's post felt like reading a description of the architecture we had already converged on, from a different direction. This article maps their stack, layer by layer, onto what we built, with concrete cases from our own product.

Anthropic's thesis in 60 seconds

Anthropic identifies three failure modes that account for most wrong answers:

• Concept and entity ambiguity. "Active users" or "revenue" can map to dozens of plausible fields. The agent picks one. It's often wrong, and still looks professional.
• Data staleness. Schemas, definitions, and business rules change; yesterday's correct procedure quietly becomes wrong.
• Retrieval failure. The right answer exists, but the search space is so large the agent builds confidently on a near miss.

Their response is a four-layer stack: data foundations (canonical datasets, metadata as a first-class product), sources of truth (semantic layer first, lineage, curated reference docs), skills (procedural playbooks, not just prompts), and validation (offline evals, adversarial review, provenance footers, correction loops). SQL sits downstream of all of it.

Now here's the same stack, as it exists today in InsightNarrator.

Layer 1 — Data foundations: one canonical dataset, not forty candidates

Anthropic's most common failure was the agent mapping a concept to the wrong table among near-duplicates. Their fix: fewer, more heavily governed logical models.

Our equivalent starts at ingestion. Whether feedback arrives from a survey platform, a review connector, or a CSV upload, it is normalized into a single canonical dataset format with per-column statistics and an automated data quality report — completeness, valid ranges, anomalies — before any agent ever touches it. An agent asking about "detractor comments" doesn't face forty plausible tables; it faces one governed dataset with a documented grain: one row, one piece of customer feedback.

This is old-school data engineering, and Anthropic is explicit that it still applies. We'd go further for unstructured text: if your raw material is language, a canonical, quality-checked dataset is the only thing standing between the agent and garbage-in-garbage-out at scale.

Layer 2 — Sources of truth: the Knowledge Pack, and reference docs written the Anthropic way

This is the layer where the resemblance is almost uncanny.

Anthropic's agents are structurally required to consult a governed semantic layer first; raw SQL is the fallback. They also warn against letting the LLM invent the semantic layer itself — auto-generated metric definitions "encoded the very ambiguities we were trying to eliminate." Their recommendation: let Claude draft the documentation, let humans own the definition.

InsightNarrator's Knowledge Pack is exactly that contract. Every workspace carries a curated knowledge base of typed entries:

• metrics — approved business metrics with their exact SQL formulas
• glossary — what "NPS", "churn", or "active customer" means here
• rules — constraints the agent must follow ("exclude test responses", "last month means the last complete calendar month")
• caveats — documented data traps
• example queries — golden question-to-SQL pairs
• reference docs — per-dataset documentation

Two mechanics mirror Anthropic's guidance directly. First, when a dataset finishes processing, an LLM drafts its reference document automatically, with a fixed structure of Business Context, Entity Grain, Standard Hygiene Filter, Dimensions, Key Columns, Gotchas. We converged on those sections independently; Anthropic's post uses the same skeleton almost verbatim. Second, and this is the part Anthropic insists on, the draft stays out of the agent's context until a human approves it. Entries live in a draft → approved / needs_review lifecycle. The model writes the documentation; the human owns the meaning.

And the staleness defense: when a dataset's schema changes, its reference doc is regenerated and flagged needs_review. The doc can drift for a day, but it can't silently stay wrong — there's always a visible, human-owned checkpoint.

Layer 3 — Skills: tool packs and playbooks, not heroic prompts

Anthropic's largest accuracy jump, 21% to 95%+, came from skills: procedural knowledge about how to work, not just what data exists.

Our agent, Scoty, follows the same operating principle. It doesn't get a raw database connection and a prayer. It gets dedicated tool packs for common operations — dataset exploration, analysis results, topics and risks — that return structured, governed results. Free-form SQL exists, but as an explicitly discouraged fallback: read-only, restricted to an allowlist of data tables, used only when no dedicated tool covers the question. That's Anthropic's "semantic layer first, raw SQL fallback" pattern, enforced at the tool-description level rather than by convention.

For deeper work, there's a multi-agent pipeline: nine specialized agents — Data, Domain, Sentiment, Themes, Trends, Segments, Risk, Synthesis, and a final QA agent — each with a bounded iteration budget, passing findings forward. The 30+ predefined analysis types (driver analysis, churn signals, theme extraction, and so on) are, in Anthropic's vocabulary, encoded analyst playbooks: a senior CX analyst's procedure, made reproducible.

Layer 4 — Validation: adversarial review and a provenance footer on every answer

Anthropic's online validation rests on two ideas we implemented in the same spirit.

Adversarial review. Before an answer is delivered, an optional review pass challenges the draft's SQL and its assumptions — the same "don't self-certify" discipline as Anthropic's mandatory sql-reviewer sub-agent, which bought them +6% accuracy at real latency cost. We made it a toggle so users choose per question whether the stakes justify the extra pass. And the QA agent at the end of the pipeline plays the same role for long-form analyses: findings get challenged before they get saved.

Provenance footer. Every Scoty answer carries a persisted provenance record: which source tier it drew from (knowledge_pack vs. raw_exploration), a confidence rating derived from that tier, the SQL actually executed with row counts and timing, and the freshness of the underlying data. Anthropic's framing is exactly right: it doesn't make the answer more correct, it tells the reader how much to trust it. A "raw exploration, freshness unknown" answer is a signal to verify before forwarding to leadership. In a CX context, where a wrong number can route a budget, that footer is not decoration.

Three concrete cases

Case 1 — ABSA on NPS verbatims: governed meaning for unstructured text. A CX team runs Aspect-Based Sentiment Analysis on 40,000 survey verbatims. The failure mode Anthropic calls concept ambiguity has a nastier cousin here: an unconstrained LLM will invent aspect categories — plausible, inconsistent, and different every run, so wave-over-wave comparison becomes meaningless. Our answer is the curated taxonomy: the model can only assign categories and labels that exist in the workspace's approved taxonomy, never free-invented ones. The taxonomy is the semantic layer for text. Humans own the definitions; the model maps messy language onto governed meaning. When wave 2 arrives, "Pricing" still means "Pricing".

Case 2 — C-Risk on support tickets: finding where churn is born. A support organization runs Complaint-Risk scoring over ticket conversations. Risk factors come from the governed taxonomy, crossed with sentiment per category and subcategory, so instead of a flat "negative sentiment up 12%", the team sees that Billing → Refund delay is the specific cell generating severe-risk complaints this month, with severity levels computed from the data, not vibes. The output feeds dashboards and, because every analysis result lands in the shared Library, it becomes evidence the next analysis can cite rather than a screenshot in someone's slide deck.

Case 3 — Scoty answers "what's driving detractors this quarter?" A stakeholder asks Scoty the kind of question that usually produces analytics slop. Scoty first consults the Knowledge Pack: the approved reference doc for the NPS dataset, the approved definition of "detractor", the workspace rule for what "this quarter" means. Only then does it query. The answer comes back tagged knowledge_pack, high confidence, with the exact SQL and data freshness attached. If the knowledge base doesn't cover the question, it falls back to raw exploration, and the provenance footer says so, in effect telling the reader: verify me before you repeat me. That honesty is the whole point.

Where we're honest about the gap

Anthropic's post is also a useful checklist of what we haven't finished. A formal offline eval harness — question/answer pairs pinned to ground truth, run in CI against skill changes — is on our roadmap, not in production. Automated correction harvesting (turning "no, that's the wrong table" into a knowledge-base patch) is today a human workflow, not a scheduled agent. We'd rather say that plainly than claim a finished 95%. Anthropic's own drift story, 95% decaying to 65% in a month without maintenance, is the best argument that these loops aren't optional, and it's shaping our roadmap the same way it shaped theirs.

The takeaway

The most useful sentence in Anthropic's post is the simplest: text-to-SQL is an execution capability, not the product. The product is the system that collapses ambiguity into governed meaning before the query is written, makes the right answer discoverable, and flags when either has gone stale.

For Voice-of-the-Customer data, that system isn't a nice-to-have. It's the difference between an AI that tells you what your customers said and one that tells you a story that happens to sound good. We built InsightNarrator as the first one.

If you're running a VoC or CX program and want to see what governed agentic analytics looks like on your own feedback data, give InsightNarrator a try.

References

• How Anthropic enables self-service data analytics with Claude — Anthropic, June 3, 2026
• Jose Parreño, Anthropic is telling you that Agentic Analytics is not just text-to-SQL — Medium
• Building effective agents — Anthropic

---

First published on LinkedIn on 4 August 2026.