The wrong answer costs the most.

Every Sunday I read release notes looking for the sentence a vendor writes when it is explaining a decision to itself. Databricks wrote one on July 16. Genie Code, the agent that authors dashboards, “now returns an error instead of retrying SQL queries that are not valid. This behavior reduces the cost of query errors.”

That is a bug fix justified in dollars. It is also the third cost-control move in six weeks. On June 25 scheduled tasks lost hourly and per-minute frequencies and were capped at daily. On July 8 Genie Agents went pay-as-you-go with 150 DBUs of free LLM usage per user per month, about $10.50 in US East. On July 15 all of it went free through July 31 with budget controls suspended. On July 21 correction records landed in system.billing.usage to negate what had already been billed, and some users who had crossed their budget threshold fell back under it.

Databricks turned the meter on, turned it off, and reversed the charges. Look at what the meter was pointed at. Not the question. The retry.

The teardown: Genie One and Genie Agents

Eval surface. The consumer surface ships no confidence score, no verification prompt, no hedge banner on an answer. The one documented accuracy caveat is about routing — workspaces with many Genie Agents “might experience reduced routing accuracy.” Benchmarks, monitoring, and a debug path all live on the authoring side, in the agent an analytics engineer configures. The inference, and it is inference: eval is owned one layer up from the person reading the number. That is a defensible split until the product starts drafting documents and emailing scheduled results, which it now does.

Cost exposure. The free allowance covers LLM usage only. Warehouse compute is billed separately and is explicitly outside Genie budgets. Block-usage thresholds enforce after a delay and do not interrupt requests already running. So the governed number is the small one. Agent mode — generally available since July 2, answering with multi-step reasoning, citations, visualizations, and tables — spends most of its money in the warehouse, where the budget is not looking. At list, $10.50 a month buys a person a handful of exploratory sessions. It does not buy a habit.

Latency shape. Ask Genie now caches its starter questions “to reduce latency,” and the ontology is credited with reducing it as well. Read that backwards: the cold path was slow enough that the vendor pre-computed the first click. Multi-step reasoning has to serialize — plan, query, read, re-query — and every hop is seconds to the user and DBUs to the account.

Data shape. Genie Ontology builds a map of the business from tables, queries, dashboards, notebooks, and connected apps, scoring each snippet by source, usage frequency, and freshness, resolving conflicts under Unity Catalog permissions. The weakest joint is that scoring. A wrong metric definition that is used constantly and was edited last week will outrank a correct one that sits unused.

The steelman is strong, and it is where the lesson lives. Databricks metered the agent, watched what customers actually did with it, and then ate the bill instead of letting it land. Capping schedules at daily, refusing the retry, and netting out the charges are three versions of one responsible instinct — an agent that can schedule itself is a background process holding a company card, and the vendor caught that before the invoice did. Plenty of vendors would have sent it and called the number consumption.

What I’d ship instead

Same user moment: someone asks your product a question in words, likes the answer, and asks to see it every morning. Two builds, two weeks each.

One: cost per accepted answer. Today you almost certainly log tokens. Tokens are not the unit that matters.

• Data shape. One row per agent turn: (turn_id, user_id, steps, retries, llm_tokens, warehouse_seconds, cost_cents, outcome). Outcome comes from an explicit accept control plus an implicit signal — a rephrase inside two minutes counts as a reject.
• System shape. Middleware around the existing loop. No extra model call. Compute cost joins in nightly from your warehouse’s own billing tables.
• Latency and cost. Async write, under 10ms on the request path; a table and a nightly job, under $50 a month at 5,000 seats.
• Eval. Forty golden questions with known answers, replayed weekly, scored against cost per accepted answer rather than accuracy alone.
• Two weeks in. You can name the ten questions eating a third of the spend, and retries per accepted answer is your leading indicator. The credible failure is that everything reads as accepted, because you shipped a thumbs-up and no rejection signal.

Two: schedule the query, not the agent. This is the direct read on the daily cap. When a user schedules a recurring answer, freeze the validated SQL from the turn that worked and run that. Call the model only when the number moves past a threshold, and only to write the sentence explaining it.

• Data shape. (schedule_id, frozen_sql, last_value, threshold, drift_state, last_validated_at).
• System shape. A query on a cron, plus a conditional single-call narration. No agent loop in the scheduled path at all.
• Cost. Roughly a tenth of re-running the agent daily, and a per-schedule monthly ceiling you can state out loud to a customer.
• Eval. Twenty frozen schedules replayed against last quarter, checking one thing: did the alert fire on the days a human would have flagged.
• Two weeks in. Most sends carry a number and no narration and nobody complains. The credible failure is schema drift rotting the frozen queries inside a month — in which case the fix is re-validation on drift, not putting the agent back in the loop.

Both builds make the bet Databricks’s own release notes make. The answer is cheap. The loop around it is expensive, and it is most expensive exactly when it is wrong. Price your loop before your customer’s finance team does it for you.

---

Sources

• AI/BI and Genie One release notes 2026 — Databricks Documentation, 2026-07-23
• AI/BI and Genie One release notes 2026 (Azure Databricks) — Microsoft Learn, 2026-07-23
• Manage budgets and cost controls for Genie — Databricks Documentation, 2026-07
• Genie pricing — Databricks, 2026-07
• Chat in Genie One — Databricks Documentation, 2026-07
• Genie Agents concepts — Databricks Documentation, 2026-07