Bring the SQL, the chatbot and the dbt project you already have

The fastest migrations we have seen are the ones that move what already works and govern it on the way in. A report's SQL keeps its logic; it gains declared parameters, a pinned response shape, grants and a ledger. A chatbot keeps its client; its answers start following your agreed definitions. Here are the four routes in.

1. Your templated SQL, as written

A data API's template can be Jinja. Paste the query a report already runs — {% if %} blocks for optional filters, {% for %}, {% set %} — and it is saved, versioned and shown exactly as written. The quoting it already does is understood: '{{ x }}' is a value, '%{{ x }}%' a LIKE pattern, and ('{{ xs | join("','") }}') an IN list. Every {{ … }} is still bound as a quoted, escaped literal, so the template cannot be talked into running something else.

What Jinja may not do is choose an identifier. A table, a column or a sort expression never comes from the request: | safe and | raw are refused, and choosing among fixed alternatives is written as {% if sort == 'premium' %}. A name the template uses but no parameter declares is reported when you check it and refused when you publish.

2. Let an agent do the provisioning

Moving fifty report queries by hand is a week of copying. Everything the UI does is an API call, and the deployment ships a provisioning guide written for a coding agent: which key to use, which calls to make in which order, and the traps. Point an agent at the client's existing queries and API specs and it can create each data API, have the API type the parameters from an example payload, check it against the warehouse, and publish it — then build the search index the client's advanced search page needs and the insights for its client and broker views.

sequenceDiagram
  autonumber
  participant A as Coding agent (admin key)
  participant C as InsightCompose
  participant W as Warehouse
  A->>C: POST /api/queries/build/params  (template + example payload)
  C-->>A: Typed parameters, lists as multi
  A->>C: POST /api/queries  (template, params, axes)
  A->>C: POST /api/queries/{slug}/check
  C->>W: Run once with the examples
  C-->>A: Columns, a sample row, any problems
  A->>C: POST /api/queries/{slug}/publish
  C-->>A: v1 at /api/v1/queries/{slug}/results
                

The agent holds an administrator-scoped key that can author, and nothing it publishes is exempt from the checks a person's publish goes through: read-only SQL, tables inside the indexed set, a template that runs, a changed column set refused unless it is declared as breaking. The consumer keys the client's applications use are separate, granted per data API and per scope.

3. Your chatbot, without changing its client

A team running a chatbot on ask-sql can point its client at InsightCompose's /api/asksql routes, which speak the same protocol, and keep its .env: every ASKSQL_* setting is read when the InsightCompose name for it is unset, and Settings › SQL engine shows which value is in force and where it came from. Its answers now go through the same engine as Ask — agreed metrics, glossary rules, verified examples, name resolution, the answer cache — and are logged like every other call.

Two things changed in how sources are indexed, because client schemas have thousands of tables. Several schemas can be asked as one source (insurance.gold+reference+platinum). And indexing is table by table: only the tables listed, or picked on Sources › Add tables, are ever read — never a whole schema implicitly. Embedding the index is off by default; columns and values are found by text until you choose otherwise, and when you do, the calls are paced so the account is never flooded.

4. Suggestions from what is already written down

Prepare › Suggestions proposes the conventions, joins, metrics and golden questions a curator would otherwise write by hand, from three places:

• The data's profile. A history table has a current-row flag and repeating keys; a date that ends at 9999-12-31 means “no end”; a status column says which rows are cancelled. Read from the index, with no warehouse query.
• The queries people already run. With read access to the warehouse's query history, the joins analysts write, the filter almost every query on a table carries, the date a fact is always grouped by and the ratio computed again and again become proposals — a filter is suggested only when a clear share of past queries use it.
• Your dbt project. Its compiled manifest already says what columns mean, what a row is (a unique test), how tables join (relationships tests) and, with the semantic layer, how metrics are computed. Upload it and those become proposals too.

Nothing is applied until someone accepts it. Accepted, a convention or metric goes into every generated query on that table, the same way a definition a curator wrote does.

And when the warehouse changes underneath

Migrations do not end on the day they finish; the warehouse keeps moving. Each re-index compares the schema with the last one, and every definition that carries SQL — data APIs, insight steps, metrics, golden questions, the dashboards on a broken data API — is parsed and checked against what the warehouse has now. A reference to a dropped or renamed column is listed with the object that owns it on the Health page, and, if you set a webhook, posted to Teams or Slack the first time it appears. Without a model call, and before a consumer finds out from an error.