# Your agent is only as good as your semantic layer Source URL: https://blueprintdata.xyz/blog/agents-need-a-semantic-layer Published: 2026-11-03 Author: Juan Pablo Rivero Site: Blueprintdata Tags: AI, agents, semantic layer, dbt, MetricFlow, analytics engineering ## Summary We put AI agents on top of our clients' own data stacks so anyone can ask a question in plain language and get the answer, without writing a query. The model is the easy part. The semantic layer is what makes the answer trustworthy. ## Article Someone in Finance needs last week's transaction volume by product. Today that usually means a Slack message to engineering, a query written between other tasks, and an answer that arrives later. The follow-up question often never gets asked. Now picture the same person typing the question into Slack and getting the answer back in the thread, without writing SQL or opening a ticket. The number is the one Finance would give, because it comes from the same models the dashboards use. That is what we build at Blueprintdata. We put AI agents on our clients' own data stacks so the people who need data can get it themselves. They ask in plain language. The agent writes SQL against the documented consumption layer, never against raw tables, runs it on the warehouse, and returns the answer. All of this runs on the client's production warehouse and dbt project. Picking a model has been the easy part. Most of the work is in what sits between the warehouse and the agent. Why "chat with your data" usually disappoints Give an agent raw tables and it can easily join them at the wrong grain, count duplicate events, or pick the timestamp that looks right instead of the one that is right. It will not tell you it guessed, and a confidently wrong number in a Slack thread can end up in a board deck. The vendors building these tools have reached the same conclusion. dbt Labs' 2026 benchmark (https://docs.getdbt.com/blog/semantic-layer-vs-text-to-sql-2026) found that, for questions covered by a well-modeled semantic layer, "accuracy approaches or hits 100%": 98.2% and 100% for the two models tested, against 90.0% and 84.1% with text-to-SQL. It also found that when the layer could not answer, it said so instead of returning a plausible wrong number. Cube's paired benchmark (https://cube.dev/blog/why-semantic-layers-make-llm-analytics-reliable-a-paired-benchmark-across-three-frontier-models) measured a jump of roughly 17 to 23 percentage points from adding semantic context alone. We have seen the same in client work. An agent is only as good as the contract it reads: give it a documented, tested layer with explicit definitions and it behaves a lot more like a careful analyst. Most of what we call agent work is semantic layer work. What we are building The core piece is a harness: the agent plus the tooling around it (context, query limits, permissions) that keeps it reliable. It reads the dbt metadata, writes SQL against the documented consumption layer, runs it, and answers. It plugs into the stack a client already has. We can change LLM provider without rewriting the agent. People reach it through Slack or through our own web interface on top of the harness, and it supports Snowflake and BigQuery today. Charts do not depend on the channel. The agent describes the chart once, and each channel decides how to render it. Then come the guardrails: read-only credentials, query timeouts and scan/row limits, and per-client credentials and context, each client in its own isolated environment. The agent reads the semantic layer Whichever way the agent reaches the data, the semantic layer is what it reads. By semantic layer we mean the documented, tested consumption layer, with or without a metrics engine like MetricFlow on top. The harness is one way to get there, and the next two sections cover the others. Another route: the agent straight on a well-documented dbt project Not every client needs the full harness. When a team already has a well-built dbt project, with the usual layers (staging, intermediate, marts) and a consumption layer on top, the agent can work directly on that project. The YAML is what makes that reliable. We treat that consumption layer as a contract. It is the only thing dashboards and agents see. Everything below it can change, and the contract only changes on purpose. We now write the metadata for an agent as much as for a person. In practice that means: - Long descriptions that state the grain, such as "One row per product per currency per day." A model without a description that states its grain is not certified for agent use. - Business synonyms: the words people actually use, mapped to the right model. - Join notes on how the model joins to others, and on what keys. - What a null means. For key columns, we write what an empty value means in the business, so the agent does not guess. - Antipatterns: the mistakes we have seen, written down so the agent does not repeat them. - An overlapswith entry. When two models look like they answer the same question, we say so and say which one to use. - Uniqueness tests. If we state a grain, we test it. A simplified example: Code example: yaml models: - name: paymentsdaily description: > One row per product per currency per day. Counts settled payments only. Use this for daily payment volume. For per-user activity, use useractivitydaily instead. config: meta: synonyms: ["payments", "payment volume", "daily payments"] joinnotes: "Join to products on productid." antipatterns: - "Do not sum amounts across currencies." overlapswith: ["transactionsdaily"] datatests: - dbtutils.uniquecombinationofcolumns: combinationofcolumns: ["productid", "currency", "paymentdate"] columns: - name: productid datatests: [notnull] - name: currency datatests: [notnull] - name: paymentdate datatests: [notnull] - name: refundedat config: meta: nullmeans: "Payment not refunded; do not treat as missing data." A few more things live next to the models: - Routing docs per product. When there are several products, the first job is knowing which product a question is about. - A list of known metric gaps: questions we know the data cannot answer yet. When someone asks one, the agent should say so instead of improvising. - Playbooks for the complex metrics that need more than a description. - Business thresholds stored as dbt vars, each with a comment explaining what it means. - A pre-merge checklist. Any change that can affect what the agent answers goes through it before merging. No semantic layer covers every question, and some vendors argue (https://hex.tech/blog/semantic-layers-are-fine/) that is a reason not to rely on one. We would rather write the gaps down, so the agent knows where its knowledge ends and the team knows what to model next. Metrics by name We can also put the agent on MetricFlow, dbt's metrics engine. Then it queries a defined metric, say revenue, by name, grouped by product at week grain, instead of rebuilding the SQL every time. The definitions stay versioned and reviewed in the dbt project like any other code. Where we stand There is no single way to put agents on a company's data. Some stacks need the full harness. Others only need a well-documented dbt project, or the agent on top of a metrics engine. Our job is knowing which path fits each stack. Whichever it is, we treat data as software (/blog/from-software-to-data) and the agent as one more consumer of the data contract. This is how we connect our agents to real data today. We don't know yet whether it is the right approach. We do think data work is heading toward asking in plain language and getting the real number back. If your team waits on someone else for every number and you want answers in plain language from the same figures as your dashboards, we should talk (/#contact). Written by Juan Pablo Rivero for the Blueprintdata team.