The Dashboard Trust Problem
AI agents can build anything — except dashboards you can trust. Here's how we're fixing that with dbt-skillz and Workshop.
Your AI Agent Can Write Code. It Can't Be Trusted With Your Data.
Coding agents have crossed a threshold. They scaffold full-stack apps, wire up APIs, and deploy to production — largely unsupervised. But point one at your data warehouse and ask for a revenue dashboard, and you'll get something that looks right, runs fine, and is quietly wrong.
We've seen this at scale. Workshop processes hundreds of thousands of project requests. Roughly one in four is a dashboard — a chart, a report, an internal tool backed by live data. The demand is enormous. The trust problem is worse.
The Problem
Agents are fluent but not grounded
A modern coding agent can generate a React dashboard with D3 charts, a FastAPI backend, and a SQL query layer in a single session. It's genuinely impressive. But it's working from general knowledge — it knows what revenue dashboards usually look like, not what your revenue metric actually means.
Does "revenue" mean bookings or recognized ARR? Does "active users" include trial accounts? Is the churn denominator start-of-period or average? The agent doesn't know. It guesses. And the guess is often plausible enough to ship.
The quiet failure: The dashboard renders. The numbers populate. The CEO sends it to the board. Three weeks later, finance flags that the churn number is off by 40%. Nobody noticed because the chart looked professional and the data was real — just wrong.
JavaScript is the best visualization layer — and the worst governance layer
There's a reason the best dashboards on the internet are built with JavaScript. D3, Observable Plot, Recharts, Chart.js — these libraries produce visualizations that are interactive, responsive, and beautiful. No BI tool matches them for flexibility.
But when an agent builds a JS dashboard, the metric logic lives in application code. It's a SELECT SUM(amount) buried in a Python backend route, or a filter condition hardcoded in a fetch call. There's no connection back to your semantic layer. No lineage. No governance. The metric exists, but it's an orphan — untraceable, unreviewable, and un-updateable when definitions change.
Metrics drift is inevitable
Data teams know this: metric definitions change. A new product line launches and "total revenue" needs a new category. The attribution model gets updated and "organic users" shifts. A data source migrates and column names change underneath.
In a traditional BI stack, you update the semantic layer and dashboards inherit the change. But agent-built dashboards are static artifacts. They were correct at the time of creation and begin drifting immediately. The more dashboards you build, the faster the drift compounds.
~25% of all Workshop project requests are dashboards or data visualizations. And Day 1 is the last day most agent-built dashboards are guaranteed to be accurate.
This is the gap: agents can build dashboards better than ever, but there's no mechanism to ensure they stay right. The creation problem is solved. The trust problem is wide open.
Our Approach
Give the agent your data's source of truth
The core insight is simple: if the agent understood your semantic layer the way your analytics team does, it would build dashboards correctly from the start. The problem isn't agent capability — it's agent context.
dbt has become the standard for how data teams define, test, and document their data transformations. It's where metric definitions live, where column descriptions are maintained, where lineage is tracked. It's the closest thing most organizations have to a single source of truth for "what does this data mean."
So we built a compiler that turns a dbt project into a skill — a structured context document that any coding agent can consume.
# Install dbt-skillz
pip install dbt-skillz
# Point it at your dbt project
dbt-skillz compile --project-dir ./analytics --output ./skills/data
# Output: structured skill files the agent can read
# skills/data/SKILL.md (entry point)
# skills/data/ref/sources.md (7 sources)
# skills/data/ref/staging.md (24 models)
# skills/data/ref/marts.md (50 models)
# skills/data/ref/lineage.md (full DAG)
The compiled skill contains everything the agent needs: model descriptions, column types and documentation, metric definitions, test constraints, and the full lineage graph. When the agent builds a dashboard, it doesn't guess what "revenue" means — it reads the definition from your dbt project and writes the query accordingly.
The flow:
dbt project → dbt-skillz compile → Agent skill → Coding agent → Grounded dashboard
dbt-skillz is open source (Apache 2.0) and available on PyPI. Learn more at dbtskillz.com.
Workshop: the runtime for trusted data apps
dbt-skillz is open source and works with any coding agent — Claude, Cursor, Windsurf, whatever your team uses. But the full vision requires a runtime that understands the lifecycle of a data app.
That's Workshop. Available as both a desktop app and a cloud platform, Workshop lets agents build data applications using the full power of the JavaScript ecosystem for the presentation layer and Python's data analytics ecosystem for the backend — with serverless compute that scales to zero.
When a dbt-compiled skill is imported into Workshop — from a private or public GitHub repo — the agent has everything it needs: semantic context for accuracy, code execution for building, and deployment infrastructure for shipping. The result is a dashboard that's not just correct, but traceable — you can audit which metrics it references, which dbt models it depends on, and whether those definitions have changed since it was built.
For the full guide on importing skills, see the Skills documentation.
Not just for dashboards
Something unexpected happened when we started using dbt-skillz internally: product managers started making dbt changes.
With the compiled skill as context, a PM can describe a new event they want to track, have the agent instrument it in the codebase, update the corresponding dbt models, and submit a pull request to the analytics team for review. The skill grounds the agent well enough that the PR is usually close to correct — the analytics engineer reviews it for edge cases, not for fundamental misunderstandings.
We're seeing full PR lifecycles — from event instrumentation to staging model updates — initiated by people who have never written a dbt model by hand. The skill acts as a bridge between domain expertise ("I know what this metric should mean") and implementation ("here's the SQL that makes it so").
How It Works Today
What ships now: dbt-skillz is an open-source compiler that converts any dbt project into structured agent context. Install it, point it at your project, and get a skill that works with any coding agent. Combine it with Workshop for a full build-and-deploy workflow grounded in your data definitions.
The skill integrates at two points in the lifecycle:
At build time — when an agent is creating a dashboard in Workshop (or any other coding environment), the skill provides the semantic context to write correct queries, use the right metric definitions, and respect the grain of each table.
At review time — the same skill can be used by a review agent (for example, a GitHub Action) to audit changes to the dbt project itself. When someone submits a PR that modifies a staging model or changes a metric definition, the review agent uses the skill to understand the full context and flag potential issues.
PR changes dbt models
↓
PR diff → Review agent ← Compiled data skill
↓
Approve or request changes
# CI integration — .github/workflows/dbt-review.yml
on:
pull_request:
paths: ['analytics/**']
jobs:
review:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- run: pip install dbt-skillz
- run: dbt-skillz compile --project-dir ./analytics
- run: |
# Agent reviews PR diff with skill as context
dbt-skillz review --skill ./skills/data --diff $GITHUB_EVENT_PATH
The Full Lifecycle
The compiler and build-time integration solve the creation problem. But we didn't stop there. The real value is what happens after a dashboard ships — when your data model inevitably changes underneath it.
1. Grounded creation
The agent builds with full semantic context from your dbt project — correct metric definitions, proper grain, accurate join logic — from the first line of code.
2. Automated review on deploy
When a new dashboard is pushed to GitHub, a review agent audits it against the data skill — verifying metric definitions, aggregation grain, join logic, and column references before it ships. Think of it as a data-aware code review that catches semantic errors, not just syntax.
New dashboard → Review agent ← Data skill
↓
Verify metrics · grain · joins · columns
↓
Ship or flag for revision
3. Downstream impact analysis
When a dbt model changes, the system automatically identifies every dashboard that depends on the affected models. You see the blast radius before the change merges — not three weeks later when finance flags a discrepancy.
4. Automated dashboard maintenance
When dbt changes are confirmed and merged, an agent reviews every affected dashboard, generates pull requests with the necessary updates, and notifies the dashboard owners. Metrics stay current without manual audits. Dashboards that used to silently drift now get explicit, reviewable update proposals.
dbt changes merged to main
↓
Dashboard registry scans
┌──────┼──────┐
App A App B App C
└──────┼──────┘
↓
Update PRs opened → Owners notified
The result is a closed loop. Dashboards aren't static artifacts that decay — they're living documents grounded in a single source of truth, maintained by agents that understand the full context, and reviewed by humans who can focus on intent rather than implementation.
And this pattern generalizes beyond dbt. Any system that defines "what this data means" — Looker semantic models, Cube.js schemas, well-documented SQL views — can be compiled into an agent skill. dbt is where we started because it's the most structured semantic layer in the ecosystem. It won't be the last.
Get started
dbt-skillz is open source. Workshop is free to start building.
dbt-skillz on GitHub — Install the compiler and generate your first data skill in minutes.
Build with Workshop — Import your skill and start building trusted dashboards, powered by your semantic layer.
See the Dashboard solution — Explore what you can build with Workshop's dashboard capabilities.
Explore the docs — Deep dive into the full trusted dashboards lifecycle.
Want the full stack?
We work directly with a limited number of teams to implement the full trusted dashboards lifecycle — from dbt skill compilation through automated review and maintenance. White-glove setup, training, and ongoing support.