Stop Shipping Dashboards on Sand: Building a Self‑Service Analytics Platform That Won’t Wake You at 2 a.m.

Stop shipping dashboards on sand

If you’ve ever been paged because the CFO’s dashboard is blank on QBR morning, you know the feeling. I’ve watched teams roll out Looker/Power BI/Superset, announce “self‑service,” and then drown in Slack pings: columns missing, numbers don’t match, dashboards time out, Snowflake spend doubling overnight. The problem wasn’t the BI tool. It was everything under it.

Self‑service only works when the platform makes the reliable thing the easy thing. That means contracts, tests, lineage, a semantic layer, and guardrails. GitPlumbers has rebuilt a half‑dozen of these in the last two years, cleaning up AI‑generated “vibe code” along the way. Here’s the pattern that sticks.

What goes wrong (and why your “self‑service” isn’t)

• No contracts: Upstream teams rename user_id to customer_id, your ELT silently casts nulls, dashboards lie.
• Dashboards before semantics: Three definitions of MRR in production; finance, product, and ops fight over who’s “right.”
• Invisible data debt: No lineage, no tests, so MTTR is a guessing game.
• Governance theater: SSO without RBAC or row‑level security; everyone queries everything; warehouse melts.
• BI as a compute engine: Users write 400‑line SQL with CROSS JOINs; your XS warehouse weeps.

I’ve seen these exact failures at startups and at FAANG‑adjacent orgs. The fix isn’t a new tool. It’s a small set of hard edges and a cultural shift backed by infrastructure.

The architecture that actually delivers

You don’t need every logo on a slide. You need a clean path from contract to metric to chart, with SLOs and rollback points.

• Ingestion & contracts: Kafka + Debezium for CDC (or Fivetran if you must), schemas in Confluent Schema Registry (Avro/Protobuf/JSON Schema). Enforce backward compatibility in CI.
• Lake/warehouse: Snowflake or BigQuery for serving; Delta Lake or Iceberg if you already have lake gravity.
• Transformations: dbt for ELT with tests; Great Expectations for richer suites on critical facts.
• Orchestration & lineage: Airflow or Dagster with OpenLineage enabled. Emit lineage so users can self‑debug.
• Semantic/metric layer: dbt metrics, Cube, or Looker’s model. One place to define “MRR,” “Active Users,” “Churn.”
• BI & access: Looker, Superset, or Power BI wired to the semantic layer. OAuth/SSO, RBAC, and row‑level security.
• Catalog & docs: DataHub or OpenMetadata indexing tables, lineage, owners, and glossary.
• GitOps: Everything declarative via Terraform; deployments via ArgoCD or CI runners; no click‑ops in prod.

This stack lets you treat data like a product with SLOs, owners, and error budgets—exactly how your SREs run prod.

Build it with guardrails: configs and code

Talk is cheap. Here’s what the controls look like in code.

• Provision Snowflake with governance baked in (Terraform)

# Terraform
resource "snowflake_warehouse" "bi" {
  name              = "BI_WH"
  warehouse_size    = "XSMALL"
  auto_suspend      = 60
  auto_resume       = true
  statement_timeout_in_seconds = 300
}

resource "snowflake_role" "bi_analyst" { name = "BI_ANALYST" }
resource "snowflake_role" "bi_admin"   { name = "BI_ADMIN" }

resource "snowflake_database" "analytics" { name = "ANALYTICS" }
resource "snowflake_schema" "mart" { database = snowflake_database.analytics.name name = "MART" }

resource "snowflake_warehouse_grant" "bi_use" {
  warehouse_name = snowflake_warehouse.bi.name
  privilege      = "USAGE"
  roles          = [snowflake_role.bi_analyst.name]
}

resource "snowflake_database_grant" "db_usage" {
  database_name = snowflake_database.analytics.name
  privilege     = "USAGE"
  roles         = [snowflake_role.bi_analyst.name]
}

resource "snowflake_schema_grant" "mart_select" {
  database_name = snowflake_database.analytics.name
  schema_name   = snowflake_schema.mart.name
  privilege     = "SELECT"
  roles         = [snowflake_role.bi_analyst.name]
}

# Cost guardrail
resource "snowflake_resource_monitor" "bi_cap" {
  name                       = "BI_CAP"
  credit_quota               = 200
  notify_triggers            = [50, 75, 90]
  suspend_triggers           = [95]
  suspend_immediate_triggers = [100]
  warehouses                 = [snowflake_warehouse.bi.name]
}

• Lock in quality with dbt tests and metrics

# models/marts/fct_subscriptions.yml
version: 2

models:
  - name: fct_subscriptions
    description: Daily subscription facts
    columns:
      - name: user_id
        tests: [not_null]
      - name: day
        tests: [not_null]
      - name: mrr
        tests:
          - not_null
          - dbt_utils.expression_is_true:
              expression: ">= 0"
    tests:
      - dbt_utils.unique_combination_of_columns:
          combination_of_columns: [user_id, day]

metrics:
  - name: mrr
    label: Monthly Recurring Revenue
    model: ref('fct_subscriptions')
    type: sum
    sql: mrr
    time_grains: [day, week, month]
    dimensions: [plan, region]

• Deeper checks with Great Expectations on critical facts

# great_expectations/expectations/fct_subscriptions_suite.json
expectation_suite_name: fct_subscriptions_suite
meta: {}
expectations:
  - expect_table_row_count_to_be_between:
      min_value: 100000
  - expect_column_values_to_be_between:
      column: mrr
      min_value: 0
  - expect_column_values_to_not_be_null:
      column: user_id

• Row‑level security that doesn’t require duct tape

-- Snowflake RLS: restrict by region
create or replace row access policy rlp_region as (region string) returns boolean ->
  case
    when current_role() in ('BI_ADMIN','SECURITYADMIN') then true
    when current_role() = 'BI_ANALYST_US' then region = 'US'
    when current_role() = 'BI_ANALYST_EU' then region = 'EU'
    else false
  end;

alter table ANALYTICS.MART.DIM_CUSTOMER add row access policy rlp_region on (region);

Wire lineage via OpenLineage so when a model fails a test, you can see exactly which dashboards are impacted. Airflow and Dagster both have plugins that emit lineage without you rewriting DAGs.

Make reliability measurable: SLOs for data

If you don’t measure it, it will rot. Treat pipelines like services.

• Freshness SLOs: e.g., “95% of hourly models under 15 min late in any rolling 7‑day window.” Use dbt source freshness and a small exporter to push latency to Prometheus; alert in PagerDuty when you breach.
• Quality SLOs: “<0.1% nulls in user_id for fct_subscriptions.” Fail the job when the budget is exhausted; roll back to the last green snapshot.
• MTTR: Instrument lineage and tests so you can get MTTR for data incidents under 30 minutes. We track “time to detect” (TTD) via test failures and “time to recover” (TTR) via successful reruns.
• Error budgets for experiments: For canary tables, allow tighter budgets; automatically drop canary exposure to BI if error budget is spent.

We’ve put these on real on‑call rotations. After a month, pages drop because the platform pushes bad data out of prod by default.

Turn access into business value

Self‑service isn’t “everybody runs SQL in prod.” It’s:

1. Publish certified datasets: Only MART schemas with owners and SLOs are BI‑visible.
2. Centralize metrics: The metric layer is the API. BI models reference metrics, not arbitrary SQL. When finance updates MRR logic, everyone gets it at once.
3. Curate explores/datasets: Ship a minimal set of explores with dimensions users actually use. Kill or archive the rest.
4. Usage telemetry: Track P95 dashboard load time (<5s), daily active explorers, and % ad‑hoc queries hitting raw tables. If P95 creeps up, scale warehouses or optimize models.
5. Guard costs: Put resource monitors on warehouses; auto‑suspend; block BI roles from creating transient tables in raw.

This is where we clean up a lot of AI‑generated transform code. “Vibe coding” is great for a prototype; in prod we refactor with tests, contracts, and ownership. GitPlumbers does the vibe code cleanup and code rescue so your team ships safely.

What changed when we did this for a subscription app

At a late‑stage consumer subscription company (~5M MAU), we replaced dashboard sprawl with a contract‑to‑metric flow in 8 weeks.

• Replaced ad‑hoc SQL with dbt models + metrics; Looker pointed only at MART.
• Added Great Expectations on three critical facts; wired alerts to Slack and PagerDuty.
• Enforced CDC schemas with Schema Registry; blocked incompatible changes in CI.
• Cataloged everything in DataHub; owners and descriptions on every certified table.
• Terraform’d Snowflake RBAC and resource monitors; added row‑level security by region.

Results in first full quarter:

• -62% data incidents reaching BI (from 34 to 13); MTTR down from 2h12m to 28m.
• P95 dashboard load from 9.4s to 3.1s; warehouse credits for BI down 31%.
• Ad‑hoc data requests to data engineering down 45%; 72% of analysts active weekly in BI explores.
• C‑level trust recovered: finance and product reporting variance on MRR under 0.5%.

No heroics. Just guardrails and discipline.

A 60‑day rollout plan that holds

• Days 1–10: Define top 10 business metrics; write contracts; set freshness/quality SLOs. Terraform your warehouse RBAC and monitors.
• Days 11–25: Build bronze/silver/gold (dbt + tests); wire Great Expectations to gold facts. Stand up lineage and catalog. Ship 1–2 certified datasets and owners.
• Days 26–45: Bring up semantic layer (dbt metrics or Cube). Point BI at MART only. Add RLS. Publish glossary and “how to get a chart in prod” doc.
• Days 46–60: Migrate top 10 dashboards to metrics; set deprecation windows; add alerting; train champions. Start killing orphaned content.

If you do nothing else: define metrics centrally and enforce contracts. Everything good cascades from that.

If you want help untangling AI‑generated ELT or migrating Looker models without breaking finance’s quarter, GitPlumbers has done this before. See our case study and the playbook we use with teams like yours.

structuredSections':[{