How do I know if partitioning or clustering actually helped?

Measure before/after on the same queries: bytes scanned, partitions/files touched, and 95p runtime during peak. On BigQuery, check `query_plan` and `referenced_partitions`. On Snowflake, look at micro-partitions pruned in the Query Profile. You should see fewer partitions/data files read and a meaningful drop in scan bytes and runtime. If not, your filter selectivity, clustering key choice, or stats may be off.

Are materialized views worth it versus dbt incrementals?

Use MVs when the engine can maintain them efficiently for your pattern (simple group-bys, filters) and you need sub-minute freshness with automatic invalidation. Use dbt incrementals when logic is complex, data volumes are large, or you need full control over refresh windows and testing. We often run both: MV for hot, simple aggregates; dbt for shaped marts. Always set freshness SLOs and alerting.

Can I just buy query acceleration features to fix this?

Acceleration (e.g., Snowflake Query Acceleration Service, BigQuery BI Engine, Databricks Serverless) can help, but they’re multipliers on good design, not substitutes. If your tables are poorly partitioned, joins skewed, and files tiny, accelerators will be expensive band-aids. We install them after we’ve fixed layout and joins so the dollars translate to predictable wins.

Data-engineering · Oct 4, 2025 · 10 minute read

The 9:05 AM Dashboard Freeze: Warehouse Optimizations That Actually Move the Needle

Stop paying for more compute to hide bad modeling. Practical techniques to cut query latency, lift reliability, and ship business value.

Back to all posts

The 9:05 AM Dashboard Freeze: Warehouse Optimizations That Actually Move the Needle

Key takeaways

Implementation checklist