pipeline-validation

1 article
sort: new top best
clear filter
0 7/10

A technical walkthrough of building a silent data loss detection system using BigQuery's native Storage Write API metadata and built-in ML anomaly detection (AI.DETECT_ANOMALIES), implemented as a dbt incremental model that monitors hundreds of tables without external infrastructure or custom ML training.

BigQuery Storage Write API WRITE_API_TIMELINE_BY_PROJECT AI.DETECT_ANOMALIES TimesFM Nordnet Pub/Sub dbt
robertsahlin.substack.com · boxer_shorts · 1 day ago · details · hn