r/bigquery • • 2h ago

I analyzed 27k BigQuery jobs to find what actually drives the bill. 68% of it was one query pattern.

3 Upvotes

Ran a cost analysis over a 30-day window on a production project — 27,076 jobs, 45.1 TB billed, ~$282 in query cost. Anonymized.

Methodological caveat first, because it matters: BigQuery doesn't report bytes per table, so a job's entire scan gets counted against every table it references. Rank with that column, never sum it. Same for the findings below — they overlap, one query can trigger three rules. Don't add them up.

The interesting part: the project was spending ~$2.30/day. On 25 August it jumped to ~$25/day and stayed there.

Cause: one query pattern, executed 2,638 times in 30 days, billing 30.8 TB. On its own that's $192 — 68% of the entire month's query cost.

Not a badly written query. A good one running far too often.

What else showed up:

  • 633 queries used SELECT * on tables over 1 GB → 4.0 TB. BigQuery bills the columns you read, not the rows you return.
  • 2,214 queries scanned 10 GB+ with no LIMIT → 34.8 TB.
  • 126 queries read a partitioned table without its partition filter → every partition scanned instead of the few needed.
  • 42 queries ran 10+ times and re-billed 40.9 TB — the result cache only helps while the underlying data is unchanged.

The part that surprised me: 98% of query spend came from a single service account — the dbt runner. Nobody was being careless. It was a pipeline doing exactly what it was told, at a scale nobody was watching.

Which is the real problem with warehouse bills. The invoice tells you the total. It never tells you that one model, executed 2,638 times, is two-thirds of it.

Everything above comes from INFORMATION_SCHEMA.JOBS_BY_PROJECT — bytes billed, who ran it, what it referenced. No table data is read.

I built a small read-only tool around this — self-hosted, metadata-only, nothing leaves your machine.

That project's report is confidential, so I can't share it. What I can share is a sample the tool generates from built-in synthetic data — nothing in it is real, but the format and the findings are exactly what you get: [sample_report]