A scheduled job scans the same data again
A daily process reads an entire history table for a limited period. Job statistics and SQL filters reveal whether relevant partitions are actually pruned.
BigQuery Cost Optimizer: lighter jobs, unchanged results
Recurring jobs read too much data. Fewer bytes scanned are not useful if dashboards become incomplete or response times deteriorate.
Select a few costly jobs with the Data team, test relevant SQL changes and compare results, duration and cost before any production rollout.
Services
When to engage?
A dashboard refresh costs too much
Joins and repeated scans increase processed bytes. Compare execution plans with expected business results: removing columns or moving filters must not alter indicators.
A table change is being considered
Partitioning and clustering can help selected access patterns but also affect storage and upstream jobs. Evaluate their value and compatibility on the chosen tables first.
A three-step engagement
- 01
Establish a baseline
I agree the jobs and reference outputs with Data owners; I record bytes processed, duration, frequency and billing model, including workload variations that would undermine comparison.
- 02
Test SQL improvements
I inspect partition filters, selected columns and joins; I test useful rewrites and, with owner approval, partitioning or clustering on the selected scope.
- 03
Compare and decide
I compare functional results, bytes, duration and cost on comparable runs. I reject changes that alter results and prepare promotion only if the owners approve it.
What you receive
- Diagnosis of selected jobsQueries and periods reviewed, expensive reads, dependencies and measurement limitations.
- Verified SQL changesQuery variants and business-output checks; table partitioning or clustering proposals remain separate where a table change is needed.
- Comparison and promotion guidanceBefore/after measures on comparable runs, explained differences, required validations and rollback for any authorized release.
Scope and prerequisites
Selected jobs and necessary access. Agreed jobs, projects, tables and periods, with SQL, run and cost history, reference outputs and a functional owner. Read access suffices for diagnosis; tests require an authorized environment and permissions.
Boundaries and production. Neither a GCP-wide FinOps audit nor a pricing review. No production schema, table or job changes without agreement, compatibility tests and a rollback plan.
A question about your environment? Tell me about it: I'll define a useful scope with you before starting.
Contact me