Data engineering

Projects

Advanced Dataform framework

Reduced a model definition from approximately 150 lines to 20 lines of configuration in the documented example, while standardizing cost-efficient, production-ready BigQuery patterns across pipelines.
Challenge
Google Cloud Dataform simplified pipeline orchestration, but the existing setup still relied on full-table assertions, custom partition-aware MERGE logic, manual schema changes, and external orchestration for some steps. That added BigQuery scans and maintenance work as workloads grew.
Approach
Extended Google Cloud Dataform with a reusable modeling framework that standardizes advanced BigQuery engineering patterns across pipelines. The framework automates incremental processing, schema evolution, partition and clustering management, and partition-scoped data quality checks. This turned complex pipeline requirements into reusable model configurations, reducing unnecessary BigQuery scans and reliance on external orchestration.
Tools
Welcome

Quick intro

I’m Bolaji, a Data Engineer & Architect. I design and build production data platforms and marketplace analytics solutions, and lead the teams behind them.

I design how data flows from source systems to the products teams use. I build batch and streaming pipelines, model warehouse and lakehouse data, and build governed applications.

Former Head of Data with 7+ years building data platforms. Statistics Graduate with a decade across analytics, data science, and data engineering. Hands on GCP & AWS.

Built and led a data team from 0 to 14. Delivered ~300 bigquery models and airflow dags for reporting, self-service and operational tools. Led workloads optimisation that reduced GCP cost by 42.57% in one quarter.