Databricks Certified Data Engineer Associate
Data Engineer
Hello!, I am Kate, a Data Engineer who enjoys tackling diverse challenges across different business domains. I am passionate about building systems that scale and enable others to self-serve their data needs. I understand that great data infrastructure requires both technical excellence and a deep understanding of the business context.
My Stack
Languages
Python, PySpark, SparkSQL, R, Linux
Data Warehouse
Snowflake, Google BigQuery, PostgreSQL, Azure Synapse Analytics
Azure Databricks
Spark Declarative Pipeline, Lakeflow Jobs, Unity Catalog, DABs, Stream and Batch pipelines
dbt
Data modelling, Contracts, Snapshots, Tests, Materializations, Packages, Macros
Azure
Azure Data Factory, Azure Data Lake, Azure SQL Databases, Azure Synapse Analytics
GCP
Google Cloud Storage, BigQuery
DataOps
GitHub, Terraform, Docker, Apache Airflow
My Projects
Streaming Pipeline on Azure Databricks using Apache Spark Structured Streaming
Tools: AzureDatabricks, Pyspark, ADLS Gen2, Event Hub, Azure Functions.
Incremental Batch Processing with with Apache Spark on Databricks and ADLS Gen2
Tools: AzureDatabricks, Pyspark, ADLS Gen2, Azure Data Factory, On-Prem SQL Database.
Real-Time End to End Transport Data Pipeline with Airflow & Python
Tools: Python, Airflow, Terraform, Kafka, dbt, BigQuery, Google Cloud Storage, Looker Data Studio.
End to End Dimensional Modelling Analytics Engineering Project using dbt and Google BigQuery
Tools: dbt, Google BigQuery, SQL.
End to End Data Pipeline with Astro CLI and Apache Airflow
Tools: Astro CLI, Apache Airflow, dbt, Google Cloud Storage, BigQuery.
Server Report Data Pipeline
Tools: Cloud Functions, BigQuery, Apache Airflow, Docker, Pandas.
Certifications
dbt Certified Analytics Engineer
Automated Deployment with Declarative Automation Bundles
Let's Chat!
Reach out for full time, contract-based or speaking opportunities.
(+44) 07721488XXX
England, United Kingdom