Senior Databricks Data Engineer

Senior Data Engineer (Databricks, AWS, DBT) x2

Location Johannesburg (Hybrid)

Work Model

50% onsite / 50% remote

Contract

Initial 3-month contract, with a strong likelihood of extension for a further 12 months.

Start Date

Immediate or candidates with a maximum 30-day notice period.

Role Overview

We are looking for two experienced Senior Data Engineers to join a high-impact data team delivering modern cloud-based data solutions. You will play a key role in designing, building and optimising scalable data pipelines using Databricks, AWS and DBT, enabling reliable analytics and supporting business-critical data initiatives.

This is an excellent opportunity to work with modern data technologies in a collaborative environment while contributing to large-scale data engineering projects.

Key Responsibilities

• Design, develop and maintain scalable data pipelines using Databricks, DBT and AWS.
• Develop and optimise ETL/ELT processes for large and complex datasets.
• Build and maintain Databricks Lakehouse solutions following Medallion Architecture (Bronze, Silver and Gold layers).
• Develop data transformations using DBT and SQL.
• Create and optimise PySpark/Spark workloads within Databricks.
• Work with Delta Lake to deliver reliable, high-performance data solutions.
• Integrate data from multiple cloud and on-premise sources into the data platform.
• Improve data quality, reliability and performance across the data estate.
• Collaborate with data analysts, architects and business stakeholders to deliver data-driven solutions.
• Implement best practices around testing, monitoring, CI/CD and data governance.

Required Skills & Experience

• Strong commercial experience with Databricks.
• Hands-on experience with:
• PySpark
• Spark SQL
• Delta Lake
• Databricks Workflows
• Unity Catalog
• Databricks SQL
• Strong experience with AWS services (such as S3, Glue, Lambda, IAM and Redshift).
• Solid experience using DBT (Data Build Tool).
• Advanced SQL skills.
• Experience building enterprise ETL/ELT pipelines.
• Strong understanding of data modelling and data warehousing principles.
• Experience working in Agile environments.
• Excellent communication and stakeholder management skills.

Key Databricks Experience

The ideal candidate will have experience with

• Databricks Lakehouse Platform
• Delta Lake
• Unity Catalog
• PySpark
• Spark SQL
• Databricks Workflows
• Databricks SQL
• Medallion Architecture (Bronze, Silver, Gold)
• Performance tuning and optimisation
• Data pipeline orchestration
• Batch and streaming data processing
• CI/CD for Databricks deployments

Desirable Skills

• Experience with Apache Kafka or streaming technologies.
• Experience with Terraform or Infrastructure as Code.
• Exposure to Azure DevOps or GitHub Actions.
• Experience implementing data governance and security best practices.

What We're Looking For

• Proven experience delivering enterprise-scale cloud data engineering solutions.
• Strong Databricks expertise with the ability to design and optimise modern data platforms.
• Someone comfortable working in a collaborative, fast-paced environment.
• Available immediately or within a maximum 30-day notice period and able to work a hybrid model in Johannesburg.