Databricks Data Engineer

About the role

Confluent Services is looking for a Data Engineer to join its Data Team in a hybrid capacity based in Pinelands, Cape Town. You will be responsible for designing, building and supporting data engineering solutions across on‑premise and cloud platforms, with a focus on modern tools such as Databricks and Microsoft Fabric.

Key responsibilities

• Design and implement end‑to‑end data pipelines, ingestion processes and ETL workflows.
• Maintain and evolve data platforms on Azure, including Azure Data Factory and Azure Databricks.
• Collaborate with senior engineers on architectural decisions and clean‑code practices.
• Support AI and machine‑learning initiatives by providing reliable data feeds.
• Participate in knowledge‑sharing sessions and technical discussions.

Required profile

• Minimum 3 years of experience as a Data Engineer or in a similar technical data role.
• Hands‑on experience with Azure services, especially Azure Data Factory and Azure Databricks (preferred).
• Proven ability to develop data pipelines using Python/PySpark and SQL/SparkSQL.
• Experience with data storage solutions such as SQL Server and Azure Data Lake Storage.
• Strong communication skills and a collaborative mindset.

Required skills

• Python
• PySpark
• SQL / SparkSQL
• Azure
• Azure Data Factory
• Azure Databricks
• Data pipeline design
• ETL processes
• SQL Server
• Azure Data Lake Storage

What we offer

• Vibrant office in a lively part of Cape Town with easy access to amenities.
• Company benefits including medical aid, retirement annuity, bursaries and insurance.
• Culture of growth with opportunities for internal mobility and professional development.
• Chance to work on projects that improve financial outcomes for South Africans.