Principal Data Engineer (Databricks)

About the role

Reporting to the Data & Analytics Lead, the Principal Data Engineer is a hands-on role combining the building and optimisation of production data pipelines with the technical leadership needed to raise engineering standards and develop the team. The role acts as a key link between business stakeholders and department heads, the wider data and analytics team, and the owners of source systems in Finance and IT. It is a full-time, office-based position, five days per week.

Key responsibilities

• Design and build robust, production-grade data pipelines on Databricks, using platform capabilities such as Spark Declarative Pipelines (SDP)
• Optimise the platform for performance and cost: tune compute, jobs and queries, manage storage and table layout, and keep platform spend under control
• Define and maintain consistent business metrics and a semantic layer (e.g. Metric Views) so reporting is built on trusted, reusable definitions
• Integrate data from core business systems such as SAP S/4HANA and other sources, using tools such as Fivetran (SaaS connectors and HVR)
• Set and uphold engineering standards: code quality, testing, documentation, CI/CD and data governance
• Coach and mentor junior and mid-level engineers, review their work and help them develop
• Work with business stakeholders to understand their needs and deliver practical solutions
• Engage data owners in Finance and IT to ensure data is well understood, valid and fit for purpose, and initiate data quality improvements

Required profile

• At least 3 years of hands-on, daily Databricks experience covering pipeline development and performance and cost optimisation
• Up to date with recent platform developments such as SDP and Metric Views
• Experience managing or coaching junior developers
• Track record of managing and influencing business stakeholders
• Ability to explain complex technical concepts simply to non-technical audiences and to drive work independently from concept to production

Required skills

• Python, SQL and Spark
• Data manipulation, data modelling, database design and query optimisation
• Beneficial: SAP S/4HANA data sets, Fivetran, Power BI semantic modelling

Candidates apply by uploading a cover letter and a recent resume by 31 July 2026.