Cloud Engineer

About the role

We are looking for a Cloud Engineer to design, build and optimise enterprise‑scale data pipelines that support the bank’s data‑driven strategy. The role will translate high‑volume, heterogeneous raw data into governed, low‑latency data products for advanced analytics, machine learning and real‑time insights.

Key responsibilities

• Architect, engineer and maintain robust, scalable and fault‑tolerant data pipelines using Ab Initio, Python/Scala, Apache Spark, Microsoft Fabric and Databricks.
• Own the end‑to‑end data lifecycle from source ingestion through batch and streaming processing, ensuring data quality, semantic consistency and metadata management.
• Implement modern DataOps practices, including automation, testing and orchestration for both batch and low‑latency streaming workflows.
• Collaborate with software engineers to design consumption APIs that provide secure real‑time data access for business applications.
• Design and manage a hybrid cloud data platform on Azure services (Fabric, Synapse, Databricks, SAS) and on‑premise technologies (DB2, Netezza, Denodo).
• Establish observability and service‑level objectives for databases, data lakes and data warehouses.
• Work with Information Security, CISO and Data Governance to enforce data‑security policies and POPIA compliance.
• Lead technical evaluations, proofs of concept and integration of new data‑engineering tools.

Required profile

• Proven experience designing and operating large‑scale data pipelines in a cloud environment.
• Strong understanding of data governance, data quality and metadata management.
• Ability to work collaboratively with cross‑functional teams and drive technical solutions aligned with business needs.

Required skills

• Ab Initio
• Python
• Scala
• Apache Spark
• Microsoft Fabric
• Databricks
• Azure (Fabric, Synapse, SAS)
• DB2, Netezza, Denodo
• Infrastructure as Code
• CI/CD and DataOps automation
• Data lineage and metadata management
• Data quality testing and validation