Expert Data Engineer – Zero Trust & Real-Time Event Streaming (Contract)

About the role

Our client seeks an expert‑level Data Engineer to join the core Data Science and Engineering team within the Group IT Hub South Africa. The role focuses on designing, building, and operating automated, AI‑enabled pipelines that stream security, identity and workspace data as part of a strategic Zero Trust transformation.

Key responsibilities

• Design, build and operate high‑throughput real‑time pipelines ingesting data from Zero Trust sources such as CMDB, identity systems and security tools.
• Implement event‑streaming solutions using Apache Kafka, schema registries and contract‑first formats (Avro, Protobuf).
• Develop idempotent, observable streaming pipelines in Python and optimise queries for operational/OLTP workloads.
• Provide technical leadership, define best‑practice standards and resolve escalated architectural challenges without senior escalation.
• Collaborate with DevOps teams to ensure resiliency, security‑by‑design and full observability of data flows.

Required profile

• Master‑level Principal Data Engineer or Streaming Systems Architect with deep expertise in real‑time event streaming.
• Proven track record of delivering complex, multi‑system integration projects in an enterprise security context.
• Highly curious, detail‑oriented and capable of leading technical teams in a hybrid work environment.

Required skills

• Apache Kafka
• Schema registries (Confluent, etc.)
• Avro and Protobuf
• Python for data pipelines
• Data normalization across CMDB, identity and security sources
• API integration patterns
• DevOps practices and operational resilience

What we offer

• Contract from August 2026 to December 2028
• Hybrid work model across Midrand, Menlyn, Rosslyn and home office
• Opportunity to shape the data architecture of a global automotive group’s Zero Trust initiative