Expert Data Engineer - Kafka & Python

About the role

We are seeking an Expert Data Engineer to design, build and operate high‑throughput, real‑time data pipelines. The role is hybrid (2 days onsite) and based in Menlyn, Pretoria, with a 3‑year renewable contract.

Key responsibilities

• Design, build and maintain high‑throughput data pipelines.
• Develop Kafka producers, consumers and streaming integrations.
• Create data‑engineering and automation solutions using Python.
• Implement APIs, webhooks and event‑driven ingestion mechanisms.
• Develop and optimise real‑time ETL/ELT processes.
• Work with enterprise databases, complex queries and data models.
• Implement data quality, metadata, lineage and governance controls.
• Design resilient pipelines with retry, replay and idempotency.
• Troubleshoot production issues and improve platform performance.
• Contribute to integration architecture, standards and best practices.

Required profile

• 6+ years of hands‑on data engineering, systems integration or event‑streaming experience.
• Strong practical experience with Apache Kafka.
• Advanced Python development skills.
• Experience with real‑time data processing and event‑driven architecture.
• Proficiency in developing and integrating REST APIs and webhooks.
• Deep knowledge of ETL/ELT pipelines and enterprise database optimisation.
• Understanding of Avro, Protobuf and schema registries.
• Ability to design secure, reliable and scalable production solutions.
• Strong troubleshooting and problem‑solving capabilities.

Required skills

• Apache Kafka
• Python
• REST APIs
• Webhooks
• ETL / ELT
• Enterprise databases (SQL)
• Avro
• Protobuf
• Schema registries
• Java
• Terraform
• Kubernetes
• MongoDB
• Hadoop
• Azure Synapse
• Confluent Cloud
• Kafka Connect
• Prometheus
• Grafana
• OpenTelemetry

What we offer

• Long‑term opportunity to work on sophisticated, business‑critical data platforms.
• Chance to influence technical direction and solve challenging engineering problems.
• Hybrid working model with a stable 3‑year renewable contract.