Data Engineer – Cloud & AI Platforms

cloudandthings.io · Johannesbourg et périphérie

🇬🇧 English

Python SQL Git CI/CD Terraform Bicep AWS Kinesis MSK (Kafka) Azure Event Hubs Kafka AWS DataSync Azure Data Factory Synapse Pipelines Fabric Pipelines APIs JDBC/ODBC CDC Amazon S3 Azure Data Lake Storage Gen2 Databricks Delta Lake Unity Catalog Microsoft Fabric OneLake Lakehouse Warehouse Parquet Avro JSON Postgres SQL Server Aurora DynamoDB Cosmos DB Redis PySpark Delta Live Tables Workflows AWS Glue Azure Databricks Synapse Spark Amazon Redshift

Job description

About the role

cloudandthings.io is an engineering‑led consultancy building modern Data and AI platforms for enterprise clients. As a Data Engineer you will design, build, and operate cloud‑native data solutions on AWS and Azure, supporting real‑time analytics, reporting, and AI use cases.

Key responsibilities

Develop and maintain scalable ingestion pipelines using services such as AWS Kinesis, MSK, Azure Event Hubs, and Data Factory.
Design data lakes and lakehouse architectures on Amazon S3, Azure Data Lake Storage, Databricks, and Microsoft Fabric.
Build ETL/ELT workflows with Databricks (PySpark, Delta Live Tables), AWS Glue, Lambda, and Azure Synapse Spark.
Implement medallion (Bronze/Silver/Gold) data models and ensure pipelines are reusable, testable, and production‑grade.
Enable analytics and machine‑learning workloads by integrating with Redshift, Azure Synapse, and other BI tools.

Required profile

Strong foundation in software engineering fundamentals, data structures, and algorithms.
Proficiency in Python and SQL; comfortable with Git, CI/CD pipelines, and Linux environments.
Experience with Infrastructure as Code tools such as Terraform or Bicep.
Ability to work across multi‑cloud environments (AWS and Azure) and collaborate with cross‑functional teams.

Required skills

Python, SQL
Git, CI/CD
Terraform or Bicep
AWS services: Kinesis, MSK, DataSync, DMS, Glue, Lambda, Redshift, S3
Azure services: Event Hubs, Data Factory, Synapse Pipelines, Fabric, ADLS Gen2
Databricks (Delta Lake, Unity Catalog, PySpark)
Microsoft Fabric (OneLake, Lakehouse, Warehouse)
Data formats: Parquet, Avro, JSON
Relational DBs: Postgres, SQL Server, Aurora
NoSQL: DynamoDB, Cosmos DB
Caching: Redis

Questions fréquentes

Le salaire n'est pas communiqué publiquement par le recruteur. Vous pouvez postuler et négocier directement avec cloudandthings.io.

Cliquez sur "Postuler maintenant" en haut de la page. Vous pouvez importer votre CV en 1 clic — Jobiglo extrait automatiquement vos informations et postule pour vous.

Why are you reporting this job?

Thank you for your report. We will review this job.

Explore further

Salaries, guides and searches in South Africa.

Salaries by job title

Apply in 30 seconds

Enter your email to apply. An account will be created automatically.

Apply now →

By continuing, you accept our terms of use.

Already have an account? Login

A question about this job?

Ask it here: you will get the full job summary by e-mail, right away.

💬 Chat with us on Telegram

3 views · 0 interested

Share Log in to earn credits by sharing

Boost your chances

Upload your CV — we will match you with relevant openings.

Analyzing your CV...

cloudandthings.io

Johannesbourg et périphérie

Related job offers

Emplois à Johannesbourg et périphérie Métier : IT / Computer Science