Senior Data Engineer (Big Data) (Durban) (HLR7982)

Location: Durban

Remuneration: Market related

HLR Reference #: HLR7982


This role is for a Senior Data Engineer to help build and maintain scalable data pipelines and related systems for a large organisation. This is a newly created position that will offer exciting new challenges

Purpose of the role:

  • Responsible for designing, developing, testing, and deploying data solutions that meet the business requirements
  • Collaborate with all stakeholders to ensure data quality, reliability, accessibility, security and governance
  • Design, develop, and maintain end-to-end technical aspects of all data pipelines
  • Support ETL processes including, data ingestion, transformation, validation, and integration processes using various tools and frameworks
  • Optimize data performance, scalability, and security.
  • Provide technical guidance and support to data analysts
  • Design data integrations and data quality frameworks.
  • Work and collaborate with the rest of the IT department to help develop the strategy for long term Big Data platform architecture
  • Document and effectively communicate data engineering processes and solutions.

Qualifications and Experience Required:

      • Bachelor’s degree or higher in Computer Science, IT, Engineering or, Mathematics
      • 7+ years’ experience in a Data Engineering, High Performance
      • Computing, Data Warehousing, Big Data Processing
      • Strong Experience working with various relational database technologies like MS SQL, MySQL, PostgreSQL as well as NoSQL databases such as MongoDB, Cassandra
      • Strong experience with high performance computing environments including Unix, Docker, Kubernetes, Hadoop, Kafka, Nifi or Spark or Cloud-based big data processing environments like Amazon
      •  Redshift, Google BigQuery and Azure Synapse Analytics
      • Experience of Big Data technologies such as Hadoop, Spark and
      • Hive
      • At least 5 years’ advanced experience and very strong proficiency in UNIX, Linux, Windows
      • Knowledge of various data related programming, scripting or data engineering tools such as Python, R, Julia, T-SQL, PowerShell
      • Experience working with Big Data Cloud based (AWS, Azure etc) technologies is advantageous
    • To apply please send through your application to:

Consultant: Debbie Wilke

More to explorer

Junior Data Analyst (HLR7986) (JHB/DURBAN)

Location: Johannesburg/Durban Remuneration: Market related HLR Reference #:  Environment: a Testing client is looking for a skilled Junior Data Analyst for their