Location: Durban
Remuneration: Market related
HLR Reference #: HLR7982
Environment:
This role is for a Senior Data Engineer to help build and maintain scalable data pipelines and related systems for a large organisation. This is a newly created position that will offer exciting new challenges
Purpose of the role:
- Responsible for designing, developing, testing, and deploying data solutions that meet the business requirements
- Collaborate with all stakeholders to ensure data quality, reliability, accessibility, security and governance
- Design, develop, and maintain end-to-end technical aspects of all data pipelines
- Support ETL processes including, data ingestion, transformation, validation, and integration processes using various tools and frameworks
- Optimize data performance, scalability, and security.
- Provide technical guidance and support to data analysts
- Design data integrations and data quality frameworks.
- Work and collaborate with the rest of the IT department to help develop the strategy for long term Big Data platform architecture
- Document and effectively communicate data engineering processes and solutions.
Qualifications and Experience Required:
-
-
- Bachelor’s degree or higher in Computer Science, IT, Engineering or, Mathematics
- 7+ years’ experience in a Data Engineering, High Performance
- Computing, Data Warehousing, Big Data Processing
- Strong Experience working with various relational database technologies like MS SQL, MySQL, PostgreSQL as well as NoSQL databases such as MongoDB, Cassandra
- Strong experience with high performance computing environments including Unix, Docker, Kubernetes, Hadoop, Kafka, Nifi or Spark or Cloud-based big data processing environments like Amazon
- Redshift, Google BigQuery and Azure Synapse Analytics
- Experience of Big Data technologies such as Hadoop, Spark and
- Hive
- At least 5 years’ advanced experience and very strong proficiency in UNIX, Linux, Windows
- Knowledge of various data related programming, scripting or data engineering tools such as Python, R, Julia, T-SQL, PowerShell
- Experience working with Big Data Cloud based (AWS, Azure etc) technologies is advantageous
- To apply please send through your application to: debbie@helenlucre.co.za
-
Consultant: Debbie Wilke