An expert in integrated corporate benefits well known for outstanding usability and integration within the banking space is looking for a specialist capable of working within the capacity of a BI Big Data Engineer who will be responsible for building and maintaining data pipelines for new data in a variety of formats using an optimal orchestration method into a modern data platform. The candidate will have deep data engineering skills with experience using different programming languages and working comfortably on Linux or Windows environments.
- Perform data analysis and normalisation to create features for machine learning applications.
- Ingest data into the data environment in batch or real-time
- Build Meta-data driven data integration solutions
- Develop and maintain APIs for seamless integration of modern data platform with other systems.
- Productise Machine Learning models on a Linux containerized environment to ensure scalability and reliability.
- Administer Linux operating system and exhibit a strong programming experience.
- Utilise distributed programming techniques on a cluster of servers for optimised data processing.
- Collaborate with teams to advise on optimal data system architecture for data warehousing, data lakes, and lakehouses.
- Conduct root cause analysis on production issues surrounding data pipelining and execution
- Technical leadership of entire information management process of both structured and unstructured data
- Optimisation of the solutions for loading and transforming multi structure data
- Create comprehensive technical documentation for data systems, APIs, and processes.
Qualifications and Experience:
- National Diploma /BTech/Mtech IT or BSC/MSC Computer Science
- 5 years plus experience
- Spark SQL and R programming (Advanced)
- Building and maintaining API(s) and Webhooks (Advanced)
- Google Analytics data (Advanced)
- Real-time streaming and batch orchestration (Advanced)
- Object Oriented Programming (Advanced)
- SQL programming (Advanced)
- Data Warehouse, Data Lake and Lakehouse (Advanced)
- Git versioning of code (Advanced)
The Reference Number for this position is NG58552 which is a Permanent position in Sandton, Johannesburg offering a salary negotiable between R750k to R950k CTC based on experience and skills. E-mail Nokuthula on email@example.com or call her for a chat on 011 463 3633 to discuss this and other opportunities.
Are you ready for a change of scenery? e-Merge IT recruitment is a niche recruitment agency. We offer our candidates options so that we can successfully place the right people with the right companies, in the right roles. Check out the e-Merge IT website www.e-merge.co.za for more great positions.
Do you have a friend who is a developer or technology specialist? We pay cash for successful referrals!