BI Big Data Engineer – Johannesburg – up to R950k per annum

An expert in integrated corporate benefits well known for outstanding usability and integration within the banking space is looking for a specialist capable of working within the capacity of a BI Big Data Engineer who will be responsible for building and maintaining data pipelines for new data in a variety of formats using an optimal orchestration method into a modern data platform. The candidate will have deep data engineering skills with experience using different programming languages and working comfortably on Linux or Windows environments. 

Responsibilities:

  • Perform data analysis and normalisation to create features for machine learning applications.
  • Ingest data into the data environment in batch or real-time
  • Build Meta-data driven data integration solutions
  • Develop and maintain APIs for seamless integration of modern data platform with other systems.
  • Productise Machine Learning models on a Linux containerized environment to ensure scalability and reliability.
  • Administer Linux operating system and exhibit a strong programming experience.
  • Utilise distributed programming techniques on a cluster of servers for optimised data processing.
  • Collaborate with teams to advise on optimal data system architecture for data warehousing, data lakes, and lakehouses.
  • Conduct root cause analysis on production issues surrounding data pipelining and execution
  • Technical leadership of entire information management process of both structured and unstructured data
  • Optimisation of the solutions for loading and transforming multi structure data
  • Create comprehensive technical documentation for data systems, APIs, and processes.

Qualifications and Experience:

  • National Diploma /BTech/Mtech IT or BSC/MSC Computer Science
  • 5 years plus experience
  • Python, Java, JavaScript, C# (Advanced)
  • Spark SQL and R programming (Advanced)
  • Building and maintaining API(s) and Webhooks (Advanced)
  • Google Analytics data (Advanced)
  • Real-time streaming and batch orchestration (Advanced)
  • Object Oriented Programming (Advanced)
  • SQL programming (Advanced)
  • Data Warehouse, Data Lake and Lakehouse (Advanced)
  • Git versioning of code (Advanced)

The Reference Number for this position is NG58552 which is a Permanent position in Sandton, Johannesburg offering a salary negotiable between R750k to R950k CTC based on experience and skills. E-mail Nokuthula on nokuthulag@e-merge.co.za or call her for a chat on 011 463 3633 to discuss this and other opportunities.

Are you ready for a change of scenery? e-Merge IT recruitment is a niche recruitment agency. We offer our candidates options so that we can successfully place the right people with the right companies, in the right roles. Check out the e-Merge IT website www.e-merge.co.za for more great positions.

Do you have a friend who is a developer or technology specialist? We pay cash for successful referrals!

    To apply for this position, contact Nokuthula Gumbo at nokuthulaG@e-merge.co.za or complete the form below:





    Attach your CV:

    More Jobs Like This

    Submit Your Application

    We are here To Assist You