Senior Data Engineer - ()

Description

Reporting to Head Data Science, the Senior Data Engineer will be responsible for overseeing activities of the junior data engineering teams, ensuring  proper execution of their duties and aide in building the organizations data collection systems and processing pipelines.  Provide oversights and expertise to the Data Engineering that is responsible for the design, deployment, and maintenance of the business’s data platforms.  

The role holder will oversee infrastructure, tools and frameworks   used to support the delivery of end-to-end solutions to business problems through high performing data infrastructure. 

S/He will be responsible for expanding and optimizing the organizations data and data pipeline architecture, whilst optimizing data flow and collection to ultimately support data initiatives. 

Responsibilities of the Senior Data Engineer 

• Oversee activities of the junior data engineering teams, ensuring  proper execution of their duties and alignment with the Company objectives.  
• Provide oversight and expertise to the Data Engineering that is responsible for the design, deployment, and maintenance of the business’s data platforms. Required to draw performance reports and strategic proposals form his gathered knowledge and analyses results for senior EDO leadership. 
• Own and extend the business’s data pipeline through the collection, storage, processing, and transformation of large data- sets and oversee the process for creating and maintaining optimal data pipeline architecture and creating databases optimized for performance, implementing schema changes, and maintaining data  architecture standards across the required databases. 
• Oversee the assembly of large, complex data sets that meet functional / non-functional business requirements and align data architecture with business requirements.  
• Oversee, design, and develop algorithms for real-time data processing within the business and to create the frameworks that  enable quick and efficient data acquisition. Deploy sophisticated analytics programs, machine learning and statistical methods. 
• Build analytics tools that utilise the data pipeline to provide actionable insights into customer acquisition, operational efficiency and other key business performance metrics. 
• Create data tools for analytics and data scientist team members that assist them in building and optimising into an innovative industry leader. 
• Monitor the existing metrics, analyse data, and lead partnership with other Data and Analytics teams in an effort to identify and implement system and process improvements.n  
• Utilise data to discover tasks that can be automated and identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc. 
• Acts as a subject matter expert from a data perspective and provides input into all decisions relating to data engineering and the use thereof. Provide guidance in terms of setting governance standards. 
• Responsible for performing thorough testing and validation in order to Ensure proper data governance and quality across EDO and the business as a whole.

Qualifications

• Master’s degree in Statistics, Machine Learning, Mathematics, Computer Science, Economics, or any other related quantitative field.  
• 8 to 10 years Experience in Technology handling data monetization; Including; big data tools: Hadoop, Spark, Kafka, relational SQL and NoSQL databases, including Postgres and Cassandra. 
• Experience with data pipeline and workflow management tools: Azkaban, Luigi, Airflow, etc. Experience with AWS cloud services: EC2, EMR, RDS, Redshift. 
• Experience with stream-processing systems: Storm, Spark- Streaming, etc. Experience with object-oriented/object function scripting languages: Python, Java, C++, Scala, etc. 
• The candidate must also have a proven and successful experience track record of leading high-performing data analyst teams leading through the successful performance of advanced quantitative analyses and statistical modeling that positively impact business performance. 
• Strong analytic skills related to working with unstructured datasets. Build processes supporting data transformation, data structures, metadata, dependency and workload management. A successful history of manipulating, processing and extracting value from large disconnected datasets. 
• Working knowledge of message queuing, stream processing, and highly scalable ‘big data’ data stores. 
• Ms Office/Software:  Outstanding skills in the use of Ms Word, Ms Excel, PowerPoint, and Outlook, which will all be necessary for the creation of both visually and verbally engaging reports and presentations, for senior data science management, executives, and stakeholders. 
• The candidate must also demonstrate exceptionally good skills in SQL server reporting services, analysis services, Tableau, integration services, Salesforce, or any other data visualization tools. 
• Technological Savvy/Analytical Skills:  Technologically adept and especially demonstrate an understanding of database and computer software. 
• Must also be highly skilled in statistical and modeling packages such as SAS, Statistica, Matlab, R, visualization and other advanced analysis tools. He will also be an expert in data management programming such as SQL, PL-SQL, and Python as well as being familiar win the workings of motion-tracking data and time-series analyses. 
• Interpersonal Skills: A suitable candidate for this position will be a team-builder, be result-oriented, be proactive and self-driven requiring minimal supervision, be open and welcoming to change, be a creative and strategic thinker, have innovative problem-solving skills, be highly organized, have an ability to handle multiple simultaneous tasks prioritize and meet tight deadlines, and demonstrate calmness in times of uncertainty and stress. 
• People Skills:  a people person who is able to form strong, lasting, and meaningful bonds with others people. This will make him an approachable and trustworthy individual who junior personnel readily follow and who senior Data and Analytics executives and stakeholders trust and who’s insights they give credit to, making execution of his duties much easier.