The Data Engineer is responsible for building and maintaining HAECO Enterprise Data Platform by developing optimized data extraction, transformation pipelines, highly performance data cleansing tools following industry and HAECO standards. He/She will be working closely with all HAECO business units and collaboratively with Group Digital and Group IT to build and maintain HAECO Enterprise Data Platform.
Part A. Key Accountabilities and Responsibilities
• Build efficient and scalable pipelines for data extractions, transformations, and cleansing
• Collaborate with data analyst, data engineers and data scientist to ensure best practice, standards and guidelines are followed
• Optimize Enterprise Data Platform completeness with data ingestion from different systems and sources inside and outside HAECO
• Build low code, clean and maintainable data management process with Azure data management tools
• Implement projects for Enterprise Data Platform in an accurate and timely manner
Part B. Key Duties
• Assemble large, complex data sets that meet functional / non-functional business requirements
• Extract, transform, and load data from a wide variety of data sources using SQL and Azure ‘big data’ technologies, including, Data Lake, Synapse Analytics, etc.
• Create data models for analytics, products, and data scientist team members building and optimizing our digital products into an innovative industry leader
• Work with data and analytics experts to strive for greater functionality in our Enterprise Data Platform
• Working within a project management/agile delivery methodology in a leading role as part of a wider team
Part C. Required Experience, Competencies & Qualifications
1. Functional and other Relevant Experience
• Over 5 years of working experience in data engineering and data management
• Advanced working SQL knowledge, data modelling knowledge including relational, data vault, and dimensional data model design
• Solid experience in manipulating, processing, and extracting value from large, disconnected datasets
• Experience in delivering solutions using the Azure Data Analytics platform including Azure Databricks, Azure Data Factory, Azure Functions, Azure Storage, Azure SQL Database, Synapse Analytics, Azure Data Lake, Logic Apps and Networking
• Experience in designing and implementing high-volume data ingestion and transformation pipelines
• Sound knowledge and experience in object-oriented/object function scripting languages including Python, Java, C++, PowerShell scripting is an advantage
2. Qualifications and other Relevant Knowledge
• Bachelor’s degree holder in Computer Science, Information Systems, Software Engineering, Statistics, Mathematics, or a related technical discipline
• Able to use Machine Learning & Artificial Learning tools, Python’s Matplotlib and NumPy data management is an advantage
• With Microsoft Azure Data Analytics, etc. certification is an advantage
• Fluent in English, Mandarin is an advantage
Candidates with more experience can be considered as Senior Data Engineer.