Data Engineer (ADF / Databricks & Python) | 12-Months Contract
Manpower Singapore| Date Posted: 27-Feb-2021
EA Licence No: 02C3423
As a Data Engineer - Azure Databricks, you know the importance of data to business. You design and set up projects that bring together information from a variety of sources, to enable analysis and decision-making. You make sure that data is accessible and easy to use, so that it can be used for routine and ad-hoc analysis.
Day to day, you will:
- Collaborate with the Product Owner and assimilate data science requirements.
- Collaborate with the remote teams to plan and execute the data science models.
- Commit requirements to a sprint on a biweekly basis. Ensure tasks are split according to roles.
- Bachelor's Degree in Computer Science or Information Technology, with around 5 years in Data Analytics
- Big Data components such as Kafka, Spark SQL, Data frames, HIVE DB etc. and their implementation using Azure Data Bricks
- Experience in migration from on-premises Spark environment to Azure using Azure Data bricks
- Azure Data bricks integration with other services
- Read and write data in Azure Data bricks
- Coding complex U-SQL, Spark (Scala or Python), T-SQL
- Good Understanding of Modern Data Warehouse/Lambda Architecture, Data warehousing concepts
- PowerShell scripting
- Effort estimation in support of project delivery or pre-sales effort
Bernice Mae Nocum Rallonza EA License No. 02C3423 Personnel Registration No. R1442141