Data Engineering Lead
1 day to apply
RAPSYS TECHNOLOGIES PTE. LTD.
a month ago
Posted datea month ago
N/A
Minimum levelN/A
Key Responsibilities
Required Technical Skills
Preferred / Nice-to-Have Skills
Qualifications
Personal Attributes
- Lead the design, development, and maintenance of ETL/ELT pipelines using Databricks Notebooks & Workflows
- Architect, implement, and manage Delta Lake and Data Lakehouse solutions
- Manage and govern data using Unity Catalog, ensuring proper access control and lineage
- Tune and optimize Databricks jobs for cost, performance, and reliability
- Develop and optimize Python and SQL code for data transformation and analysis
- Integrate and manage data with Azure Synapse
- Ensure scalability, availability, and security of data pipelines
- Establish data engineering best practices, standards, and operating procedures
- Support data quality checks, monitoring, and alerting
- Collaborate with data scientists, analysts, architects, and business stakeholders
- Provide guidance and mentorship to junior data engineers (for Lead role)
Required Technical Skills
- Strong hands-on experience with Databricks (Notebooks & Workflows)
- Expertise in ETL / ELT pipeline management
- Strong experience with Delta Lake and Data Lakehouse architecture
- Experience managing Unity Catalog (data governance & access control)
- Strong hands-on coding experience in Python and SQL
- Experience integrating with Azure Synapse Analytics
- Experience in performance tuning & cost optimization of Databricks jobs
Preferred / Nice-to-Have Skills
- Experience in banking or regulated industries
- Knowledge of Azure Data Services (Data Factory, Data Lake Storage Gen2, Event Hub)
- Experience with streaming data and real-time pipelines
- Knowledge of data quality frameworks and monitoring tools
- Experience in CI/CD for data pipelines (Azure DevOps / GitHub)
Qualifications
- Bachelor's Degree in Computer Science, Engineering or related field
- 5-10 years of experience in Data Engineering (3-5 for non-lead role)
- Proven experience delivering enterprise-grade data platforms
Personal Attributes
- Strong analytical, problem-solving and leadership skills
- Excellent communication and stakeholder management abilities
- Highly organised with attention to detail
- Ability to work under pressure and manage multiple priorities
JOB SUMMARY
Data Engineering Lead
RAPSYS TECHNOLOGIES PTE. LTD.
Singapore
a month ago
N/A
Contract / Freelance / Self-employed
Data Engineering Lead
1 day to apply