Data Engineer
SOURCEO PTE. LTD.
Requirements
- Degree in Computer Science, Software Engineering, Data Science, or equivalent experience
- 2+ years in data engineering, analytics, or related AI/ML role
- Proficient in Python for ETL/data engineering and Spark (PySpark) for large-scale pipelines
- Experience with Big Data frameworks and SQL engines (Spark SQL, Redshift, PostgreSQL) for data marts and analytics
- Hands-on with Airflow (or equivalent) to orchestrate ETL workflows and GitLab CI/CD or Jenkins for pipeline automation
- Familiar with relational (PostgreSQL, Redshift) and NoSQL (MongoDB) stores: data modeling, indexing, partitioning, and schema evolution
- Proven ability to implement scalable storage solutions: tables, indexes, partitions, materialized views, columnar encodings
- Skilled in query optimization: execution plans, sort/distribution keys, vacuum maintenance, and cost-optimization strategies (cluster resizing, Spectrum)
- Experience with cloud platforms (AWS): S3/EMR/Glue, Redshift and containerization (Docker, Kubernetes)
- Infrastructure as Code using Terraform or CloudFormation for provisioning and drift detection
- Knowledge of MLOps/LLMOps: auto-scaling ML systems, model registry management, and CI/CD for model deployment
- Strong problem-solving, attention to detail, and the ability to collaborate with cross-functional teams
JOB SUMMARY
Data Engineer
SOURCEO PTE. LTD.
Singapore
9 days ago
N/A
Full-time
Data Engineer