Responsibilities:

• Test and validate data pipelines for all new releases.

• Ensure data integrity and completeness after deployments.

• Debug and troubleshoot pipeline issues.

• Implement fixes and document root causes.

• Optimise pipeline performance and efficiency.

• Write clean, maintainable Python code with unit tests.

• Collaborate with data scientists, engineers, and stakeholders.

• Support AWS-based pipeline infrastructure (e.g., S3, Lambda, EC2, CloudWatch).

Requirements:

• Proficient in Python (unit testing, clean code practices).

• Experience in data pipeline development and troubleshooting.

• Familiarity with AWS services (S3, Lambda, EC2, CloudWatch).

• Strong debugging and problem-solving skills.

• Attention to detail and focus on data quality.

• Good to have:

• Basic understanding of data science workflows.

• Experience with CI/CD and version control (e.g., Git).