About the Role:

We are seeking a highly skilled Senior Big Data Platform Engineer with strong IT experience, including 9+ years of specialisation in Big Data technologies and platform engineering. The ideal candidate will possess in-depth expertise in managing and administering large-scale Big Data ecosystems, particularly within the banking and financial sectors.

Key Responsibilities:

• Maintain and administer Big Data platforms (Hadoop, Confluent Kafka, Spark, Hive, HBase, Impala, Druid, etc.) across multiple environments (DEV, UAT, QA, PROD).
• Automate deployment and configuration using Ansible and other scripting tools.
• Provide platform support, performance tuning, and capacity planning for Kafka and Hadoop clusters.
• Implement security protocols including Kerberos, TLS/SSL, and Sentry.
• Work closely with engineering and infrastructure teams to design scalable, secure Big Data solutions.
• Manage OS patching, cluster upgrades, and disaster recovery procedures.
• Supervise scheduled workloads (e.g., via Airflow or Control-M).

Requirements:

• Bachelor's degree in Science, Engineering, Technology

Proven experience with:

• Hadoop ecosystem (HDFS, YARN, Hive, HBase, Spark, Sqoop, Oozie, etc.)
• Confluent Kafka & Kafka cluster administration
• Security integration (Kerberos, Sentry, TLS/SSL)
• Cloud platforms: AWS, Azure
• Monitoring tools: ELK, Grafana
• Strong knowledge of Ansible and scripting (Shell)
• Excellent communication, troubleshooting, and team collaboration skills.

Preferred Qualifications:

• Hands-on experience working in banking domain
• Familiarity with Druid, Alluxio, Cloudera Director, and CDSW
• Experience in hybrid cloud deployments and on-premise cluster setups
• Familiarity with DevOps principles for Big Data environments