Key Responsibilities

• As part of the security operation teams, operate, support and maintain a Hadoop Date Lake environment in a dedicated customer environment
• Perform data engineering to extract, transform, load for data science analysis
• Work with customer’s and internal data science team to design effective data models for optimal storage and retrieval
• Ensure data segregation in accordance to role-base access control matrix
• Maintain up-to-date technical documentation for deployed systems

Qualifications

• At least 3 years of production experience working with Hadoop data engineering
• Demonstrate production experience in data engineering, data management and/or cluster management
• Experience designing data queries against data in the HDFS environment using tools such as Apache Hive
• Experience in implementing MapReduce, Spark jobs
• Experience maintain and expanding multi-node Hadoop clusters
• Experience in systems administration or DevOps experience with one or more open-source operating systems
• Experience implementing operational best practices such as alerting, monitoring, and metadata management
• Strong understanding with various enterprise security practices and solutions such as LDAP and/or Kerberos
• Familiarity with scripting tools such as bash shell scripts, Python and/or Perl
• Experience with Splunk Enterprise and Isilon storage is an added advantage
• Good verbal and written communications