Job Description
Senior Data Engineer - Data Migration
Exp - 5 to 7years
Np- 15days to Immediate
5+ years of working experience in scalable data pipeline using Hadoop, Apache Spark, Spark SQL, Kafka, Nifi, ETL and Incremental Data Load with Big data technologies.
Experience of working with Databases like Oracle, Netezza and have strong SQL knowledge
Developing scalable streaming solutions based on Hadoop/ Big Data stack such as HDFS, Ozone, Hive/ Impala, Apache Spark and Kafka.
Experience in Data Engineering and implementing multiple end-to-end DW projects in Big Data environment
Experience of building data pipelines through Git, Shell, Spark with Java/Python on Hadoop or Object storage (S3/CEPH)
Strong analytical skills required for debugging production issues, providing root cause and implementing mitigation plan
Strong communication skills - both verbal and written ? and strong relationship, collaboration skills and organizational skills
Nice to have development experience in Python based AI/ML workloads on Hadoop.
Experience of working on Nifi, APIs and Unix Environment will be an added advantage
Experience of working in Agile teams
Abili
- Nice to have development experience in Python based AI/ML workloads on Hadoop.
- Experience of working on Nifi, APIs and Unix Environment will be an added advantage
- Experience of working in Agile teams
- Abili
- ty to multi-task across multiple projects, interface with external / internal resources and provide technical leadership to Junior team members
- Ability to be high-energy, detail-oriented, proactive and able to function under pressure in an independent environment along with a high degree of initiative and self-motivation to drive results
- Ability to quickly learn and implement new technologies, and perform POC to explore best solution for the problem statement
- Contribute ideas to help ensure that required standards and processes are in place and actively look for opportunities to enhance standards and improve process efficiency
- Perform assigned tasks and production incident independently
- Proven track record of delivering and willingness to roll up sleeves to get the job done.