Job Title: Data Engineer
Experience:
12-16 Years
Notice
Period: Immediate to 15 days
Location: Bangalore
JD:
· Responsible for the delivery of the
development activities he/she owns including Data Engineering standards,
patterns and best practices.
· Work with team to ensure standards
and patterns are followed.
· Determines organizational strategies
for data integrity validation processes.
· Design and develop new data
ingestion patterns into IntelDS raw and/or unified data layers based on the
requirements and needs for connecting new data sources or for building new data
objects. Working in ingestion patterns allow to automate the data pipelines.
· Participate to and apply DevSecOps
practices by automating the integration and delivery of data pipelines in a
cloud environment. This can include the design and implementation of end-to-end
data integration tests and/or CI/CD pipelines.
· Analyze existing data models,
identify and implement performance optimizations for data ingestion and data
consumption. The objective is to accelerate data availability within the
platform and to consumer applications. Target technologies are Apache Spark,
Apache Presto and RedShift.
· Support client applications in
connecting and consuming data from the platform, and ensure they follow our
guidelines and best practices.
· Participate in the monitoring of the
platform and debugging of detected issues and bugs.
· Partner with the Data modelers to
drive improvements and enhancements to the current data landscape and future
strategy.
· Identify opportunities for uses of
those technologies to enhance current or anticipated information systems and
business goals needs
· Evaluates existing applications that
could address client requirements and makes recommendations from complex
projects.
· Assists in the development of Data
Engineering-related Service Level Agreements.
· Communicates risks and ensures
understanding of these risks.
· Strong analytical &
problem-solving skills
· Strong verbal and business communication
skills
· Skill in identifying data issues and
anomalies during the analysis
· Strong business acumen &
demonstrated an aptitude for analytics that incite action
QUALIFICATIONS:
· B.E / B TECH/MASTERS IN COMPUTER
SCIENCE, ELECTRONICS, RELEVANT TECHNICAL CERTIFICATION
Technology
Skills and Project Experience:
· 12 years of experience in modeling
and business system designs.
· 10 years hands on experience as Data
Engineer/Architect with proven experience on Big Data and Data Lakes on AWS
· 7 years of experience in below
Technologies
· Scripts/Programming language: SQL,
Python, PROC, Windows Functions
· GitHub
· AWS: S3, RedShift, Glue
· Big Data: Spark, Hive, Presto
V 0.0.1