Job Title: Advisor data engineer
Experience:
8-10 Years
Notice
period: Immediate to 15 days
Location: Bangalore
JD:
Responsibilities:
· Design, develop, and maintain Python
applications and scripts for data processing, analytics, and automation tasks.
· Utilize Apache Spark for big data
processing, data transformations, and machine learning tasks.
· Design and implement data pipelines,
ETL processes, and data workflows using Spark and Python.
· Work with cloud technologies such as
Amazon S3, AWS services (e.g., EC2, Lambda, Redshift), or Azure services (e.g.,
Blob Storage, Azure Functions) for data storage, processing, and deployment.
· Develop and optimize SQL queries,
data models, and database interactions for efficient data retrieval and
manipulation.
· Collaborate with data engineers,
data scientists, and business stakeholders to understand data requirements and
deliver scalable data solutions.
· Implement best practices for code
quality, testing, version control, and documentation.
· Monitor and troubleshoot data
pipelines, performance issues, and system integrations.
· Stay updated with the latest trends,
tools, and technologies in Python, Spark, and cloud computing.
· Provide technical leadership, mentorship,
and guidance to junior team members.
Requirements:
· Bachelor's degree in Computer
Science, Engineering, or a related field.
· 8-10 years of hands-on experience in
Python development, including libraries such as Pandas, NumPy, and SQLAlchemy.
· Strong proficiency in Apache Spark
for big data processing, data transformations, and analytics.
· Experience working with cloud
platforms such as Amazon Web Services (AWS), Microsoft Azure, or Google Cloud
Platform (GCP).
· Knowledge of cloud services like
Amazon S3, EC2, Lambda, Redshift, or Azure Blob Storage, Azure Functions.
· Solid understanding of SQL, database
design, and optimization techniques.
· Familiarity with containerization
technologies like Docker and orchestration tools like Kubernetes is a plus.
· Excellent problem-solving,
analytical thinking, and communication skills.
· Ability to work independently and
collaboratively in a fast-paced environment.
· Experience with Agile methodologies
and DevOps practices is desirable.
V 0.0.1