About the role
Job Title: Senior Data Engineer
Locations: Multiple, Bangalore, Chennai, Hyderabad, India
Work Arrangement: Remote Work from Home
About This Role:
We are seeking a talented and experienced Senior Data Engineer to join our data engineering team. You will be responsible for designing, building, and maintaining scalable data pipelines and infrastructure on cloud platforms. This role requires expertise in modern data engineering tools and a strong foundation in data transformation and optimization.
Key Responsibilities
Design and develop robust, scalable data pipelines using Azure Data Factory (ADF)
Build and optimize data processing workflows using Databricks and Apache Spark
Implement declarative pipeline solutions using Lakehouse architecture and best practices
Write and optimize complex SQL queries for data transformation and analysis
Develop PySpark applications for large-scale data processing
Collaborate with data scientists, analysts, and stakeholders to understand data requirements
Ensure data quality, security, and compliance across all pipelines
Monitor, troubleshoot, and optimize data pipeline performance
Document technical solutions and maintain best practices documentation
Mentor junior team members and contribute to team knowledge sharing
What we're Looking For
We need a Senior Data Engineer who is:
Proficient in building scalable data solutions using Azure and Databricks
Skilled in SQL and PySpark for data transformation and processing
Capable of implementing declarative, maintainable pipeline architectures
Detail-oriented with a focus on data quality and performance optimization
Collaborative and able to work effectively with cross-functional teams
Proactive in staying current with emerging data engineering technologies and best practices
Communicative and able to explain complex technical concepts clearly
Required Skills & Experience
Must-Have Technical Skills
Azure Data Factory (ADF)“ Proven experience designing and implementing data integration solutions
Databricks“ Hands-on experience with Databricks platform and workspace management
Apache Spark “ Strong expertise in distributed data processing using Spark framework
Lakeflow & Declarative Pipelines – Experience building declarative, maintainable data pipelines
SQL – Advanced proficiency in writing optimized SQL queries for complex data transformations
PySpark “ Strong programming skills in PySpark for data processing and ETL operations
Additional Desired Qualifications:
Experience with cloud platforms (Azure, AWS, or GCP)
Knowledge of data warehousing concepts and modern data architecture
Familiarity with version control systems (Git)
Understanding of CI/CD pipelines and DevOps practices
Experience with data quality frameworks and testing methodologies
Knowledge of big data technologies and distributed computing
Required Experience
5+ years of professional experience in data engineering or related roles
3+ years of hands-on experience with cloud-based data platforms
Proven track record of designing and implementing production-grade data pipelines
Strong problem-solving skills and attention to detail
Education
Bachelor's Degree in Computer Science, Engineering, Mathematics or related field OR
Related Field or Professional experience
Why Join Us?
Work with cutting-edge data engineering technologies
Collaborate with a talented and innovative team
Opportunity to build impactful data solutions at scale
Professional growth and continuous learning opportunities
Competitive compensation and benefits package
How to Apply
If you meet the above requirements and are excited about this opportunity, please submit your resume, a cover letter, and links to your portfolio or GitHub profile (if available).
Originally posted on Himalayas