Meaningful work,
meaningful life

Role Overview

We are seeking a Data Engineer Lead who can deliver end-to-end data projects, lead a team of engineers, and collaborate with stakeholders to design scalable solutions. The ideal candidate will have strong technical expertise in PySpark, AWS Glue, Lambda, Postgres, Jenkins, Cloud DevOps, SQS, and experience in data migrations and reporting (PDF/CSV). This role requires leadership, solution design, and the ability to work in overlapping PST/IST time zones.

Key Responsibilities

• Lead and manage a team of up to 7 developers, ensuring timely and high-quality deliverables.

• Engage with stakeholders to gather requirements, define scope, and design solutions aligned with business objectives.

• Deliver end-to-end data engineering projects, including architecture, development, testing, and deployment.

• Build and optimize ETL pipelines using AWS Glue and PySpark for large-scale data processing.

• Develop serverless workflows leveraging AWS Lambda and SQS for event-driven architectures.

• Oversee data migrations from on-premise systems to cloud environments.

• Implement and maintain PostgreSQL databases for analytics and transactional workloads.

• Set up and manage CI/CD pipelines using Jenkins and enforce Cloud DevOps best practices.

• Generate reports in PDF and CSV formats for operational and executive dashboards.

• Ensure data security, compliance, and performance optimization across all solutions.

• Work in overlapping PST/IST time zones to collaborate with global teams.

• Mentor team members and promote best practices in coding, testing, and deployment.

Required Skills & Experience

• 8+ years of experience in data engineering, with at least 3 years in a leadership role.

• Proven ability to manage teams of 10+ engineers and deliver complex projects.

• Strong proficiency in Python, PySpark, and distributed data processing frameworks.

• Hands-on experience with AWS Glue, Lambda, and SQS.

• Solid understanding of PostgreSQL and relational database design.

• Expertise in CI/CD automation using Jenkins and Cloud DevOps tools.

• Experience in report generation (PDF/CSV) and data visualization.

• Excellent communication skills for stakeholder engagement and requirement gathering.

Preferred Qualifications

• AWS certifications (e.g., AWS Certified Data Analytics, Solutions Architect).

• Experience with data governance, cataloging, and metadata management.

• Familiarity with containerization (Docker/Kubernetes) and Infrastructure as Code (Terraform/CloudFormation).