Azure Data Engineer (Lead) with skills Data Engineering, Python, ETL, Databricks, Azure Data Factory, Pyspark for location Bangalore, India
ROLES & RESPONSIBILITIES
Qualifications:
Bachelor’s degree in Data Science, Statistics, Mathematics, Computer Science, Economics, or a related field.
8+ years of experience in data engineering, with a focus on scalable architecture design.
Strong proficiency in Python, PySpark, Python-based UI libraries and Gradio or Streamlit.
Strong proficiency in Python, especially in Pandas and NumPy.
Strong proficiency in Databricks environment and Native Apps
Hands-on experience with Databricks (notebooks, jobs, lakehouse architecture).
Experience building charts and dashboards using Matplotlib, Seaborn, or Plotly.
Experience in python webapp and rest API development
Good to have understanding of SAS (PROC steps, data handling, stored processes).
Understanding of data structures, statistical methods, and basic analytical modeling.
Good understanding of cloud data architecture and deployment best practices.
JD:
Analyze existing SAS web-based forecasting and reporting applications.
Convert SAS stored processes and PROC logic into PySpark equivalent Native Apps.
Translate SAS data handlers and preprocessing routines to Python-based scripts.
Rewrite forecasting logic and model calculations in Python.
Develop interactive dashboards using Python + Matplotlib/Plotly
Design and deploy user-friendly Apps for real-time interaction with data and forecasting outputs.
Package and deploy the final application within the Databricks App environment.
Collaborate with business SMEs to understand requirements and ensure functional parity with legacy SAS systems.
EXPERIENCE
- 8-11 Years
SKILLS
- Primary Skill: Data Engineering
- Sub Skill(s): Data Engineering
- Additional Skill(s): Python, ETL, Databricks, Azure Data Factory, Pyspark
ABOUT THE COMPANY
Infogain is a human-centered digital platform and software engineering company based out of Silicon Valley. We engineer business outcomes for Fortune 500 companies and digital natives in the technology, healthcare, insurance, travel, telecom, and retail & CPG industries using technologies such as cloud, microservices, automation, IoT, and artificial intelligence. We accelerate experience-led transformation in the delivery of digital platforms. Infogain is also a Microsoft (NASDAQ: MSFT) Gold Partner and Azure Expert Managed Services Provider (MSP).
Infogain, an Apax Funds portfolio company, has offices in California, Washington, Texas, the UK, the UAE, and Singapore, with delivery centers in Seattle, Houston, Austin, Kraków, Noida, Gurgaon, Mumbai, Pune, and Bengaluru.