Lead Data Engineer

Responsibilities: Data Pipeline Development and Maintenance: Design, build, and optimize scalable ETL/ELT pipelines to ingest data from diverse sources such as APIs, cloud platforms, and databases. Ensure pipelines are robust, efficient, and capable of handling large volumes of data. Data Integration and Harmonization: Aggregate and normalize marketing performance data from multiple platforms (e.g., Adobe Analytics, Salesforce Marketing Cloud, Deltas EDW, ad platforms). Implement data transformation and enrichment processes to support analytics and reporting needs. Data Quality and Monitoring: Develop and implement data validation and monitoring frameworks to ensure data accuracy and consistency. Troubleshoot and resolve issues related to data quality, latency, or performance. Collaboration with Stakeholders: Partner with marketing teams, analysts, and data scientists to understand data requirements and translate them into technical solutions. Provide technical support and guidance on data-related issues or projects. Tooling and Automation: Leverage cloud-based solutions and frameworks (e.g., AWS) to streamline processes and enhance automation. Maintain and optimize existing workflows while continuously identifying opportunities for improvement. Documentation and Best Practices: Document pipeline architecture, data workflows, and processes for both technical and non-technical audiences. Follow industry best practices for version control, security, and data governance. Continuous Learning and Innovation: Stay current with industry trends, tools, and technologies in data engineering and marketing analytics. Recommend and implement innovative solutions to improve the scalability and efficiency of data systems. Lead and mentor other junior members within the team What you need to succeed (minimum qualifications): Bachelor of Science degree in Computer Science or equivalent At least 6 years of post-degree professional experience as a data engineer developing and maintaining data pipelines Extensive experience with databases and data platforms (Teradata and AWS preferred) 5 Hands-on experience in designing, implementing, managing large scale data and ETL solutions utilizing AWS Compute, Storage and database services (S3, Lambda, Redshift, Glue, Athena) Proficiency in Python, SQL, PySpark, Glue, Lambda, S3 and other AWS tool sets Strong knowledge of relational and non-relational databases Good understanding of data warehouses, ETL, AWS architecture, Airflow and Redshift Ability to create clean, well-designed code and systems Proven ability to work with large and complex datasets Strong analytical and programming skills with the ability to solve data-related challenges efficiently At least 7 years of post-degree professional experience as a data engineer developing and maintaining data pipelines Strong attention to detail and a commitment to data accuracy. Mentor other junior members in the team. Proven ability to learn new data models quickly and apply them effectively in a fast-paced environment. Excellent communication skills with the ability to present complex data findings to both technical and non-technical audiences. What will give you a competitive edge (preferred qualifications): Desired Airline industry experience Desired experience working with marketing and media data Experience working with SAS to develop data pipelines AWS certifications: Solution Architect (SAA / SAP) or Data Analytics Specialty (DAS) Experience migrating data pipelines and systems to modern cloud-based solutions Familiarity with marketing data platforms ID: DELYB01

Location: bangalore, IN

Posted Date: 6/11/2025