Edgematics - Technical Lead - Data Engineering

Job Title: Tech Lead (Sr. Data Engineer with 9 Years of Experience). Location : Pune. Job Type : Full-time, Onsite. Edgmatics Technologies is a leading global consulting and solutions integrator in the field of data management, business intelligence, and analytics. We specialize in delivering end-to-end data management solutions to help organizations make data-driven decisions and achieve their business objectives. About This Role. We are looking for a talented and experienced Data Engineer with Tech Lead with hands-on expertise in any ETL Tool with full knowledge about CI/CD practices with leading a team technically more than 5 and client facing and create Data Engineering, Data Quality frameworks. As a tech lead must ensure to build ETL jobs, Data Quality Jobs, Big Data Jobs performed performance optimization by understanding the requirements, create re-usable assets and able to perform production deployment and preferably worked in DWH appliances Snowflake / redshift / Synapse. Responsibilities : - Work with a team of engineers in designing, developing, and maintaining scalable and efficient data solutions using Any Data Integration (any ETL tool like Talend / Informatica) and any Big Data technologies. - Design, develop, and maintain end-to-end data pipelines using Any ETL Data Integration (any ETL tool like Talend / Informatica) to ingest, process, and transform large volumes of data from heterogeneous sources. - Have good experience in designing cloud pipelines using Azure Data Factory or AWS Glues/Lambda. - Implemented Data Integration end to end with any ETL technologies. - Implement database solutions for storing, processing, and querying large volumes of structured and unstructured and semi-structured data. - Implement Job Migrations of ETL Jobs from Older versions to New versions. - Implement and write advanced SQL scripts in SQL Database at medium to expert level. - Work with technical team with client and provide guidance during technical challenges. - Integrate and optimize data flows between various databases, data warehouses, and Big Data platforms. - Collaborate with cross-functional teams to gather data requirements and translate them into scalable and efficient data solutions. - Optimize ETL, Data Load performance, scalability, and cost-effectiveness through optimization techniques. - Interact with Client on a daily basis and provide technical progress and respond to technical questions. - Implement best practices for data integration. - Implement complex ETL data pipelines or similar frameworks to process and analyze massive datasets. - Ensure data quality, reliability, and security across all stages of the data pipeline. - Troubleshoot and debug data-related issues in production systems and provide timely resolution. - Stay current with emerging technologies and industry trends in data engineering technologies, CI/CD, and incorporate them into our data architecture and processes. - Optimize data processing workflows and infrastructure for performance, scalability, and cost-effectiveness. - Provide technical guidance and foster a culture of continuous learning and improvement. - Implement and automate CI/CD pipelines for data engineering workflows, including testing, deployment, and monitoring. - Perform migration to production deployment from lower environments, test & validate. Must Have Skills : - Must be certified in any ETL tools, Database, Cloud. (Snowflake certified is more preferred). - Must have implemented at least 3 end-to-end projects in Data Engineering. - Must have worked on performance management optimization and tuning for data loads, data processes, data transformation in big data. - Must be flexible to write code using JAVA/Scala/Python etc.as required. - Must have implemented CI/CD pipelines using tools like Jenkins, GitLab CI, or AWS CodePipeline. - Must have managed a team technically of min 5 members and guided the team technically. - Must have the Technical Ownership capability of Data Engineering delivery. - Strong communication capabilities with client facing. - Bachelor's or Master's degree in Computer Science, Engineering, or a related field. - 5 years of experience in software engineering or a related role, with a strong focus on Any ETL Tool, database, integration. - Proficiency in Any ETL tools like Talend , Informatica etc for Data Integration for building and orchestrating data pipelines. - Hands-on experience with relational databases such as MySQL, PostgreSQL, or Oracle, and NoSQL databases such as MongoDB, Cassandra, or Redis. - Solid understanding of database design principles, data modeling, and SQL query optimization. - Experience with data warehousing, Data Lake , Delta Lake concepts and technologies, data modeling, and relational databases. (ref:hirist.tech)

Location: pune, IN

Posted Date: 5/1/2025