Data Engineer - ETL

Job Title : Data : Bangalore : 3 : the Opportunity : We are urgently looking for experienced Data Engineers to join our team at Hexamobile, Bangalore. Ideal candidates will have a strong background in Python, PySpark, and ETL processes, with Azure Cloud experience being a strong : - Design, develop, and maintain scalable and efficient data pipelines using Python and PySpark. - Build and optimize ETL (Extract, Transform, Load) processes to ingest, clean, transform, and load data from various sources into data warehouses and data lakes. - Work with large and complex datasets, ensuring data quality, integrity, and reliability. - Collaborate closely with data scientists, analysts, and other stakeholders to understand their data requirements and provide them with clean and well-structured data. - Monitor and troubleshoot data pipelines, identifying and resolving issues to ensure continuous data flow. - Implement data quality checks and validation processes to maintain high data accuracy. - Develop and maintain comprehensive documentation for data pipelines, ETL processes, and data models. - Optimize data systems and pipelines for performance, scalability, and cost-efficiency. - Implement data security and governance policies and procedures. - Stay up-to-date with the latest advancements in data engineering technologies and best practices. - Work in an agile environment, participating in sprint planning, daily stand-ups, and code reviews. - Contribute to the design and architecture of our data Skills : - Python : Strong proficiency in Python programming, including experience with data manipulation libraries (e.g., Pandas, NumPy). - PySpark : Extensive hands-on experience with Apache Spark using PySpark for large-scale data processing and distributed computing. - ETL Processes : Deep understanding of ETL concepts, methodologies, and best practices. Proven experience in designing, developing, and implementing ETL pipelines. - SQ L: Solid understanding of SQL and experience in querying, manipulating, and transforming data in relational databases. - Understanding of Databases : Strong understanding of various database systems, including relational databases (e.g., PostgreSQL, MySQL, SQL Server) and potentially NoSQL databases. - Version Control : Experience with version control systems, particularly Git, and platforms like GitHub or GitLab (i.e., working with branches and pull Preferred Skills : Azure Cloud Experience: Hands-on experience with Microsoft Azure cloud services, particularly data-related services such as : - Azure Data Factory - Azure Databricks - Azure Blob Storage - Azure SQL Database - Azure Data Lake Storage - Experience with data warehousing concepts and : - Bachelor's degree in Computer Science, Engineering, or a related field. - Minimum of 3 years of professional experience as a Data Engineer. - Proven experience in building and maintaining data pipelines using Python and PySpark. - Strong analytical and problem-solving skills. - Good verbal and written communication skills. - Ability to work effectively both independently and as part of a team. - Must be available to join Points : - Experience with other big data technologies (Hadoop, Hive, Kafka, Apache Airflow). - Knowledge of data governance and data quality frameworks. - Experience with CI/CD pipelines for data engineering workflows. - Familiarity with data visualization tools (Power BI, Tableau). - Experience with other cloud platforms (AWS, GCP). (ref:hirist.tech)

Location: bangalore, IN

Posted Date: 5/1/2025