Azure Data Engineer - Python/PySpark

Azure Data Engineer - Python/PySpark We are seeking a skilled and experienced Azure Data Engineer with expertise in PySpark and Python coding. The ideal candidate should have a strong background in data engineering, with a focus on developing solutions using PySpark and Python within the Azure Databricks environment. Key Responsibilities : - PySpark Development : Lead and contribute to the development of data engineering solutions using PySpark and Python within the Azure Databricks platform. - Azure Databricks Expertise : Utilize deep knowledge and experience with Azure Databricks to design, implement, and optimize data workflows and pipelines. - Coding Proficiency : Demonstrate a minimum of 3 years of hands-on experience in PySpark with Python coding, ensuring high-quality and efficient code development. - Data Integration : Work collaboratively with cross-functional teams to integrate data from various sources, ensuring data accuracy, consistency, and reliability. - Performance Optimization : Identify and implement performance optimization techniques to enhance the efficiency of data processing and analytics. - Data Transformation : Develop and maintain data transformation processes to support the extraction, transformation, and loading (ETL) of data into data lakes or warehouses. - Collaboration : Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and contribute to the design and implementation of scalable and reliable data : 1. Bachelor's degree in Computer Science, Information Technology, or a related field. 2. Minimum of 6 years of experience in data engineering, with a focus on PySpark and Python development. 3. Proven experience (at least 3 years) in PySpark with Python coding. 4. Solid understanding of Azure Databricks and its components. 5. Strong problem-solving and analytical skills. 6. Excellent communication and collaboration abilities. 7. Experience with data lakes, data warehouses, and ETL processes. 8. Familiarity with big data technologies and distributed computing. (ref:hirist.tech)

Location: India, IN

Posted Date: 4/24/2024