Azure Data Engineer - PySpark/Databricks/Data Factory

Azure Data Engineer Mandatory Skills : Pyspark Databricks Adf Synapse is mandatory Job Description: We are looking for a skilled and motivated Azure Data Engineer to join our team. As an Azure Data Engineer, you will play a crucial role in designing, implementing, and maintaining data solutions on the Microsoft Azure platform. The ideal candidate will have a strong technical background in data engineering, hands-on experience with Azure data services, and the ability to develop efficient and scalable data Pipeline Development: - Design, develop, and maintain scalable data pipelines using Azure data services such as Azure Data Factory, Azure Databricks, and Apache Spark. - Implement efficient Extract, Transform, Load (ETL) processes to move and transform data across various sources. Data Modeling and Integration: - Work with data architects to understand data requirements and implement effective data models. - Integrate data from diverse sources, both on-premises and in the cloud, ensuring data consistency and accuracy. Azure Data Services Utilization: - Utilize Azure SQL Database, Azure Blob Storage, Azure Data Lake Storage, and other Azure data services to store and retrieve data. - Implement and optimize data solutions using Azure Synapse Analytics (formerly SQL Data Warehouse). Data Quality and Governance: - Implement data quality checks and ensure adherence to data governance and compliance standards. - Collaborate with data architects to enforce data quality and integrity across the data pipelines. Collaboration with Cross-Functional Teams: - Collaborate with data scientists, business analysts, and other stakeholders to understand data requirements and deliver effective solutions. - Work closely with IT teams to integrate data solutions into existing systems. Performance Optimization: - Optimize data pipelines for performance, scalability, and cost efficiency. - Conduct performance tuning and troubleshooting to address bottlenecks and enhance overall system Create and maintain comprehensive documentation for data pipelines, data integrations, and ETL processes. - Ensure documentation is kept up-to-date with changes in the data environment. Continuous Learning: - Stay updated on the latest Azure data services and technologies. - Engage in continuous learning and share knowledge with the team. (ref:hirist.tech)

Location: Pune, IN

Posted Date: 4/24/2024