MonoSpear Technologies - Azure Databricks Engineer

Job Title : Azure Databricks Expert (3 years of experience). Job Description : We are seeking an experienced Azure Databricks Expert with hands-on expertise in Spark and PySpark to join our team. The ideal candidate will have a minimum of 3 years of experience in delivering Azure-based projects and working with various Azure services such as Azure Data Lake Storage (ADLS), Azure Data Factory (ADF), Azure Synapse Analytics, and Azure SQL Data Warehouse (SQL DW). As an Azure Databricks Expert, you will be responsible for developing and optimizing data pipelines, implementing scalable data solutions, and contributing to end-to-end data workflows. You will also be required to provide guidance on best practices for working with Databricks in the Azure cloud environment. Key Responsibilities : - Design, implement, and optimize data pipelines using Databricks, Spark, and PySpark for large-scale data processing. - Work with Azure Data Lake Storage (ADLS) to manage and store large volumes of structured and unstructured data. - Utilize Azure Data Factory (ADF) for orchestrating and automating data workflows. - Implement end-to-end data solutions using Azure Synapse Analytics and Azure SQL Data Warehouse (SQL DW) for data warehousing and analytics. - Collaborate with data engineers, data scientists, and stakeholders to build scalable, high-performance data pipelines. - Troubleshoot and optimize performance issues related to Databricks and Spark jobs. - Contribute to the development of best practices for data processing, pipeline design, and data integration. - Ensure data solutions are designed and implemented with a focus on scalability, reliability, and security. - Document processes, workflows, and code to ensure maintainability and knowledge sharing. Qualifications : - Minimum 3 years of experience in working with Databricks, Spark, and PySpark. - Hands-on experience with Azure Data Lake Storage (ADLS) for storing and managing large datasets. - Proficiency in using Azure Data Factory (ADF) for building and orchestrating data pipelines. - Experience with Azure Synapse Analytics and Azure SQL Data Warehouse (SQL DW) for data warehousing and analytics. - Strong understanding of cloud technologies, particularly Azure, and its data platform services. - Proven track record of delivering Azure-based projects with a focus on data engineering and analytics solutions. - Excellent problem-solving skills and the ability to troubleshoot and optimize data pipelines. - Strong programming skills in Python and SQL for data processing and manipulation. - Familiarity with Azure DevOps for version control and continuous integration/continuous deployment (CI/CD) pipelines (optional but preferred). - Strong communication and collaboration skills to work effectively with cross-functional teams. Preferred Skills : - Certification in Azure Data Engineer or other relevant Azure certifications. - Experience with other Azure services such as Azure Machine Learning, Azure Functions, or Azure Logic Apps. (ref:hirist.tech)

Location: in, IN

Posted Date: 5/1/2025