Senior Data Engineer - Hive/Spark

Responsibilities : - Design, develop, and maintain scalable and efficient data pipelines using Azure Data Factory. - Build and optimize data processing workflows using Databricks (or Apache Spark, Hadoop, and Hive). - Develop and maintain data models and data warehousing solutions, applying dimensional modeling concepts. - Write complex and performant queries using T-SQL and PL-SQL. - Develop data visualizations and dashboards using Power BI or equivalent Business Intelligence tools to communicate insights effectively. - Implement data extraction, transformation, and loading (ETL) processes from various data sources into our Azure data platform. - Collaborate with data scientists, analysts, and other stakeholders to understand data requirements and deliver robust solutions. - Ensure data quality, integrity, and performance across all data pipelines and systems. - Utilize Git for version control, adhering to established branching and merging strategies. - Participate actively in Agile Software Development Life Cycle and Scrum methodology. - Troubleshoot and resolve data-related issues in a timely manner. - Contribute to the documentation of data pipelines, data models, and ETL Requirements : - Associate-level experience in Data Engineering and Analytics. - Strong knowledge and practical application of the following technical skills: - T-SQL/PL-SQL for database querying and manipulation. - PySpark for large-scale data processing. - Azure Data Factory for building ETL pipelines. - Databricks (or strong experience with Apache Spark, Hadoop, and Hive for distributed data processing). - Power BI or equivalent Business Intelligence tools (e.g., Tableau, Qlik Sense) for data visualization. - Solid understanding of dimensional modeling and Data Warehouse concepts. - Experience with Git repository for version control and a thorough understanding of branching and merging strategies. - Experience developing in an Agile Software Development Life Cycle and utilizing Scrum Skills (Advantageous) : - Programming skills in Python, PowerShell, .Net, or Java. (ref:hirist.tech)

Location: pune, IN

Posted Date: 5/1/2025