Data Engineer - Azure Databricks

We need data engineers (both should be able to code) with the below skillsets for the customer service projects. Required Skills : - 5 years of experience in big data and streaming pipelines. - Strong expertise in Apache Spark Structured Streaming (Databricks preferred). - Experience with Kafka and real-time CDC patterns. - Proficiency in Python (PySpark)/ Scala, REST APIs. - Solid understanding of secure API patterns (e.g., OAuth, JWT), AWS IAM, and working with secured secrets (Databricks Secrets / Vaults). - Knowledge of Delta Lake, Spark checkpointing, and best practices for streaming resilience. - Proficiency in Python, SQL, and Apache Spark (PySpark preferred). - Experience with Databricks and Snowflake, including pipeline development and optimization. - Strong understanding of ETL processes, data modeling, and distributed computing. - Ability to work independently and manage multiple tasks in a fast-paced environment. - Hands-on experience with orchestration tools (e.g., Airflow, Databricks Workflows). - Familiarity with cloud platforms (AWS) is a have all of these skills : - Spark Streaming - PySpark, Python, Scala - DataBricks - AWS knowledge or experience is a must. (ref:hirist.tech)

Location: chennai, IN

Posted Date: 5/1/2025