Senior Data Engineer - Python/Scala

Role : Sr. Data Engineer Mode : Hybrid Location : Chennai Qualifications : - 5-8 years of experience in data engineering, with a focus on building and managing data pipelines. - Strong proficiency in relational databases (e.g., MySQL, PostgreSQL) and NoSQL databases (e.g., MongoDB, Cassandra). - Experience in building data pipelines with data warehouses like Snowflake, Redshift - Experience in processing unstructured data stored from S3 using Athena, Glue etc. - Hands-on experience with Kafka for real-time data streaming and messaging. - Solid understanding of ETL processes, data integration, and data pipeline optimization. - Proficiency in programming languages like Python, Java, or Scala for data processing. - Experience with Apache Spark for big data processing and analytics. - Familiarity with cloud platforms like AWS, GCP, or Azure for data infrastructure is a plus. - Excellent problem-solving skills and attention to detail. - Strong communication and collaboration skills, with the ability to work effectively in a team Responsibilities : - Design, build, and maintain efficient and scalable data pipelines to support data integration and transformation across various sources. - Work with relational databases (e.g., MySQL, PostgreSQL, etc.) and NoSQL databases (e.g., MongoDB, Cassandra, etc.) to manage and optimize large datasets. - Utilize Apache Spark for distributed data processing and real-time analytics. - Implement and manage Kafka for data streaming and real-time data integration between systems. - Collaborate with cross-functional teams to gather and translate business requirements into technical solutions. - Monitor and optimize the performance of data pipelines and architectures, ensuring high availability and reliability. - Ensure data quality, consistency, and integrity across all systems. - Stay up-to-date with the latest trends and best practices in data engineering and big data technologies. (ref:hirist.tech)

Location: chennai, IN

Posted Date: 5/1/2025