Data Engineer - Python/Big Data

We are seeking a talented and experienced Data Engineer to join our team in Noida or Pune. The ideal candidate will have 6-9 years of experience working with large-scale data systems and platforms. This role is responsible for designing, building, and maintaining robust data pipelines, ensuring data quality, and optimizing data processing workflows to meet business needs. If you have expertise in working with big data technologies, cloud platforms, and data modeling, we would love to hear from you. Key Responsibilities : - Design, build, and maintain scalable and high-performance data pipelines for large data sets using technologies such as Hadoop, Spark, Kafka, etc. - Work with data scientists, analysts, and other stakeholders to understand data requirements and deliver solutions that meet business needs. - Integrate, transform, and load (ETL) data from various sources into data lakes, warehouses, or cloud-based systems. - Optimize the performance of data pipelines and workflows for data retrieval, processing, and storage. - Collaborate with cross-functional teams to ensure data consistency, accuracy, and quality. - Manage and monitor data infrastructure, ensuring data integrity and security. - Create and maintain data models, schemas, and documentation for easy data access and understanding. - Troubleshoot data-related issues, identify bottlenecks, and implement solutions. - Leverage cloud platforms like AWS, Azure, or Google Cloud to build, deploy, and scale data systems. - Ensure the scalability and efficiency of data systems, including managing real-time data ingestion and processing. Required Skills & Qualifications : - 6-9 years of experience in data engineering, big data systems, or data integration. - Strong experience with ETL tools and data integration frameworks (e.g., Apache NiFi, Talend, Informatica). - Proficiency in programming languages like Python, Java, Scala, or SQL for data manipulation and pipeline creation. - Expertise with cloud platforms (AWS, Azure, or Google Cloud) and data storage technologies (e.g., S3, Redshift, BigQuery, or Snowflake). - Hands-on experience with Hadoop ecosystem (MapReduce, HDFS, Hive, Pig), Apache Spark, and Kafka. (ref:hirist.tech)

Location: in, IN

Posted Date: 5/1/2025