Lead Data Engineer - Scala/Python

Role : Lead Data : : Udaipur Job Description : We are seeking a highly skilled Lead Data Engineer to join our team. The ideal candidate will be responsible for designing, developing, and optimizing data pipelines and architectures to support our growing data and analytics needs. You will work closely with data scientists, analysts, and software engineers to ensure efficient data flow, storage, and : - Design, build, and maintain scalable, efficient, and reliable data pipelines. - Develop and optimize ETL/ELT processes to transform raw data into usable formats. - Implement data warehousing solutions and manage data lakes. - Ensure data quality, consistency, and security across various platforms. - Collaborate with cross-functional teams to understand business requirements and translate them into technical solutions. - Monitor and troubleshoot data pipeline performance issues. - Optimize database queries and improve data retrieval times. - Implement best practices for data governance, compliance, and security. - Stay up-to-date with emerging trends and technologies in data engineering. Requirements : - Bachelor's or Masters degree in Computer Science, Information Systems, or a related field. - 5 years of experience in data engineering or a related role. - Proficiency in programming languages such as Python, Java, or Scala. - Strong experience with SQL and relational databases (e.g., PostgreSQL, MySQL, SQL Server). - Hands-on experience with big data technologies (e.g., Hadoop, Spark, Kafka). - Experience with cloud platforms such as AWS, Azure, or GCP (e.g., Redshift, BigQuery, Snowflake). - Knowledge of data modeling, ETL frameworks, and data warehouse design. - Familiarity with CI/CD pipelines and DevOps practices for data engineering. - Strong problem-solving skills and ability to work in an agile environment. Preferred : - Experience with real-time data processing and streaming technologies. - Knowledge of machine learning pipelines and AI-driven analytics. - Understanding of data governance, privacy regulations (GDPR, CCPA), and security best practices. - Experience with containerization and orchestration tools (Docker, Kubernetes, Airflow). (ref:hirist.tech)

Location: jaipur, IN

Posted Date: 5/1/2025