Associate Technical Data Engineer - Java/Python

PRIMARY RESPONSIBILITIES : - Harness modern application best practices with code quality, test Coverages, Agile Development, DevOps, and Observability and support. - Design, develop, and deploy microservices using Java and Python to facilitate data processing, transformation, and integration. - Create APIs and endpoints to interact with data sources, databases, and other services within the data Pipeline Design and Implementation : - Design and implement data pipelines to ingest, process, and deliver data from diverse sources into our data lake or data warehouse. - Ensure data quality, consistency, and reliability throughout the data pipeline. - Develop data transformation workflows to clean, enrich, and aggregate data as per business requirements. - Implement Extract, Transform, Load (ETL) processes to integrate data from various sources and systems. - Identify performance bottlenecks and optimize data processing workflows to improve overall system efficiency. - Develop and execute unit tests, integration tests, and end-to-end tests to ensure the reliability and accuracy of data pipelines and Set up monitoring and logging systems to proactively identify and resolve issues in data processing workflows. - Investigate and troubleshoot data-related problems, providing timely and Communication : - Work closely with data scientists, analysts, and other stakeholders to understand their data requirements and provide data solutions accordingly. - Collaborate with the development and operations teams to deploy and manage microservices effectively. - Implement OAuth Okta integration for communication between API producers and AND EXPERIENCE : (Academic, Professional, Experience) - 5 Years of experience with Bachelor's / Masters degree in Computer Science, Engineering, or a related field. - Solid experience in developing microservices using Java and Python. - Proficiency in data processing frameworks and libraries, such as Apache Spark, Kafka, or Flink. - Strong understanding of data modeling, data structures, and algorithms. - Experience with ETL tools and methodologies. - Familiarity with data warehousing concepts and databases (e.g., PostgreSQL, MySQL, etc.). - Knowledge of cloud-based data services and platforms (AWS, Azure, GCP) is a plus. - Understanding of DevOps practices and containerization technologies (Docker, Kubernetes) is advantageous. - Excellent problem-solving and analytical skills with a keen eye for detail. - Strong communication skills and ability to work collaboratively in a team-oriented environment. - Significant experience of designing, deploying, and supporting production cloud environments like Azure and Kubernetes - Experience with Azure DevOps CI/CD Tools to build and deploy Java/API packages and API deployment automation. - Hands-on experience in designing and developing high volume REST using API Protocols and Data Formats. - Deep Understanding of Databases, API Frameworks, Governance Frameworks, and expertise in hosting and managing platform environments like : Spark, Flink, Kafka, SpringBoot, BI Tools like Tableau, Alteryx, Governance Tools like Callibra, Soda, Amazon DeeQu Additional Qualifications : - Financial proven experience: Capital markets/Private Equity and/or securities processing - Familiar in NoSQL\NewSQL databases - Strong documentation capability and adherence to testing and release management standards - Design, development, modification and testing of databases designed to support Data Warehousing and BI business teams - Familiarity with SDLC methodologies, defect tracking (JIRA, Azure DevOps, ServiceNow etc.) - Monitor and fine-tune microservices for optimal performance and scalability. - Implement data security best practices and ensure data privacy compliance in all data engineering processes. - Collaborate with the security and compliance teams to uphold data governance standards. Skills : - Candidate must have an analytical and logical thought process for developing project solutions - Strong interpersonal and communication skills; works well in a team environment - Ability to deliver under competing priorities and pressures. - Excellent organizational skills in the areas of code structuring & partitioning, commenting and documentation for team alignment and modifications (ref:hirist.tech)

Location: mumbai, IN

Posted Date: 5/1/2025