Oneture Technologies - Data Engineer - AWS/Python

Role : Data Engineer (AWS) Location : Mumbai, Pune Experience : 1 to 3 yrs Technologies / Skills : Advanced SQL, AWS, ETL pipelines development and optimization, Data Modelling, Spark - Scala/Python, Airflow, Kafka Job Overview : We are looking for a Data Engineer to join our data team to solve data-driven critical business problems. You will be responsible for data infrastructure and analytics initiatives. The Data Engineer will collaborate with software developers, database architects, data analysts, data scientists and platform team on data initiatives and will ensure optimal data delivery architecture is consistent throughout ongoing projects. The right candidate should have hands on in developing a hybrid set of data-pipelines depending on the business : - Design, develop, and maintain scalable data pipelines to ingest, process, and transform large volumes of data from various sources using AWS services such as AWS Glue, AWS Lambda, and AWS Step Functions - Implement data modeling techniques to structure and organize data for efficient analysis. - Build and maintain data warehouses and data lakes on AWS platforms such as Amazon Redshift, Amazon S3, and Amazon Athena, optimizing storage and query performance. - Design and implement ETL processes to extract data from source systems, transform it into usable formats, and load it into target systems, ensuring data quality and reliability. - Align architecture with business requirements and provide solutions which fits best to solve the business problems. - Monitor data pipelines and infrastructure performance, troubleshoot issues, and optimize processes for improved efficiency, scalability, and cost-effectiveness. - Collaborate with the team to identify opportunities for performance optimization. Education level : Bachelor's degree (B.E. / B. Tech) in Computer Science or equivalent from reputed institute Experience : - 1 to 3 years hands-one as a Data Engineer. - Financial services industry experience or familiarity with the stock exchange/capital market domain will be an added advantage. Required Technical Skills : - Advanced SQL knowledge, Data-Modelling and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of databases. - Experience in developing and optimizing ETL pipelines, big data pipelines, and data-driven architectures. - Strong understanding of AWS Cloud services, including but not limited to S3, EC2, EMR, Glue, Lambda, Amazon Redshift, AWS IAM etc. - AWS Data Analytics certification is preferred. - Must have strong big-data core knowledge & experience in programming using Spark - Python/Scala - Good to have experience in working with real-time data streaming using Kafka - Experience with orchestrating tool like Airflow or similar - Build processes supporting data transformation, data structures, metadata, dependency and workload management. - Good understanding of Git workflow, Test-case driven development and using CICD is good to have - Good to have some understanding of Delta tables About Oneture Technologies : Founded in 2016, Oneture is a cloud-first, full-service digital solutions company, helping clients harness the power of Digital Technologies and Data to drive transformations and turning ideas into business realities. Our team is full of curious, full-stack, innovative thought leaders who are dedicated to providing outstanding customer experiences and building authentic relationships. We are compelled by our core values to drive transformational results from Ideas to Reality for clients across all company sizes, geographies, and industries. Oneture team delivers full lifecycle solutions-from ideation, project inception, planning through deployment to ongoing support and maintenance. Our core competencies and technical expertise includes Cloud powered: Product Engineering, Big Data and AI ML. Our deep commitment to value creation for our clients and partners and "Startups-like agility with Enterprises-like maturity" philosophy has helped us establish long-term relationships with our clients and enabled us to build and manage mission-critical platforms for them. (ref:hirist.tech)

Location: Mumbai, IN

Posted Date: 4/24/2024