About the company:
The Company is a global leader in providing financial services, specializing in data-driven solutions. With a focus on innovative technologies, the Company aims to help organizations unlock the full potential of their data. Their team is committed to building cutting-edge solutions and driving business growth through actionable insights.
They are looking for a Data Engineer to join their team. The position would be ideal for someone passionate about working with modern data stacks and collaborating with cross-functional teams to deliver high-quality data solutions.
Roles & Responsibilities:
- Your primary role involves developing and implementing scalable data pipelines, ensuring data is efficiently ingested, transformed, and processed from various sources.
- You are required to manage data storage solutions by optimizing data warehouses and cloud storage systems for better performance and accessibility.
- You'll be responsible to maintain data quality and enforce governance policies, ensuring data integrity and following best practices in security.
- Administering the performance of data pipelines and optimizing query performance is an essential part of your role, ensuring smooth data flow and infrastructure scalability.
- Your role also involves collaborating with data scientists, analysts, and engineers to meet data needs, enabling data-driven insights that drive decision-making.
Expectations & Requirements:
- Should have extensive experience in Python and SQL, with a proven track record of creating and managing data pipelines.
- Quite an acquaintance of modern data architecture, including working with Databricks, DBT, and cloud platforms like AWS (S3, EC2, DMS, Kinesis, Lambda).
- Must be skilled in managing semi-structured data formats such as JSON, and have a solid understanding of tools like Apache Spark, Hadoop, and Kafka.
- Ensure that you deliver scalable and efficient data pipelines, while improving the overall infrastructure and query performance.
- Ensuring quality projects are delivered within defined timelines, while collaborating with team members to resolve any technical challenges is essential to this role.
- You are also expected to bring a strong understanding of CI/CD principles and modular coding, contributing to continuous improvements in software delivery.
