Role : Kafka Engineer
Location: London OR Sheffield (2-3 days per week onsite)
Type: Contract
Mandatory: Kafka & Python
Role Overview:
CLIENT is seeking a skilled Kafka and Python Automation Engineer to contribute to its innovative financial technology initiatives. The ideal candidate will have a strong technical background in event-driven architecture, automation frameworks, and modern development practices. The role involves building and managing scalable Kafka environments, creating automation scripts, and ensuring robust integration with various enterprise systems.
Key Responsibilities:
Kafka Infrastructure Management:
- Design, deploy, and maintain Apache Kafka clusters in a secure and scalable manner.
- Implement Kafka Connectors, Streams, and KSQL for data pipeline automation.
- Monitor Kafka performance, resolve bottlenecks, and optimize throughput.
Python Automation:
- Develop Python-based automation scripts for system monitoring, error handling, and data pipeline management.
- Implement CI/CD pipelines integrating Python scripts for testing and deployment.
- Ensure reusability of automation frameworks and tools.
System Integration:
- Integrate Kafka with CLIENT’s core banking systems and other third-party applications.
- Collaborate with teams to implement APIs for seamless data exchange.
- Work with DevOps tools (e.g., Jenkins, Ansible) to automate deployment processes.
Performance and Troubleshooting:
- Analyze system logs and errors, resolving issues related to Kafka and Python scripts.
- Develop tools to monitor and alert potential system failures.
- Stakeholder Collaboration:
- Work closely with CLIENT’s infrastructure and security teams to meet compliance standards.
- Partner with cross-functional teams to design solutions addressing business needs.
Skills and Qualifications:
Technical Expertise:
- Strong proficiency in Apache Kafka (setup, configuration, and administration).
- Advanced knowledge of Python for automation and scripting.
- Familiarity with Kafka Streams, Connectors, and Schema Registry.
Automation and Tools:
- Experience with tools like Jenkins, Ansible, Terraform, or similar for automation.
- Knowledge of CI/CD pipelines and DevOps practices.
Cloud and Database:
- Experience with cloud platforms such as GCP, AWS, or Azure.
- Understanding of relational (SQL) and NoSQL databases.
Soft Skills:
- Strong analytical and problem-solving skills.
- Excellent communication and collaboration abilities.
Preferred Qualifications:
- Experience in financial services or banking.
- Familiarity with security frameworks and encryption in Kafka.
- Knowledge of additional languages such as Java or Scala is a plus.
