RP International are currently looking for an ambitious Pyspark Data Engineer to join a product engineering team. This is a hybrid role, paying up to 80K dependent on experience & alignment to the position.
Responsibilities:
- Working in one of the squads, dealing with data ingestion and storage processes, customer ML analytics platform and insights and visualisation dashboard.
- Working closely as part of a team of data engineers based in Spain, UK and USA. Don’t expect to only work together with them but to be part of product, sales and client services discussions too.
- Facing challenging problems and finding new solutions for them.
- Giving back to the open-source community by contributing to the tools used, and releasing new ones, for the benefit of everybody.
Qualifications and Required Skills:
‘ Experience engineering data platforms, advanced analytics tools, machine learning capabilities or artificial intelligence products.
‘ Some experience of microservice architectures and testing practices.
‘ Some experience with CI/CD tools.
‘ Some experience in Agile/Lean environments would be an advantage
‘ Some experience understanding distributed application architecture would be an advantage.
Technologies and Approaches used:
‘ Python and pyspark to develop most of the backend code, including data pipelines and training and inference processes
‘ Pair programming, testing and continuous integration and deployment as daily development practices.
‘ Github platform as git repository, issue tracker, continuous integration and package registry.
Cassandra, PostgreSQL, Elasticsearch, S3, HDFS as the main data stores.
Docker, Nomad, Terraform, Consul and Vault in the operational stack.
Spark for transforming multi-terabyte sets of data into valuable insights.
This is a great opportunity to work alongside the engineering squads to build data capabilities within their product and engineering scalability into its foundation to enable it to consume and process trillions of rows of data reliably and quickly. Apply now to be part of a company that strives in developing the skills of their staff members as well as their capabilities and character.
This role does NOT offer any type of sponsorship, thank you!
Submit your CV now for a confidential conversation!
