Senior Data Engineer (Python)
Salary: 70k- 100k
My client is the worlds leading connected car services provider used by millions of drivers and organisations such as Apple, TomTom and over 20 automotive brands.
They are looking for Data Engineers to help support the Data Science teams ingestion, ETL, infrastructure, and the production of models. The system is currently responsible for making sense of over a billion data points per day.
The ideal candidate will have a deep love for engineering, want to be in a machine learning and data science ecosystem, and get a kick from delivering great code into production. You'll be part of a close knit team where your opinion counts, have a lot of scope to be creative and come up with new ideas, and you'll feel at home working with extremely bright colleagues where learning is a top priority.
- Developing fast and reliable APIs
- Designing and deploying big data capable infrastructure and software components using infrastructure as code (AWS, CDK)
- Developing highly scalable big data processing pipelines with Spark
- Developing streaming processing pipelines using Kafka
- Championing software best practices
- 4+ years professional Software/Data Engineering experience
- Strong CompSci background
- Data oriented engineer
- Extensive experience in Python, including a thorough knowledge of the Python data science/engineering ecosystem (e.g. Pandas, Numpy)
- Experience with containerisation (e.g. Docker), and container orchestration (e.g. k8s, ECS)
- PySpark and working with big data, geospatial data
- AWS, CDK
- API development (eg. FastAPI or Starlette)
- Experience with workflow management tools (Apache Airflow)
- Experience with setting up CI/CD pipelines, ideally in an ML environment