Senior Data Engineer
90,000- 100,000 + equity
‘ Python, Databricks, Terraform
London, twice a week in office
Crypto Wallet tracking
I am working with a startup organisation operating in the crypto space who are looking for a highly specialised Senior Data Engineer to work alongside their Core and Data Science teams and help ensure their data is meeting customer requirements as their database grows.
The role…
As the Senior Data Engineer, it is essential you have senior-level experience and technical expertise with Databricks, PySpark, and ETL pipelines, so you can integrate into their existing workflows. You also need to be capable of working autonomously on several difficult problem sets, as well as being able to communicate and collaborate across the organisation confidently.
What you'll be doing
- You will design and implement scalable streaming architectures using Databricks and Spark Structured Streaming.
- You will develop custom User Defined Functions.
- You will optimise streaming jobs processing large-scale data volumes.
- You will implement performance enhancements like partition pruning, Z-ordering, and file compaction in Databricks.
- You will architect checkpoint management strategies for streaming jobs
- You will define infrastructure as code using Terraform to enable reproducible deployments.
What they are looking for
- Technical expertise with Databricks and Spark Structured Streaming.
- Experience using PySpark to modularise complex data transformations
- High level of proficiency with Python and SQL.
- Hands-on experience using Terraform and CI/CD practices.
If you are interested in finding out more, please apply or contact me directly!
jordanf@burnssheehan.co.uk
Senior Data Engineer
90,000- 100,000 + equity
‘ Python, Databricks, Terraform
London, twice a week in office
Crypto Wallet tracking
