Snr Big Data/Google Cloud Engineer (Python) – Remote/UK
£90k basic + bonus + shares
My client is building one of the largest independent technology and business services firms in the world are looking for a Snr Big Data and Google Cloud Engineer to join their growing remote team.
We need a Big Data Engineer to join our innovative UK Advanced Analytics team tasked with creating and managing Big Data infrastructure and tools, typically designing and building big data pipelines and platforms.
Your future duties and responsibilities
We have an exciting opening for a Senior Big Data Consultant to work on the collecting, storing, processing, and analysing of huge sets of data. The primary focus will be on choosing optimal solutions to use for these purposes, then maintaining, implementing, and monitoring them. You will also be responsible for integrating them with the architecture used across the business.
As a Senior Consultant and Big Data expert you will build ETL/ELT best practices in a big data environment, featuring big data pipelines, real time streaming using Kafka and leveraging high levels of proficiency across distributed computing principles.
You will be able to select and integrate any Big Data tools and frameworks required to provide requested capabilities, implementing ETL/ELT process, working closely with Cloud big data architecture (GCP) and be an SME in relation to knowledge of big data tools/platforms, data management tools and necessary infrastructure changes.
Python and/or Spark for data pipelines
Appreciation of ETL/ELT, Lambda architecture, data security & GDPR controls
Strong data engineering and technical architecture skills on GCP Cloud Platform is essential. Candidate must have hands on designing and delivering GCP data applications covering Google Cloud composer, Bigquery, Dataproc, PubSub. Other cloud platforms are desirable but not essential.
Streaming frameworks like Kafka
Understanding of Data Warehousing, Data Modelling techniques
Appreciation of database technologies such as noSQL, graph, in-memory databases
Attitude to be excited with new technology experimentation in a lab style setup