We are hiring for cloud data engineer for the UK location ( not offering sponsorship)
Job description:
‘ Building new data pipelines and optimizing data flows using the Azure cloud stack.
‘ Building data products from scratch.
‘ Support Business Analysts and Data Architects with discovery and best practices.
‘ Self-directed and comfortable supporting the data needs of multiple teams, systems and products.
What’s required ?
‘ Advanced Azure knowledge and experience working and migrating data products from on-prem to Azure.
‘ Experience building and optimizing ‘big data' data pipelines, architectures and data sets using Py-spark.
‘ Experience building real-time data pipelines using Event-hub, storage queues and Azure stream analysis.
‘ Strong analytic skills related to working with unstructured datasets.
‘ Build processes supporting data transformation, data structures, metadata, dependency and workload management.
‘ Data Ingestion and Storage including Azure Data Factory, Azure Databricks, Azure Data Lake, Kafka and Spark Streaming, Azure EventHub/IoT Hub, and Azure Stream Analytics
‘ Experience with Bigdata Tools: Hadoop Spark, Kafka
‘ Experience with Stream processing systems: Spark-streaming, Kafka
‘ Experience with Object-oriented/ object function scripting languages: Python preferred.
