Data & Analytics Engineer

Overview

In this role you will own the design, build and optimisation of end-to-end data pipelines and governed datasets on the Databricks Lakehouse. You collaborate with business stakeholders and SI partners to translate requirements into scalable data solutions that drive value. You’ll improve data quality, lineage and analytics readiness across bronze to gold layers, enabling fast, secure BI with modern tools. This is a transformational data capability role at a company investing in data-driven decision making.

Pay / Benefits

• competitive salary
• 33 days’ holiday
• company pension
• onsite gym
• in-house doctor and dentist
• ULEV car scheme

Responsibilities

• Design and implement batch and streaming data pipelines across bronze, silver and gold layers
• Apply CDC patterns, incremental loading, partition tuning and data layout optimisations (Z Order, OPTIMIZE/VACUUM)
• Build tests, data quality checks and observability (alerts, metrics, runbooks)
• Orchestrate workflows using Databricks Jobs
• Collaborate on CI/CD with Platform (Repos, Git, Terraform) and deployment pipelines
• Model curated gold datasets and semantic layers for BI tools (Tableau/PowerBI)
• Tune SQL Warehouses for performance, concurrency and cost; maintain versioned metrics
• Document tables with catalog cards including lineage and data contracts; enable discoverability
• Implement Unity Catalog based access controls and uphold governance standards
• Participate in access reviews, audit remediation and privacy/compliance processes

Key requirements

• experience designing and building data pipelines on Databricks Lakehouse or similar
• strong SQL and PySpark skills
• experience with Delta Live Tables, Auto Loader and Spark SQL
• data modelling for BI and semantic layers
• data governance, Unity Catalog and access control experience
• ability to work with cross-functional stakeholders and translate needs into data products
• excellent documentation and runbook creation
• experience with CI/CD tooling (Git, Terraform) and DataOps practices
• attention to detail
• customer-focused mindset
• strong stakeholder communication
• PySpark
• Spark SQL
• Delta Live Tables (DLT)

Report this job