Databricks Data Engineer

Senior Databricks Data Engineer 3-Month Contract | Outside IR35 | Fully Remote | £500 per day The Opportunity We're supporting a business that is investing heavily in its data capabilities and is looking for an experienced Senior Databricks Data Engineer to take ownership of a growing cloud-based data platform.

This assignment will focus on building a robust and scalable data ecosystem that supports reporting, analytics and strategic decision-making across a range of business functions. You'll work closely with both technical and non-technical stakeholders to transform fragmented data sources into trusted, well-governed datasets that drive real business value.

This is a hands-on engineering role suited to someone who is comfortable leading technical decisions, establishing best practices and delivering production-grade Databricks solutions. What You'll Be Doing Building and enhancing data pipelines using Databricks, PySpark and SQL.

Designing and implementing scalable Lakehouse architectures using Delta Lake. Integrating data from a variety of platforms through APIs, file-based ingestion and third-party systems. Developing and maintaining Bronze, Silver and Gold data layers to support analytics and reporting requirements.

Defining data governance standards, lineage, security controls and access management. Delivering robust ETL/ELT solutions across cloud environments. Establishing monitoring, testing and data quality frameworks to improve reliability and trust in data.

Implementing CI/CD practices and automated deployment processes. Optimising Databricks workloads for performance, scalability and cost efficiency. Working closely with analytics, reporting and business teams to turn requirements into technical solutions.

Supporting and mentoring other members of the wider data function. Required Experience Strong commercial experience delivering enterprise-scale data engineering solutions on Databricks. Advanced Python, PySpark and SQL skills.

Proven experience designing Lakehouse architectures and working with Delta Lake. Strong understanding of Databricks Workflows and production pipeline orchestration. Experience implementing data governance controls using Unity Catalog or similar tooling.

Experience building cloud-native data solutions within Azure, AWS or GCP environments. Strong understanding of data modelling and analytical data structures. Experience with source control, automated testing and deployment pipelines.

Knowledge of security, governance and compliance requirements when handling sensitive data. Ability to communicate effectively w...

Report this job