Senior Data Engineer - Databricks

Job Role

We are seeking a Senior Databricks Data Engineer to lead delivery of data engineering workstreams within large-scale data platform and migration programmes. This role combines hands-on technical expertise with consulting leadership, ensuring the successful design, delivery and operation of modern data platforms while working closely with client stakeholders. You will play a key role in shaping architecture decisions, mentoring engineers and driving delivery across complex engagements.

Key Responsibilities

• Lead design and implementation of enterprise-scale data pipelines and lakehouse architectures
• Provide technical leadership across Databricks, Spark and cloud data platforms
• Drive delivery of data engineering workstreams within Agile programmes
• Engage with client stakeholders to translate business objectives into technical solutions
• Mentor engineers and establish engineering best practices
• Lead architecture and design discussions with architects and platform teams
• Oversee performance optimisation and platform reliability
• Support data migration strategy and execution
• Ensure governance, security and compliance standards are met
• Produce architecture documentation, design artefacts and delivery roadmaps

Key Requirements

• 7+ years experience in data engineering or data platform development
• Strong experience with Databricks, Spark and distributed data processing
• Experience with Unity Catalog, Delta Lake and streaming architectures
• Advanced Python and Microsoft SQL Server development
• Proven experience leading engineering teams or workstreams
• Strong stakeholder engagement and consulting skills
• Working knowledge of Hadoop
• Experience with test-driven development and CI/CD
• REST APIs and Kafka experience

Nice to Have

• Experience leading data platform migrations
• Experience managing delivery in large transformation programmes
• Experience delivering projects in regulated environments

Report this job