Data Engineer (Databricks)

Data Engineer (Databricks) Leeds (Hybrid) 6 Month Contract £(Apply online only)/day (Inside IR35) Hybrid Working Model - primarily working from home with travel to the Leeds office 1-2 days per month. Start ASAP in October 2026.

Data Engineer needed with strong hands‐on Databricks, PySpark, AWS, SQL + Python experience. A chance to work with a leading global IT transformation on a large‐scale Government project: Hands‐on Data Engineer to run + maintain production data pipelines on Databricks and AWS.

Keeping scheduled data processing reliable, resolve incidents, fixes + improving data services performance. The role combines L2/L3 production support and Data Engineering tasks. Databricks experience covering notebooks, scheduled jobs, cluster configuration, driver / executor logs, Spark UI + performance troubleshooting.

Supporting + maintaining production data pipelines, including incident investigation, safe recovery, root cause analysis + permanent remediation. Experience of Spark SQL, Delta Lake, Parquet + Hive metastore tables, including schemas, partitions and the relationship between table metadata and underlying files.

Hands‐on AWS experience, particularly S3, IAM and CloudWatch, with an understanding of role‐based access, KMS encryption and diagnosing data access failures. Owning incidents from investigation through recovery and closure.

Diagnosing issues in Python, PySpark, SQL, Databricks jobs and AWS integrations + providing clear progress updates and timely escalation. Working knowledge of Git, code reviews, CI/CD and controlled production deployments, including testing, rollback and release validation.

Advantageous: AWS Glue, Lambda, Step Functions, Linux, Shell Scripts, Terraform, Gitlab, Jenkins

Report this job