Lead Data Engineer

Lead Data Engineer | 6 Month Contract | (Inside IR35) | Hybrid, 2 days pw Edinburgh/Glasgow | Starting ASAP Day Rate: DOE

About the role

Harvey Nash's Pub Sec client is seeking an experienced AWS Data Engineer to design, develop, and support scalable data warehouse and cloud-based data solutions. The successful candidate will work within an Agile Scrum team, delivering end-to-end data engineering solutions, ETL pipelines, and analytics platforms that support business intelligence and digital applications.

You will

Design, build, and maintain cloud-based data warehouse and data lake solutions on AWS. Develop and support ETL/ELT processes for data ingestion, transformation, and integration. Build and optimise data pipelines using Python, SQL, Spark, and AWS services.

Develop serverless and event-driven solutions using AWS Lambda, Step Functions, EventBridge, SQS, and SNS. Support and enhance existing data platforms, applications, and integrations. Work closely with business and technical stakeholders to deliver solutions in 2-week Agile sprints.

Create and maintain dashboards, reporting, and monitoring solutions using QuickSight, Power BI, Grafana, and ELK. Implement data quality, security, governance, and operational monitoring best practices. Essential Criteria Strong experience with AWS data and analytics services including: Redshift Glue S3 Athena Lambda Step Functions RDS/Aurora DynamoDB DMS EMR CloudWatch IAM Strong programming experience in Python and SQL.

Experience with Spark (PySpark preferred) and large-scale data processing. Experience developing ETL/ELT pipelines and data warehouse solutions. Knowledge of API integration and microservices architectures. Experience working in Agile/Scrum delivery environments.

This role has been deemed Inside IR35 by the client. Applicants must hold, or be happy to apply for, a valid Basic Disclosure Scotland. Please click the link to apply.

Report this job