Data Engineer

**Data Engineer** (Contract) Duration: 6 Months (Possibility for extension) Location: London/Hybrid (3 days per week on site) Rate: A highly competitive Umbrella Day Rate is available for suitable candidates Role Overview The Data Engineer will be responsible for designing, building, and maintaining scalable data pipelines and architectures.

The ideal candidate will possess a strong background in developing and maintaining shared component and framework libraries while collaborating closely with Technology, Product, and Business stakeholders to ensure the availability, quality, and reliability of data.

Key Responsibilities

Design, develop, and optimise robust data pipelines and ETL processes. Engage in hands-on development with Scala, Java, or Python through system integration projects or as a data scientist. Utilise technologies such as Spark, Elastic, Restful APIs, JSON, and Tomcat.

Build reusable libraries and shared capabilities to expedite time to market for Data Engineering delivery teams. Continuously improve platform capabilities and maintain overall platform health to address evolving needs. Collaborate with cross-functional teams to understand data requirements and deliver effective solutions.

Ensure data quality, integrity, and security across all data systems. Monitor, troubleshoot, and enhance data workflows and performance. Provide automation capabilities to assist delivery teams with data validation, testing, and deployments.

Stay abreast of emerging data engineering technologies and trends. Gain experience with Azure cloud services, including Azure Data Factory, Gen 2, and Azure Databricks. Work with Elasticsearch and maintain advanced SQL knowledge, preferably on MS SQL Server and Azure cloud databases.

Navigate the DevOps lifecycle using tools such as Git, ADO, and Jenkins. Utilise Jira/Confluence and apply Agile methodologies in projects. Maintain high customer satisfaction through the timely delivery of high-quality solutions and effective problem resolution.

Communicate proactively and effectively with business users, support teams, vendors, and stakeholders. Utilise strong analytical skills for diagnosing and troubleshooting potential system issues, demonstrating a quick turnaround in solutions.

Skills & Experience

Knowledge of and or exposure to Quantexa would be preferential Highly proficient in Scala, Python, and SQL. Well-versed in software engineering and systems design p...

Report this job