Data Engineer
Derby. Full time.
An exciting opportunity has arisen for a Data Engineer to join the team. As a Data Engineer, you will work on implementing complex data projects with a focus on collecting, parsing, managing, analysing and visualising large sets of data to turn information into value using multiple platforms. You will work with business analysts and data scientists to understand customer business problems and needs, secure the data supply chain, implement analysis solutions and visualise outcomes that support improved decision-making for customers. You will understand how to apply technologies to solve big data problems and develop innovative big data solutions.
Responsibilities:
•Working with colleagues to understand and implement requirements.
•Securing the data supply chain, understanding how data is ingested from different sources and combined or transformed into a single data set.
•Analysing, cleansing, joining and transforming data.
•Implementing designed and specified solutions into the chosen platform.
•Working with colleagues to ensure the on-premises and cloud infrastructure available is capable of meeting solution requirements.
•Planning, designing and conducting tests of implementations, correcting errors and re-testing to achieve acceptable results.
•Managing data, including security, archiving, structure and storage.
Requirements:
•Degree-level education in Mathematics, a Scientific, Computing or Engineering discipline, or equivalent experience.
•5+ years of experience across various levels of Data Engineering roles, including 3 years in a technical lead role managing end-to-end solutions.
•Hands-on experience with Azure Databricks using PySpark, with the ability to undertake cluster capacity planning and workload optimisation.
•Holds Security Check (SC) clearance OR eligible for Security Check (SC) clearance level and willing to apply for one through RR and go through the vetting process to successfully receive the SC clearance.
•Experience designing solutions using databases and data storage technologies, including Relational Database Management Systems (RDBMS), specifically MS SQL Server.
•Experience building and optimising Big Data pipelines, architectures and data sets using Microsoft Azure data management and processing components across IaaS, PaaS and SaaS implementation models, including custom components.
•Experience using Azure Data Factory.
•Proficiency in Python scripting, including experience using Python modules for data munging.
•Up-to-date knowledge of data processing technologies and platforms, such as Spark (Databricks).
•Experience in data modelling to optimise solution performance.
•Experience using Azure DevOps, with exposure to Continuous Integration and Continuous Delivery/Deployment (CI/CD).
•Experience of defining data contract and implementing it through relevant data management practices using processes and tools.
•Good understanding of infrastructure components and their application within different types of data solutions.
•Experience designing solutions deployed on Microsoft and Linux operating systems.
•Experience working in an agile environment within a self-organising team.
Additional information:
•Grade: SNBG – 7-10
•Closing Date: 17th November 2026
•People Leader: Sadashiva Bolanthur
•For further information, please contact: Jaspreet Panesar
