101 to 125 of 410 Data Pipeline Jobs in the UK

Data Engineer/Scientist London (Hybrid) £50,000-£60,000 (+bens) 83DATA are partered with a leading Internet provider seeking the skills of a Data Engineer/Scientist. The successful candidate can expect to be tasked to connect data sources to pre-existing Azure … data infrastructure. You will lead projects with a data centric approach, for this reason, a passion and aptitude for critical analysis of large and complex datasets is vital. The role will directly report to the Head of Data - a high degree of resilience is … required to provide solutions to the executive board thus enabling data driven decisions. Responsibilites: Build data engineering pipelines to ingest, transform, store, and serve high quality data for Data Scientists Writing and tuning complex data pipelines, workflows, and data more »

Data Engineer/Scientist London (Hybrid) £50,000-£60,000 (+bens) 83DATA are partnered with a leading Internet provider seeking the skills of a Data Engineer/Scientist. The successful candidate can expect to be tasked to connect data sources to pre-existing Azure … data infrastructure. You will lead projects with a data centric approach, for this reason, a passion and aptitude for critical analysis of large and complex datasets is vital. Responsibilities: Build data engineering pipelines to ingest, transform, store, and serve high quality data for Data Scientists Writing and tuning complex data pipelines, workflows, and data structures. Identify opportunities for increasing performance, minimising costs and increase platform monitoring. Experience with components such as Azure Active Directory, Management Groups, Subscriptions, vNets, Route Tables, NSG’s and Azure more »

get the chance to get matched to this role and other similar roles, click on Apply to set up your free profile. Their Data Team is made up of 50+ people, covering data engineering, data science, and data architecture specialisms. The team … works with a range of public and private sector clients, building production grade cloud solutions focussed on streaming, graph, big data, and data science use cases. Job Description Due to the nature of the client engagements, every employee needs to be able to achieve Security Clearance. … and are proud to have an organisational culture where employees can bring their authentic selves to work. Team They are constructing the top data engineering practice in the UK, and their team build bespoke, mission critical data solutions for the UK Government as well as bluechip more »

Job title: Lead Data Engineer Client: Elite Quant Fund Salary: Up to £160,000 Location: London/Hybrid My client are seeking a Lead Data Engineer to own delivery of scalable data pipelines that are critical to the entire firms infrastructure. Being a Lead … Data Engineer at a truly elite hedge fund, you will join an environment that prides themselves on working with the most cutting edge technologies and boast a world class technical team where you will work with and learn from the best. Responsibilities: As a Lead Data … based on very specifically tailored data requirements. Architect and implement data ingestion, transformation, and storage processes. Optimise data pipeline performance. Keep up with market developments in the data engineering and AI space and implement these best practices. Requirements and Skills: Bachelor more »

Senior Data Platform Engineer Up to £80,000 1 day a week in office (flexible) Xcede are delighted to be working with a well-renowned insurance company based in Hertfordshire. The company have an excellent Data Science and Machine Learning unit supported by a group of … Data Engineers who are responsible for Data Pipelines. As a bonus - Data Engineers are also able to help with ML model deployment and MLOps. Responsibilities Design and implement Data Pipelines for both structured and unstructured data. Support and improve ML infrastructure – particularly … Azure ML. Data Engineering: Guarantee data access, Data quality, ETL, Data pipelines Platform administration: Help set up accesses, infrastructure, create new storage, enforce best practices Production: Help the preparation of the data science pipelines, interact with data scientists more »

Senior Data Engineer Hybrid AND Flexible Up to £75,000 + 10% bonus + share scheme MBN Solutions are delighted to have been retained to secure a Data Engineer (Big Data Technologies) for our client who are firmly positioned in the "Tech For Good … space. This is a key hire for the client during the early stages of the growth of their Data team and it is therefore essential this person is highly experienced in all the areas of skills required. Our client are a start up/scale up and we … are ideally looking for a Senior Data Engineer who has previous experience and success in this environment previously. Your Role: Building reusable data pipelines at scale with the capability of integrating data from multiple sources automatically. Work with structured and unstructured data more »

Lead Data Engineer. What's it all about? Information, Systems and Change and Data and Digital Solutions. You will be responsible for leading the technical design and implementation of data integration projects using Microsoft Azure and Integration Services alongside other cloud components. The Lead … Data Engineer will also oversee the quality, performance, and security of the data integration solutions. How you'll make a difference By delivering data integration solutions on their data platform, you will contribute to the continuous improvement of data-driven … that improve the services the client provides for their customers. How you'll do it Lead the Engineering Team in the development of data and integration solutions leveraging the latest technologies, tools, processes, and best practice. Create and provide data products following the data more »

Senior Data Engineer - Azure Paying up to 70K Hybrid: 2 days onsite About the Role My food retail business client is looking for a senior Data Engineer to join the team. My client operate on a global scale, but their Data and Analytics function … career progression opportunities as part of our technology transformation journey. Paying up to £70K Hybrid (2 days onsite) Main Duties Design and deliver data pipelines. Build and manage Azure Data Factory data flows. Ingest data into an Azure Data Lake. … Design data models for large datasets, including creating dimensional multivariate models. Utilize non-relational data storage solutions such as MongoDB. Work with SQL Server Integration Services (SSIS) and other ETL tools. Develop solutions with SSIS, SSRS, T-SQL (On-prem/Cloud). Manage NoSQL environments more »

Job Title: Data Engineer Job Type: Full Time, Permanent Working location: London, Hybrid Role Purpose At Travelex we are developing modern data services, which will be at the heart of our relationship with our customers. Our data architecture is becoming more service-oriented and … for you to solve. Our work is fast-paced although our team remains collaborative and flexible even when delivering fast. Your role as data engineer has a wide remit: working with product owners, subject matter experts and managers across the company, to design and deliver new components of … to the traditional data engineering type! Significant hands-on experience with AWS, especially AWS services focused on data flow, pipelines, data transformation, storage and streaming. Excellent data engineering skills, for example with SQL, Python, DBT and Airflow. Good understanding of service-oriented more »

New 1 year contract opportunity for a mid-senior (outside IR35) Data Engineer to join an independent Civil Society with immediate effect. Data Engineer Responsibilities Design and implement scalable data architectures for large-scale web-scraping tools. Develop and optimize data pipelines … for ETL from online platforms, including textual and unstructured data, and REST APIs. Test, validate, and secure acquired data to ensure accuracy and quality. Work closely with the AI/ML Engineer to ensure seamless integration and alignment of data and AI … models. Data Engineer Required Experience Data engineering experience (2+ years) Cloud platform proficiency (e.g., AWS, Azure, GCP) Data pipeline development (e.g., Airflow, Apache Spark) SQL proficiency, database design Visualization tools knowledge (e.g., Tableau, PowerBI, Looker) Data Engineer Application Process This is more »

Job Title: Data Engineer Location: Edinburgh, Scotland (Hybrid) - All employees must be eligible to work without sponsorship Salary: £50,000 - £70,000 The Role: Design and develop ETL processes Design, build and maintain end-to-end data pipelines Identify, root cause analysis, and fix incidents related … to data management Manage company data, monitor accuracy and completeness, and advise ways that data quality can be improved Routinely provide status updates to colleagues and stakeholder groups Data Engineer Requirements: Hands-on experience with SQL database design Experience writing automated tests … for data pipelines Technical expertise with data models, data mining, and segmentation techniques Knowledge of programming languages (e.g. Python, R) Data Engineer Desirable's: Proven experience of Azure Services Knowledge of Data bricks and Jupyter notebooks Experience in data more »

our award-winning team at Sparta Global. About this role You'll become versatile in a wide array of tools across topics covering data focused coding, data visualisation, Cloud Services, Big Data and Machine learning. You will be equipped to work within a variety … of roles including but not limited to; Data Engineer, Data Analyst, Data Scientist, Information Engineer. You'll be designing, building, maintaining, and troubleshooting the data pipelines that enable organizations to store, process, and analyse their data, and ensuring the data is reliable through testing and debugging. Looking for efficiencies and optimising the data pipelines for scalability and performance will be a focus. You'll be handling and working with large sets of structured and unstructured data and will be responsible for ensuring that the more »

our award-winning team at Sparta Global. About this role You'll become versatile in a wide array of tools across topics covering data focused coding, data visualisation, Cloud Services, Big Data and Machine learning. You will be equipped to work within a variety … of roles including but not limited to; Data Engineer, Data Analyst, Data Scientist, Information Engineer. You'll be designing, building, maintaining, and troubleshooting the data pipelines that enable organizations to store, process, and analyse their data, and ensuring the data is reliable through testing and debugging. Looking for efficiencies and optimising the data pipelines for scalability and performance will be a focus. You'll be handling and working with large sets of structured and unstructured data and will be responsible for ensuring that the more »

My leading Energy and Commodities conglomerate are currently seeking a skilled individual with experience in Python-based data pipeline, data lake house development and an appreciation for Object orientated languages such as C# or Java, using leading orchestration tools. In this role, you'll lead … the development of robust, fully tested data pipelines in Python using cutting-edge platforms like Dagster/Airflow and apply your expertise in real-time data streaming solutions using Kafka. You'll play a key role in expanding their on prem data lake infrastructure … to help enhance support for new lines of businesses they have including Systematic trading, Trading and Shipping commodities, ensuring strong data lineage. The successful candidate will have: A strong background in data engineering using Python Strong understanding of Test Driven Development Experience building Star Schemer Databases more »

Palantir Data Engineer £85,000 - £95,000 UK Wide Must be able to be eligible for SC clearance! Travel to client site when required! A leading consultancy-based organisation is currently looking to grow their cloud platforms team and wanting to add a Palantir Data Engineer … to their practice. The successful Palantir data engineer will be passionate about cloud technology being involved in using Palantir data focussed reference architecture to assist in designing and building data engineering solutions supporting planning and implementations. Skills and Experience Experience of developing enterprise grade … and deliver the data solutions in Palantir Strong experience with Palantir Data Engineering features such as Code Repo, Code Workbook, Pipeline Builder, migration techniques, Data Connection and Security setup Developing data integration pipelines, transformations, pipeline scheduling, Ontology, and applications in more »

Data Engineer Location: London – Hybrid Employment: Full Time All applicants must hold a DV (Developed Vetting) clearance to apply. The Opportunity: For the Ministry of Defence (MOD), maintaining National Security is a 24-hour-a-day, 365-day-a-year mission. For this reason, it partners with this … Big 4 Firm to support its multidisciplinary AI & Cloud Data Engineering Capabilities. The Role: As an AI & Cloud Data Engineering practitioner, you’ll build large-scale data pipelines and infrastructure, utilising data integration tools to optimise complex data solutions. Collaborating … with high-performing Data Science and ML Engineering teams to drive innovation and insight through AI and Data across the Defence & Security sector. Required Skills/Experience: British Passport Holder with active DV (Developed Vetting Clearance) End-to-end development with data pipelines, ETL more »

Azure Data Engineer - Contract - Inside IR35 This is an Azure Data Engineer position for a financial services end client helping them on delivering and implementing new data pipelines. You will need prior financial services experience as well as having strong Azure Data Factory and Databricks experience. Work Involved: Implementing and delivering data pipelines Providing mentorship to junior members of the team Working with the Platform team defining capabilities for the lake … framework Developing business conceptual data models Producing new frameworks using design patterns and data ingestion Requirements: Strong Data Pipeline experience Databricks Azure Data Factory Spark Azure cloud Financial Services If you're interested get in touch ASAP more »

Runcorn, WA7 3DJ Salary: £46,000 - £49,000 and extensive benefits package Hours: 37.5 hours per week Are you a seasoned expert in data engineering looking to make a significant impact in the healthcare industry? Phoenix Medical Supplies is seeking a Senior DataOps Engineer to lead our data … services across the region. As a Senior DataOps Engineer, you will be instrumental in architecting and implementing solutions across the entire data pipeline, from ingestion to visualisation. The role will require you to showcase your skills in data ingestion, storage, modelling, transfer, hosting, visualisation, analysis … and consumption, all while mentoring and training team colleagues. Key Responsibilities: Design and develop robust data pipelines for ingestion, storage, processing, and interface creation. Produce both machine consumable interfaces and human consumable data products, including standard and ad hoc reports. Maintain and optimise data more »

Vertex Search is representing a fast-growing financial services firm that is now looking to make a Senior Data Engineering hire, the first role of its kind into the business with huge scope for impact and influence. The nascent data engineering team will collaborate with internal … teams around the business with a focus on data engineering, pipelines, tooling and services. Key Responsibilities: Data Engineering: Design and develop data services across various business functions Data Pipelines: Design and develop data pipelines for extracting, transforming, and loading data into appropriate storage systems. Data Tools and Services: Develop data tools and services to discover, understand, and utilize data effectively. Best Practice: Advise on and contribute to the development of data governance and management practices. Collaborate across teams to understand more »

consultancy client. The successful candidate will be responsible for designing, developing, and maintaining KDB databases, as well as creating and optimizing real-time data analytics solutions. You will work closely with the technology team to enhance our data management systems, ensure data accuracy and … integrity, and help us make data-driven decisions. Key Responsibilities: Design and develop KDB databases and data processing systems to support real-time data analysis and reporting. Optimize database performance, troubleshoot issues, and implement improvements as needed. Collaborate with data analysts and … other teams to understand data requirements and develop efficient data retrieval processes. Write and maintain KDB Q scripts, ensuring code is well-documented and follows best practices. Monitor data quality and accuracy, implementing data validation and cleansing processes. Develop and maintain data more »

DATA ENGINEER LONDON HYBRID - 3 DAYS IN THE OFFICE UP TO £60K This role does not have the ability to sponsor THE COMPANY A leading provider in accommodation focused on taking advantage of their raw data sources to provide a nurturing environment facilitating personal growth and … vibrant communities THE ROLE AND RESPONSIBILITIES You will be responsible for bringing together all data sources into an Azure Cloud Data Warehouse to be leveraged for BI reporting and analytics In this role you will: Designing, implementing, and maintaining data pipelines across multiple data pipelines across multiple cloud SQL environments to ensure data integrity, security, and performance are maintained to a high standard Work with the BI team to design and build best-in-class data warehouse components Designing, building, and maintaining quality ETLs and ELTs to make more »

two days) at our Leeds, Edinburgh, or Bristol locations. About this opportunity Join IP&I as we embark on a transformative journey in data engineering. We are seeking a dynamic and experienced Senior Manager within Advanced Analytics to lead our team in architecting and implementing cutting-edge data solutions. If you are passionate about driving innovation and possess a proven track record in cloud and legacy data engineering leadership, we want to hear from you! You will be at the forefront of our data engineering initiatives. You'll lead a talented team … in designing, developing, and optimizing scalable data solutions. Your role will involve providing technical leadership, driving innovation, and ensuring the successful delivery of complex data projects. Here's where you'll make the difference: Lead and mentor a team of data engineers Architect and more »

Job Role: Azure Data Engineer Location: Central London - Hybrid/3 Days in Office Salary: £60,000 - £70,000 + 12.5% Bonus Chapman Tate is collaborating with a leading property management company that is revolutionizing the industry through innovative data-driven solutions. As they continue to … expand their operations, they are seeking a talented Azure Data Engineer with strong SQL expertise to join their dynamic team. Are you passionate about leveraging data to drive business insights and enhance operational efficiency? Do you have experience working with Azure data technologies and … so, this is an exciting opportunity to contribute to the success of a forward-thinking property management company. Responsibilities: Design, develop, and maintain data pipelines and ETL processes using Azure Data Factory. Optimize data storage and retrieval processes to ensure scalability, reliability, and performance. more »

I'm looking for a Data Engineer with skills in Databricks, Azure Data Factory and SQL Server, for a Fixed Term Contract role until August 2025. It's with an international organisation headquartered in London, and this role will require you to be in their offices … near Victoria station 3 days per week. You will be joining a small cross-functional data team, where you will focus on the development of their newly-created Azure Data Platform - this will involve designing and developing ETL pipelines using Azure Data Factory, and … utilising Databricks to support the ongoing development of the cutting-edge Data Platform. You'll seek opportunities for improvement where possible, and will collaborate with both business and technology stakeholders to ensure that the solution is fit for purpose. You will also design and develop data more »

Job Title: Data Operations Analyst Company: Global Asset Manager Location: City of London, 3 days per week in office Salary: £85,000 per annum + Benefits About the Company: Join a prestigious global asset manager with over $200 billion in assets under management. Our firm is renowned for … excellence. Based in the heart of London's financial district, you'll be part of a dynamic team shaping the future of financial data operations. Role Overview: As a Data Operations Analyst, you will play a pivotal role in implementing a cutting-edge global platform aimed … at centralizing data management processes. Leveraging your expertise in R, Python, SQL, and Enterprise Data Management (EDM), you'll contribute to streamlining data workflows and optimizing data quality across the organization. Key Responsibilities: Collaborate with cross-functional teams to understand data more »