1 to 25 of 997 Data Pipeline Jobs in the UK

Data Strategy and Management Consultant (Consultant to Manager, Financial Services) London, United Kingdom req2552 Posted on: 19/08/2025 - Application Deadline: 09/09/2025 Data Strategy and Management Consultants (Consultant to Manager) About Baringa: We set out to build the world's most trusted consulting firm - creating lasting impact for clients and … from a wind farm in Wyoming to a boardroom in Berlin. Find us wherever there's a challenge to be tackled and an impact to be made. Our Data, AI, Solutions & Engineering practice are looking for an experienced Data Strategy and Management consultants to join the team. Our Data, AI, Solutions & Engineering team loves … helping our clients solve their complex problems with data, by developing practical and high-impact data strategies; designing and implementing data operating models; creating actionable insights; embedding machine learning solutions; or building data platforms. We bring the right team and skills on the table and can offer deep expertise across Data More ❯

About Us: ATREIDES is a leader in advanced geospatial technologies, big data solutions, and data analytics. We specialize in providing cutting-edge software platforms that enable our clients to process, analyze, and visualize large-scale geospatial and sensor data for critical decision-making. Our mission is to deliver innovative, scalable, and high-performance platforms … that help organizations unlock the full potential of their data for various industries, including defense, national security, environmental monitoring, and urban planning. We are seeking a highly skilled Senior Data Engineer with a strong focus on big data, data analytics, and geospatial data intelligence to join our engineering team. In … and technologies (AWS, Azure, GCP) for big data processing and platform deployment. Strong knowledge of data warehousing, data lakes, and data pipeline design for large-scale data integration and storage. Familiarity with machine learning and AI techniques for data analytics (e.g., classification, regression, clustering, anomaly detection). More ❯

that solves real-world problems for large blue-chip companies and governments worldwide. We're pioneers of meaningful AI : our solutions go far beyond chatbots. We are using data and AI to solve the world's biggest issues in telecommunications, sustainable water management, energy, healthcare, climate change, smart cities, and other areas that have a real impact on … re underpinned by over 300 engineers and scientists working across Africa, Europe, the UK and the US. ABOUT THE ROLE Sand Technologies focuses on cutting-edge cloud-based data projects, leveraging tools such as Databricks, DBT, Docker, Python, SQL, and PySpark to name a few. We work across a variety of data architectures such as Data … data engineering projects. You will work closely with cross-functional teams and contribute to the strategic direction of our data initiatives. RESPONSIBILITIES Data Pipeline Development: Lead the design, implement, and maintain scalable data pipelines for ingesting, processing, and transforming large volumes of data from various sources using tools such More ❯

that solves real-world problems for large blue-chip companies and governments worldwide. We're pioneers of meaningful AI : our solutions go far beyond chatbots. We are using data and AI to solve the world's biggest issues in telecommunications, sustainable water management, energy, healthcare, climate change, smart cities, and other areas that have a real impact on … re underpinned by over 300 engineers and scientists working across Africa, Europe, the UK and the US. ABOUT THE ROLE Sand Technologies focuses on cutting-edge cloud-based data projects, leveraging tools such as Databricks, DBT, Docker, Python, SQL, and PySpark to name a few. We work across a variety of data architectures such as Data … modelling, ETL (Extract, Transform, Load) processes, and big data technologies it becomes possible to develop robust and reliable data solutions. RESPONSIBILITIES Data Pipeline Development: Design, implement, and maintain scalable data pipelines for ingesting, processing, and transforming large volumes of data from various sources using tools such as databricks More ❯

turning down headhunters for years. Whatever your priorities - work/life balance, career progression, sustainability, volunteering - you'll find like-minded people driving change at Howden. Job Title: Data Architect Location: London/Hybrid Job Type: Full-time Job Summary Are you passionate about designing scalable, high-performance data architectures on Azure and Databricks ? Do you … thrive on solving complex data challenges, integrating modern data platforms, and defining best practices for enterprise data management? We are looking for a Data Architect to lead the design and implementation of cutting-edge data solutions in a cloud-native environment. You will play a key role in shaping data strategy, designing high-level and low-level architectures , and ensuring the seamless integration of data management tools. If you enjoy working with big data technologies, governance frameworks, and modern analytics platforms , this role is for you! Key Responsibilities Design and architect scalable, secure, and cost-effective Azure Databricks solutions for business and analytical workloads. More ❯

consistency, and proactively collaborating with upstream and downstream teams to enable seamless data flow across the organization. In this role, you will not only troubleshoot and resolve pipeline issues but also contribute to enhancing data architecture, implementing best practices in data governance and security, and ensuring the scalability and performance of data … stakeholders This position requires client presence between 25%-50% of the time per month at the client's office, which is located in London. Key Responsibilities: Data Pipeline Development & Maintenance Build, maintain, and optimize scalable ETL/ELT pipelines using tools such as Dagster, or similar. Ensure high data availability, reliability, and consistency through rigorous … data validation and monitoring practices. Collaborate with cross-functional teams to align data pipeline requirements with business objectives and technical feasibility. Automate data workflows to improve operational efficiency and reduce manual intervention. Data Integrity & Monitoring Perform regular data consistency checks, identifying and resolvinganomalies or discrepancies. Implement robust monitoring frameworks More ❯

best platform, people, and partners together to create limitless opportunities for growth. For more information, visit About Choreograph: A Leading WPP Media Brand Choreograph is WPP's global data products and technology company. We're on a mission to transform marketing by building the fastest, most connected data platform that bridges marketing strategy to scaled activation. … We work with agencies and clients to transform the value of data by bringing together technology, data and analytics capabilities. We deliver this through the Open Media Studio, an AI-enabled media and data platform for the next era of advertising. We're endlessly curious. Our team of thinkers, builders, creators and problem solvers … are over 1,000 strong, across 20 markets around the world. Role Summary and Impact We are seeking a meticulous and experienced Data Architect to play a crucial role in designing, documenting, and ensuring the quality of our next-generation media data lake on the Microsoft Azure platform. This role will focus on developing and upholding More ❯

Senior Data Engineer - (Azure/Databricks) page is loaded Senior Data Engineer - (Azure/Databricks) Apply locations London - Scalpel time type Full time posted on Posted 7 Days Ago job requisition id REQ05851 This is your opportunity to join AXIS Capital - a trusted global provider of specialty lines insurance and reinsurance. We stand apart for our … civil union status, family or parental status, or any other characteristic protected by law. Accommodation is available upon request for candidates taking part in the selection process. Senior Data Engineer (Azure/Databricks Job Family Grouping: Chief Underwriting Officer Job Family: Data & Analytics Location: London How does this role contribute to our collective success? The Data … services. Implement end-to-end data pipelines, ensuring data quality, data integrity and data security. Troubleshoot and resolve data pipeline issues while ensuring data integrity and quality. Implement and enforce data security best practices, including role-based access control (RBAC), encryption, and compliance with industry More ❯

Job Responsibilities: Translate Data Pipeline requirements into data pipeline design, guide and direct the design by working closely with a range of stakeholders including the rest of the architecture team, external developers, data consumers, data providers, and internal/external business users. Contribute to use case development, e.g. workshop to … gather and validate business requirements. Model and design the ETL pipeline data structure, storage, integration, integrity check and reconciliation. Standardize all exception control and ensure good traceability during trouble shooting. Document and write technical specifications for the function and non-functional requirements of the solution. Design data model/platform allowing managed growth of the … data model to minimize risk and cost of change for a large-scale data platform. Analyze new data sources with structured Data Quality Evaluation approach and work with stakeholders to understand the impact of integrating new data into existing pipelines and models. Bridge the gap between business requirements and ETL More ❯

Position : Senior Data Engineer Salary: £55,000 - £65,000 Location : Hybrid - Bristol, Manchester, or London Security Requirement : SC Clearance to start and be willing/able to obtain DV Clearance We are seeking Data Engineers, with a keen interest/experience in AI/ML and strong proficiency in Python Scripting. To join our Consultancy client … working on meaningful projects in the Defence and Security sector. Required Experience: End-to-End Data Development : Strong experience with data pipelines, ETL processes, and workflow orchestration, demonstrating best practices across tech stacks. Strong Python Skills Diverse Data Handling : Familiarity with batch, streaming, real-time, and unstructured data sources. Architectural Design & Systems … AI Integration : Apply data engineering tools, integration frameworks, and query engines to create high-quality, standardised data for AI applications and reporting. Data Pipeline Development : Design and implement robust data pipelines and stores in collaboration with other engineers and developers. Innovative Problem Solving : Bring fresh approaches to challenging data More ❯

Meet the team behind this journey DeepL has millions of passionate users around the globe, both consumer and enterprise. These users produce a massive quantity of data. The Data Domain teams increase operational efficiency and drive innovation at DeepL by facilitating the flow of this data from producers to consumers, in order to maximize business impact. … Through the creation of stable and user-friendly analytical data models, we make customer data models and user behavioral data accessible and reusable for data professionals throughout the company. In this role, your main responsibilities and deliverables will be to build and maintain ETL pipelines, create foundational data models in … You may work with revenue, customer, or product usage data and will be expected to gain an analytical understanding of these datasets. Your responsibilities Data pipeline development: design, implement, and maintain scalable data pipelines and architecture in both our on-prem and cloud environments. Build the next generation of DeepL's data More ❯

Content Link Select how often (in days) to receive an alert: Create Alert Competitive salary UK/Glasgow: hybrid working model (2-3 days on site) At NTT DATA, we know that with the right people on board, anything is possible. The quality, integrity, and commitment of our employees are key factors in our company's growth, market … within a dynamically evolving technical stack. Collaborate seamlessly across diverse technical stacks, including Databricks, Snowflake, etc. Developing various components in Python as part of a unified data pipeline framework. Contributing towards the establishment of best practices for the optimal and efficient usage of data across various on-prem and cloud platforms. Assisting with the testing … and deployment of our data pipeline framework utilizing standard testing frameworks and CI/CD tooling. Monitoring the performance of queries and data loads and perform tuning as necessary. Providing assistance and guidance during QA & UAT phases to quickly confirm the validity of potential issues and to determine the root cause and best resolution of More ❯

are a brand embarking on a very exciting plan and we are seeking new members of the team who want to contribute to this. THE ROLE As a Data Architect (contract) for Project Nova, you will be instrumental in shaping AllSaints' journey to become a truly data-driven, AI-powered organisation. You will lead the design … and implementation of our enterprise data architecture. This role is central to the Data & Integration workstream of Project Nova, building on our existing data warehouse and BI capabilities to accelerate our data transformation and enable the use of greater AI-driven insight and actions. You will ensure our data assets … techniques and tools Strong hands-on experience with cloud data platforms, specifically Google Cloud Platform - BigQuery Experience with leading ETL/ELT tools and data pipeline orchestration (e.g., Dataflow, Apache Airflow, Talend, Informatica) Advanced SQL skills and deep knowledge of various database technologies (relational, columnar, NoSQL) Practical experience in establishing data governance frameworks More ❯

Radius is seeking a highly talented Data Architect and Governance lead for my client going through a digital transformation. Team Leadership Data Strategy Define and lead enterprise wide data strategy Design and maintain a scalable, secure data architecture Deep understanding of Data Governance, Modern data platforms: Cloud Data quality Management, metadata, lineage and data modelling Experience with Data integration, master data management (MDM) and designing scalable data pipelines Data Privacy, regulatory compliance, and security best practises BI Tools Power BI, Tableau, Looker proficient in data modelling, SQL, and cloud platforms - Azure, AWS, GCP and … their Data services Key Responsibilities Define and implement data architecture strategies in line with the Customer Journey programme Guide the data-driven decision-making process, ensuring that data is effectively utilised to inform business strategies and initiatives. Develop data plan to address data quality issues in line with More ❯

Radius is seeking a highly talented Data Architect and Governance lead for my client going through a digital transformation. Team Leadership Data Strategy Define and lead enterprise wide data strategy Design and maintain a scalable, secure data architecture Deep understanding of Data Governance, Modern data platforms: Cloud Data quality Management, metadata, lineage and data modelling Experience with Data integration, master data management (MDM) and designing scalable data pipelines Data Privacy, regulatory compliance, and security best practises BI Tools Power BI, Tableau, Looker proficient in data modelling, SQL, and cloud platforms - Azure, AWS, GCP and … their Data services Key Responsibilities Define and implement data architecture strategies in line with the Customer Journey programme Guide the data-driven decision-making process, ensuring that data is effectively utilised to inform business strategies and initiatives. Develop data plan to address data quality issues in line with More ❯

Radius is seeking a highly talented Data Architect and Governance lead for my client going through a digital transformation. Team Leadership Data Strategy Define and lead enterprise wide data strategy Design and maintain a scalable, secure data architecture Deep understanding of Data Governance, Modern data platforms: Cloud Data quality Management, metadata, lineage and data modelling Experience with Data integration, master data management (MDM) and designing scalable data pipelines Data Privacy, regulatory compliance, and security best practises BI Tools Power BI, Tableau, Looker proficient in data modelling, SQL, and cloud platforms - Azure, AWS, GCP and … their Data services Key Responsibilities Define and implement data architecture strategies in line with the Customer Journey programme Guide the data-driven decision-making process, ensuring that data is effectively utilised to inform business strategies and initiatives. Develop data plan to address data quality issues in line with More ❯

Radius is seeking a highly talented Data Architect and Governance lead for my client going through a digital transformation. Team Leadership Data Strategy Define and lead enterprise wide data strategy Design and maintain a scalable, secure data architecture Deep understanding of Data Governance, Modern data platforms: Cloud Data quality Management, metadata, lineage and data modelling Experience with Data integration, master data management (MDM) and designing scalable data pipelines Data Privacy, regulatory compliance, and security best practises BI Tools Power BI, Tableau, Looker proficient in data modelling, SQL, and cloud platforms - Azure, AWS, GCP and … their Data services Key Responsibilities Define and implement data architecture strategies in line with the Customer Journey programme Guide the data-driven decision-making process, ensuring that data is effectively utilised to inform business strategies and initiatives. Develop data plan to address data quality issues in line with More ❯

Radius is seeking a highly talented Data Architect and Governance lead for my client going through a digital transformation. Team Leadership Data Strategy Define and lead enterprise wide data strategy Design and maintain a scalable, secure data architecture Deep understanding of Data Governance, Modern data platforms: Cloud Data quality Management, metadata, lineage and data modelling Experience with Data integration, master data management (MDM) and designing scalable data pipelines Data Privacy, regulatory compliance, and security best practises BI Tools Power BI, Tableau, Looker proficient in data modelling, SQL, and cloud platforms - Azure, AWS, GCP and … their Data services Key Responsibilities Define and implement data architecture strategies in line with the Customer Journey programme Guide the data-driven decision-making process, ensuring that data is effectively utilised to inform business strategies and initiatives. Develop data plan to address data quality issues in line with More ❯

Data Architect Location: Hybrid working (minimum 2 days a week in Staffordshire Head Office) Type: Full-time Salary: Up to £70,000 Are you a visionary Data Architect ready to shape the future of enterprise data and AI ? I have an amazing opportunity to join my national market leading client to lead the design … and delivery of scalable, secure, and high-performing data solutions that power transformation across a complex, data-rich organisation. What you'll be doing As a Data Architect, you'll be at the heart of enterprise-wide change programmes, working closely with the Lead Data Architect to: Design and deliver enterprise Data & AI architecture aligned with strategic goals. Develop conceptual, logical, and physical data models for both operational and analytical use cases. Define data architecture for solutions involving ETL , data integration, and migration. Establish and maintain data architecture assets, including standards, policies, and integration patterns. Collaborate with Data Stewards, Analysts More ❯

Senior Data Engineer (Grade E3) Application Deadline: 7 September 2025 Department: Corporate Services Employment Type: Permanent - Full Time Location: Edinburgh - Hybrid Compensation: E3 Grade £46,392 - £54,003. Civil Service Pension. Flexible Working Hours. Hybrid Working. Description At Scottish Funding Council, we're embarking on an exciting digital transformation to redefine how we use data to … drive innovation and deliver exceptional value to our customers. Central to this journey is the creation of the Azure-based data ecosystem, enabling us to unlock the full potential of our data through transformative and intelligent solutions. Our Data Engineering team plays a pivotal role in this mission. We manage the entire data lifecycle-from integrating diverse data sources and optimizing storage and processing pipelines to delivering actionable insights-while ensuring compliance with company standards and data protection regulations. As a Senior Data Engineer, you will be a technical leader within this dynamic team, driving innovation and engineering excellence. You'll work closely with teams More ❯

condition, or are neurodiverse. If you'd like to apply and need adjustments made, you can let us know in your application. About the role As an Azure Data Engineer, your role involves designing, implementing, and managing data solutions on the Microsoft Azure cloud platform. Your responsibilities will include, but not limited to: Data Integration Data Integration and ETL (Extract, Transform, Load): You'll develop and implement data integration processes to extract data from diverse sources and load it into data warehouses or data lakes. Azure Data Factory is a key tool for building efficient ETL pipelines. Your goal is … to ensure that data is effectively cleaned, transformed, and loaded. Your sources will include Rest API from various systems. Being able to onboard data from API's is essential. Use of Postman for testing outside of ADF is a preferred skill. Data Engineering As an Azure Data Engineer, you'll handle and More ❯

that adventure. Our employees are at the heart of delivering impactful and meaningful work for our clients and helping them to reach for and realise their vision. Snowflake Data Architect Slalom is seeking an experienced Snowflake Data Architect to contribute to its growing Data Capability. The ideal candidate is a professional with experience in … designing, developing, validating and communicating enterprise data solutions using Snowflake. Deep experience in developing enterprise data management strategies including data lake/warehouse implementations, data movement, data services, data acquisition, data conversion, and archive/recovery. What will you do? Lead as a hands-on architect … and governance concepts such as data quality, metadata management, etc. Experience in designing data engineering solutions using open source and proprietary cloud data pipeline tools Experience in leading data platform development teams Knowledge of data science and visualisation Must have Snowflake SnowPro Core Certification and ideally SnowPro Advanced too More ❯

Main purpose of job: Responsible for facilitating the implementation of the data strategy, under the strategic direction of the Data Steering Group, through iterative technical delivery of a modern data platform solution that provides the business with the following: Improved data quality and consistency, More readily and reliably accessible MI, Enhanced data … driven business insights, Leveraging of digitalisation and automated solutions to achieve greater resource and process efficiencies. This will include contributing towards the design and building of a modern Data platform solution to handle large and complex data sets. Key tasks: Design Data Lake and Data Warehouse solutions Design Data Models … using Data Vault and Dimensional modelling methods Implement automated, reusable and efficient batch data pipelines and streaming data pipelines Work closely with Governance and Quality teams to ensure metadata, catalogue, lineage and known solutions to data issues are optimised Work closely with other data specialists and technicians in the organisation More ❯

Main purpose of job: Responsible for facilitating the implementation of the data strategy, under the strategic direction of the Data Steering Group, through iterative technical delivery of a modern data platform solution that provides the business with the following: Improved data quality and consistency, More readily and reliably accessible MI, Enhanced data … driven business insights, Leveraging of digitalisation and automated solutions to achieve greater resource and process efficiencies. This will include contributing towards the design and building of a modern Data platform solution to handle large and complex data sets. Key tasks: Design Data Lake and Data Warehouse solutions Design Data Models … using Data Vault and Dimensional modelling methods Implement automated, reusable and efficient batch data pipelines and streaming data pipelines Work closely with Governance and Quality teams to ensure metadata, catalogue, lineage and known solutions to data issues are optimised Work closely with other data specialists and technicians in the organisation More ❯

Main purpose of job: Responsible for facilitating the implementation of the data strategy, under the strategic direction of the Data Steering Group, through iterative technical delivery of a modern data platform solution that provides the business with the following: Improved data quality and consistency, More readily and reliably accessible MI, Enhanced data … driven business insights, Leveraging of digitalisation and automated solutions to achieve greater resource and process efficiencies. This will include contributing towards the design and building of a modern Data platform solution to handle large and complex data sets. Key tasks: Design Data Lake and Data Warehouse solutions Design Data Models … using Data Vault and Dimensional modelling methods Implement automated, reusable and efficient batch data pipelines and streaming data pipelines Work closely with Governance and Quality teams to ensure metadata, catalogue, lineage and known solutions to data issues are optimised Work closely with other data specialists and technicians in the organisation More ❯