1 to 25 of 364 Data Pipeline Jobs in the UK

At AX, we are looking for a Data Engineering Lead to join our dynamic team. As the Data Engineering Lead you will have a strong background in data analysis, team management, and expertise in utilizing various Microsoft BI tools and cloud-based technologies including … SSRS, SSIS, SSAS, Azure Synapse, Azure Data Factory and lakes and curated business models. A little bit about us AX is an accident assistance and after care service provider who deliver replacement vehicles to customers who have been involved in non-fault accidents. We pride ourselves on keeping … and engaging with our colleagues is fundamental to ensure we provide an industry leading service to our partners and our customers. As our Data Engineering Lead, your responsibilities will include: Team Leadership Provide leadership, guidance and support to team members, ensuring the successful execution of projects and tasks. more »

Description Who we are At The Stepstone Group, we have a simple yet very important mission: The right job for everyone. Using our data, platform, and technology, we create opportunities for job seekers and companies around the world to find a perfect match, in fair and equitable way. … and you can bring your dog to the office! Job Description The job at a glance As a Director of Product Analytics and Data Science, you will responsible for leading an organisation comprising several Product Analytics, Data Analytics and Data Science teams, covering the … Analytics and Data Science needs in the Product organisation at Stepstone. You will be working closely with Senior Leaders in Product, Tech, Marketing and other areas to ensure we turn our data and product strategy into reality. Equally important, you will enable the managers to foster more »

Senior Data Engineer Python, SQL, Big Data Frameworks London, Geneva, Dubai - 4x a week (no flex) £110-120,000 + bonus Seeking a skilled Data Engineer to join a company specialising in quantitative trading and investment management. This role is pivotal in building and … maintaining the infrastructure that supports their cutting-edge trading strategies. Key Responsibilities: Design and manage scalable data pipelines. Integrate data from various sources and develop ETL processes. Optimize database systems for high performance. Work with data scientists and analysts to meet data needs. Automate data processing tasks to enhance efficiency. Implement data security measures and ensure compliance with industry standards. Requirements: Master's degree in Computer Science, Engineering, Mathematics, or related field from a top uni. Proven experience as a Data Engineer in the more »

Dive into a pivotal role leading data management and analytics transformation. We're on the hunt for an SC Cleared Azure Data Lead with a knack for innovation and a comprehensive skill set: MUST HAVE ACTIVE SC CLEARANCE Key Responsibilities Oversee the evolution of the data platform, encompassing data pipelines, storage, and processing. Establish and nurture robust relationships with stakeholders to guarantee a positive user experience and to identify any potential opportunities for additional services or products. Play a key role in the formulation and execution of the data management … record in the development and management of an Azure cloud-based data & analytics platform. Comprehensive experience in the complete data pipeline development process, encompassing data warehousing, data analytics, and ETL processes. Profound understanding of best practices in data governance more »

Type: 6+ month and possibility of extension Looking for an enthusiastic Spark Scala Engineer who will be responsible for designing, building, and maintaining data pipelines using Apache Spark and Scala. This includes tasks like: Extracting data from various sources (databases, APIs, files) Transforming and cleaning the … data … Loading the data into data warehouses or data lakes (eg, BigQuery, Amazon Redshift) Automating the data pipeline execution using scheduling tools (eg, Airflow) Data analysis and modelling: While the primary focus might be on data engineering, some more »

Principal Data Engineer Job Type: Full-Time Sector: PropTech Location: Preston, Lancashire - Predominantly Remote!! (average expectation of one site visit per month) Salary: £85,000.00+ Standard benefits + Annual Bonus My well established PropTech client is looking for an experienced Principal Data Engineer to join their … team and be instrumental in transforming the way that they consume data and transitioning from On Premise to Google Cloud. This is an exciting time to join a growing business function and gain hands on experience working with market leading technology such as BIG Query, Looker, Data Flow, and more. The Data Team: The Data and Platform Engineering team are the foundation for the Data Office function. Responsible for designing, building, and maintaining their data platform, they extract data from source, transform it into a more »

Stride Recruitment are looking for skilled and motivated SAS Data Engineers to join a consulting firm offering end-to-end data solutions, delivering best-in-class experiences for their external clients. Technical Background: SAS SAS Base … Azure or AWS or GCP Python/PySpark Proficiency in SQL and/or similar data technologies Familiarity with data pipeline tools and ETL processes Knowledge of cloud platforms and data architecture Additional Skills: Excellent communication and stakeholder management skills Ability to translate … complex data into actionable insights. Strong analytical and problem-solving abilities This is a great opportunity to work with an award-winning Data Management and Analytics consultancy, working closely with leading software vendors and top industry experts across a range of sectors, unlocking value and insight more »

We are seeking a highly skilled and motivated Data Engineer. The ideal candidate will have extensive experience in data engineering, particularly with Snowflake, ETL processes, and Power BI. The Data Engineer will be responsible for designing, developing … and maintaining our data infrastructure, ensuring seamless data integration and delivery of high-quality insights. Key Responsibilities: Data Pipeline Development: Design, build, and maintain efficient and reliable ETL processes to move data from various sources into our Snowflake data warehouse. Data Modelling: Develop and maintain data models and schemas to support business needs and ensure data integrity and quality. Data Integration: Integrate data from multiple sources, including APIs, databases, and flat files, into Snowflake. Performance Optimization: Optimize more »

ML systems, CI/CD and data pipelines. Participate in enterprise-wide architecture initiatives for AI/ML. Understand the workflow and pipeline architectures of ML and deep learning workloads. Conduct security risk assessments and threat modeling for AI/ML and other business projects performed thorough … test specification, development, and execution. Define and enforce robust access controls, authentication mechanisms, and encryption standards to safeguard AI/ML systems, training data, provenance, and any sensitive data. Stay abreast of emerging AI security threats, trends, and technologies, evaluating their impact and proactively adapting security measures to … tools, technologies, and frameworks to enhance the overall security posture of AI/ML and other business systems. Develop and maintain relationships between data scientists, data engineers, developers, operations (DevOps, DataOps, MLOps) and business unit leaders to govern and support scalability of the ML and AI more »

Specialist/Architect to join our team. The ideal candidate will possess extensive knowledge and hands-on experience with Palantir Foundry, including its data integration, analytics, and operational decision-making capabilities. As a Palantir Foundry Specialist/Architect, you will be responsible for designing, developing, and maintaining data … driven solutions that leverage Palantir Foundry's platform to address complex business challenges. Key Responsibilities: *Solution Architecture:* - Design and architect end-to-end data solutions using Palantir Foundry, ensuring alignment with business goals and technical requirements. - Develop scalable and efficient data models, pipelines, and workflows within … the Foundry environment. - Collaborate with stakeholders to understand business needs and translate them into technical requirements and solutions. *Data Integration:* - Implement data integration solutions to ingest, transform, and manage large datasets from various sources into Palantir Foundry. - Ensure data quality, consistency, and security throughout more »

Truth Builds Trust If our mission, values, and purpose align with your own, we would love to hear from you! Your opportunity The Data Engineer is a crucial member of the Investments Data, Technology team, tasked with developing, managing, and optimizing data pipelines within … our data ecosystem. Reporting to the Investments Data Lead, this role is instrumental in implementing the data strategy that supports front office stakeholders, systems, and clients. The Data Engineer will leverage cutting-edge technology in Snowflake, Python, SQL, and Azure to enhance … within our subsidized onsite canteen Must have skills Proven experience as a Data Engineer, with a strong background in data pipeline construction, data architecture, and data warehousing. Expertise in Snowflake, Python, SQL, and cloud-native ETL/ELT tools. Familiarity with more »

Data Engineer Remote working (1 day per month on-site) Competitive salary plus an industry leading bonus and pension scheme My client is looking for a Data Engineer to work on innovative projects that balance essential services, optimise cutting-edge technologies, and safeguard valuable assets. As … a Data Engineer, you will develop future-proof pipelines and uncover deep insights from diverse datasets, collaborating with technical teams and clients to design, build, and leverage data architectures for ambitious projects. Why You Should Apply Work on impactful and diverse projects. Flexible working hours and … Architecture, Data Insights, Cloud Platforms, Python Programming, Linux, Windows, Version Control, Git, Unit Testing, Integration Testing, Relational Databases, SQL, ETL Pipelines, Data Cleaning, Data Merging, Data Visualisation, Exploratory Data Analysis, Technology Readiness, Proofs-Of-Concept, Data-Driven more »

This is an exciting opportunity to join one of Europe's fastest growing health-tech startups, working in a Data team that has made huge steps in moving our innovative platform forwards This role is Hybrid but only requires 1-2 days a month in our London office … the UK leader in revolutionising affordable care for the better, for our service users and society and the perfect coordination across care practice, data, science and technology is how we’ll get there. About Data at Cera Data has a critical role to play … in two distinct areas. First, data about our operations, covering everything from hiring and retaining carers to the delivery of care, helps us to observe and understand how we are performing, what is working and what we could do to better deliver care. Second, data about more »

Truth Builds Trust If our mission, values, and purpose align with your own, we would love to hear from you! Your opportunity The Data Engineer is a crucial member of the Finance Analytics technology team, tasked with developing, managing, and optimizing data pipelines within our data ecosystem. The Data Engineer will leverage cutting-edge technology in Snowflake, Python, Informatica, and Azure to improve our data capabilities and support our Finance business partners in their decision-making processes. Key responsibilities include: Design, build, and maintain efficient, reliable data pipelines … use within our subsidized onsite canteen Must have skills Experience as a Data Engineer, with a solid background in data pipeline construction and data architecture using Snowflake, Python, and Informatica Familiarity with Azure and other cloud-native technologies Understanding of finance industry data more »

Data Technical Lead Location: Hybrid - London Package: Negotiable + Benefits Brown & Brown, Inc. is one of the largest insurance brokers globally having in excess of $4bn revenue in 2023 by providing risk management solutions to help protect what our customers value most. We have experienced significant growth over … size of the business by improving customer experience, growing organically and acquiring, accelerating knowledge and capabilities of teammates and using technology with purpose. Data and Data services is a key strategic enabler in Brown and Brown Europe as we look to integrate acquired business, enable them … to provide great customer service and grow through the provision of superior data services As a Technical Lead for our Data Team, you will play a critical role in shaping our data strategy, driving technical excellence, and ensuring the successful delivery of data more »

are a tech for good business dedicated to promoting transparency in the corporate world they are seeking a highly skilled and experienced Lead Data Engineer. You will be the first hire in the Data Engineering team allowing you to take the drivers seat regarding the direction … taken by the data team to follow. Key Responsibilities: Platform Development: Design, develop, maintain, test, and optimise a scalable and reliable data platform for collecting, processing, and transforming large volumes of company data from various sources. Collaboration: Work with cross-functional teams to understand … comprehensive documentation for data processes, schemas, and data dictionaries. Process Improvement: Identify and implement enhancements to improve data pipeline efficiency and reliability. Cloud Management: Actively manage cloud-based data services to balance performance and cost. Mentorship: Mentor and guide junior data more »

Requirements At least two years of professional experience as a data engineer with a proven ability to provide effective data solutions in a production setting. Knowledge of developing real-time data stream systems (ideally Kafka). Proven track record in developing data systems using PySpark and Apache Spark for batch processing. Capable of managing data intake from various sources, including data streams, unstructured data, relational databases, and NoSQL databases. Extensive knowledge in distributed systems, cloud architecture, and data pipelines. Proficiency in Python … programming (knowledge of Scala or Rust is a plus). Familiarity with ETL principles in contemporary data applications (Dagster, Airflow, Perfect). Familiarity with AWS services such as Glue, Redshift, Athena, and S3. Proficiency with Terraform, Kubernetes, and ArgoCD (expertise not required; cloud team support available). Thorough more »

AWS Data Architect (Must have current SC) Amber Labs was born out of the recognition that many organisations require support on their data maturity journey. Our team is passionate and experienced in delivering cutting-edge data capability. Amber Labs was established with a vision … to provide specialist services, automation, accelerators competitively to organisations across EMEA. Job Summary We are seeking a highly skilled AWS Data Architect to join our dynamic team. The ideal candidate will be responsible for designing, implementing, and optimizing our data architecture on the AWS platform. You … will work closely with various teams to ensure data integrity, security, and availability, driving our data strategy forward Key Responsibilities Design and Implement Data Solutions: Develop scalable and efficient data architecture solutions on AWS. Create and maintain data pipelines using more »

be a developer in the Client Reporting Product Team within Distribution Technology. The Client Reporting Product team partners with Marketing, Reporting Operations and Data Management teams in Technology to build data integration solutions for the automation of a wide variety and high volume of reports. This … software development skills and can pick up new technologies and assist in all aspects of the team’s work such as development using data warehouse and master data vendor platform tools. You will not only need excellent problem-solving skills, but you should also be able … to see the strategic impacts of any cross-team dependencies and escalate where necessary. You will require strong teamwork skills. The distribution data space is in a transition period. Significant data assets are located in on-premise hardware and software, such as Oracle, Informatica, Autosys, and more »

s overall commitment to responsible business practices. Ready to level up your career? Playtech‘s BI Unit is looking for an expert Senior Data Engineer with excellent analytical mind set and problem-solving skills to join our Data Programme team Job Description Your influential mission. You … will... Be designing, implementing, and maintaining scalable and efficient data pipelines, ensuring data quality, and optimizing data processing performance Strength our data stack to bear the load of our ever expanding, data driven business Develop and maintain data models, ETL processes, and data warehousing solutions to support analytics, machine learning, and reporting Ensure data security and compliance with industry regulations and best practices Identify, analyse, and resolve data-related issues to improve data reliability and maintain data more »

Job Overview Position: Senior Data Engineer In this role, you will be responsible for storing, processing, modeling, and applying data science to ensure that data and insights are available for data analytics and business intelligence (BI) systems. You must be proficient in … data assessment and profiling, source system analysis, and providing a clear vision for processing and interpreting data to support its end-use. Ideal Candidate Profile The ideal candidate will have a background in Data … Engineering or Data Warehousing (Data Engineer/DW Developer) with robust DW development and ETL/ELT data pipeline experience, along with expert knowledge of SQL. Key Skills - Mandatory Strong data modeling and SQL/database design skills Proficiency in ETL more »

As a Senior Data Engineer, you will play a central role in shaping our data infrastructure and operations. Your primary focus will be on establishing a robust foundation that is sustainable for future growth. You'll be responsible for collecting data from various sources … ensuring its structure, and making it easily accessible for Business Intelligence and Data Science purposes. Key Responsibilities: Data Infrastructure Ownership : Take ownership of the end-to-end data infrastructure, from aggregating data to designing, building, and maintaining scalable and efficient pipelines. Cross … Cloud Platform (GCP) is highly preferred (experience with other cloud platforms like AWS or Azure is also considered). Familiarity with data pipeline scheduling tools like Apache Airflow. Ability to design, build, and maintain data pipelines for efficient data flow and processing. Understanding more »

Data Engineer We are seeking a Data Engineer to join our friendly team. Our client is building a new data platform centred around a Microsoft SQL server-based data warehouse that runs on AWS RDS. This role is crucial in expanding our … client's data capabilities and supporting the launch of our new platform and mobile app. Key Responsibilities: Support and maintain the central Data Platform, ensuring its availability and performance to meet business needs. Build data pipelines to connect APIs, files, databases, and data streams to our client's Data Platforms. Shape data from various sources into a transactional ODS for detailed reporting and a Kimball-modelled Data Warehouse for analytics and dashboards. Define and embed data quality measures within data pipelines. more »

About Us: iOpt Ltd is a IoT technology company using sensors to capture property data to help build smart solutions for sustainable and efficient living. The company works with a range of clients across different sectors. We are an innovative and rapidly growing company and we now require … a talented and motivated Data Analyst to join our team and contribute to our exciting journey. Job Purpose This role has been created to support the growth in our Data team in iOpt. You will be responsible for making sure we have a strong understanding of … Project Management: Apply knowledge and experience of project management methodologies. Utilize appropriate tools and techniques to contribute effectively to project success. Data Pipeline Development: Design, build, test, and deploy data pipelines to support analytic products. Ensure the efficiency and reliability of data flows more »

We are seeking an experienced Python Data Engineer Consultant to join a leading hedge fund. The ideal candidate should have a proven ability to design and implement data solutions, with expertise in extracting, transforming, and loading (ETL) data from various sources, including websites, PDFs … and Real Time data streams. Role: Python Data Engineer Client: Leading Hedge Fund Type of role: contract (6 months) Location: London office Rate: up to £1500/day Key Responsibilities: Architect, develop, and optimize data pipelines for extracting, transforming, and loading data from diverse alternative data sources, such as web scrapes, PDF files, and Real Time data infrastructures. Implement and manage database solutions in SQL Server and various cloud-based environments, ensuring data integrity, efficiency, and accessibility. Build and maintain Real Time data more »