1 to 25 of 32 Contract Data Pipeline Jobs in the UK

Job Title: Data Validation Engineer Location: Whitley, Coventry Rate: £34.46ph via an Umbrella Company (Inside IR35) or £26.16ph PAYE Employment Type: Contract (31.3.26 (Maternity Cover)) Hours: Standard 40 Hours Per Week Reference: J39970 Job Description: Be a part of the team testing the next generation of electrified propulsion … testing environment? Have you developed system/tools using Python or Matlab or Concerto any other program language? Have you set data pipeline and data storage in the industrial cloud platforms? Have you used Tableau or other data visualisation tool? Experience Required: Educated … natural science, mathematical or engineering or computing discipline. Designing applications in different programming languages (preferable Python, Concerto). Build data systems & pipelines, data validation and management in cloud platforms (AWS, Google Cloud). Complex data analysis skills, including hypothesis testing, uncertainty analysis, and process more »

Data Management Consultant - DV Cleared Location: London Rate: £750 a day (inside IR35) Clearance: Site specific DV Clearance Keywords: data lakes, data warehouses, data pipelines, data meshes, data marketplaces. A global IT consultancy are looking to grow their … Data Office team, working directly with the client to ensure that all aspects of their Data Management strategy are delivered. Due to the nature of the end client you would need to hold site specific DV Clearance and the role is on site, 4 days a … week. Your skills and experience: Data Modelling: Have developed Physical Data Models, Logical Data Models or Conceptual Data Models. Have experience in producing dictionaries, database designs or data flow diagrams Have experience working with Ontologies or Knowledge Graphs Have experience more »

Senior Data Engineer - Bristol - 90% onsite - 6 month contract - £78.70 per hour, outside IR35 - Sole UK national and DV Clearance required This role requires strong expertise in building and managing data pipelines using the Elastic Stack (Elasticsearch, Logstash, Kibana) and Apache NiFi. The successful candidate will … design, implement, and maintain scalable, secure data solutions, ensuring compliance with strict security standards and regulations. This is a UK based onsite role with the option of compressed hours. The role will include: - Design, develop, and maintain secure and scalable data pipelines using the Elastic Stack … with a focus on ensuring data accuracy, quality, and security. • Proactive problem-solving mindset and ability to troubleshoot complex data pipeline issues Nice To Have • Experience working in government, defence, or highly regulated industries with knowledge of relevant standards. • Experience with additional data more »

Senior Data Engineer - Bristol - 90% onsite - 6 month contract - £78.70 per hour, outside IR35 - Sole UK national and DV Clearance required This role requires strong expertise in building and managing data pipelines using the Elastic Stack (Elasticsearch, Logstash, Kibana) and Apache NiFi. The successful candidate will … design, implement, and maintain scalable, secure data solutions, ensuring compliance with strict security standards and regulations. This is a UK based onsite role with the option of compressed hours. The role will include: - Design, develop, and maintain secure and scalable data pipelines using the Elastic Stack … with a focus on ensuring data accuracy, quality, and security. • Proactive problem-solving mindset and ability to troubleshoot complex data pipeline issues Nice To Have • Experience working in government, defence, or highly regulated industries with knowledge of relevant standards. • Experience with additional data more »

Job Overview: We are seeking a highly skilled and detail-oriented Data Engineer with expertise in Power Automate to join our growing team. The ideal candidate will have strong experience in designing, building, and optimizing data workflows, automation processes, and integrating systems using Power Automate. This … role will involve working closely with cross-functional teams to support data operations, optimize data workflows, and automate business processes efficiently and effectively. Key Responsibilities: Design, develop, and implement automated workflows and processes using Power Automate to streamline data operations, integrate systems, and enhance … data pipelines. Collaborate with business stakeholders to identify automation opportunities and define data-driven solutions that increase efficiency and reduce manual work. Build and maintain ETL pipelines for data extraction, transformation, and loading, using Power Automate and other related tools. Develop, deploy, and manage more »

Azure Data Engineer Neogotiable Day Rate - Outside IR35 Remote - with some travel to sites 12 Month Duration Role Overview: Supporting Project AIilluminate, the data engineer will leverage large language models (LLMs) to enhance service efficiency, ensure compliance, optimize processes, analyze customer sentiment, and extract insights from … large text datasets. Key Competencies: - Strong data design, modeling, and metadata expertise. - Ability to translate business problems into data solutions. - Identifies opportunities for change and data-driven decision-making. - Excellent communication skills to convey complex ideas clearly. - Quick adaptability to new technologies and concepts. … Responsibilities: - Consolidate disparate data hubs into a centralized enterprise data platform. - Design and build scalable Azure-based big data solutions. - Develop data ingestion, transformation, and visualization pipelines. - Lead cloud adoption and automation initiatives on Azure. - Implement automated data workflows for more »

Data Engineer Initial 6-month Contract Role Hybrid working in Northampton office £450 - £500, Inside IR35 We're recruiting on behalf of an IT Services Provider who are looking for a Data Engineer to to work on large scale digital & data solutions. As a … Data Engineer you will be responsible for: Designing, developing, and maintaining data processing systems utilising the Hadoop framework to efficiently manage and analyse large-scale datasets. Assemble large, complex data sets that meet functional/non-functional business requirements. Develop … modern data management tools automate processes and enhance data quality for the customer. Create and maintain optimal data pipeline architecture. Additional qualifications that could help you succeed even further in this role include: Minimum of 6 years' experience working as a Data more »

Data Engineer - Azure Data Factory - Remote contract for initial 3 months Outside IR35 with rate of £(Apply online only) available We are looking for a strong Data Engineer … with Azure (Including ADF, Azure Data Factory) experience. A large part of the role will be to deliver the Data Pipeline Development and subsequent Performance Tuning. Building ETL processes using Azure Data Factory. We are looking for skills in the following areas - Azure … Database - ETL (Extract, Transform, Load) - Knowledge of SQL and C# - Experience with Data warehousing concepts and practices - Some exposure Azure DevOps for pipeline integration. This is a fully remote role with interviews being booked for later this week. Please share your CV if you have the required more »

In this role you will: Work alongside a team of data engineers in the design, development, and maintenance of data management solutions, including data modeling, ETL processes, and data pipelines. Develop and implement data quality checks and monitoring processes to … ensure data accuracy and integrity. Design and build infrastructure. Design and develop reports, dashboards, and visualisations to provide insights into business performance. Proactively identify and resolve technical issues, including data anomalies, performance bottlenecks, and data quality problems. Drive technical governance initiatives, including documentation, data lineage, and data access control. Participate in architecture discussions. Evaluate and recommend new technologies and tools to improve the efficiency and effectiveness of the team. Work with a collaborative global team of varied disciplines, skills, and experience. To be successful in this role, you should meet more »

Job Title: PySpark Engineer - Data Specialist Engagement: Contract Rate: £350 - £500pd Client: Tier 1 Investment Bank Duration: 10 months to Permanent Start Date: ASAP (HAS TO BE BY 31ST March) Project: PySpark/SQL Developer - Investment Banking/Data Processing/Automation - Sought by a Tier … investment bank based in London. Hybrid - Contract. Inside IR35 - Umbrella Key Responsibilities: Develop, maintain, and optimize PySpark data processing pipelines in a fast-paced investment banking environment, dataframes, spark streaming, Python, Spark, PySpark xp vital CICD (Jenkins, Git) Collaborate with cross-functional teams, including data engineers … improve processing efficiency. Qualifications: Proven experience in data processing and automation. Strong proficiency in PySpark and Apache Spark for data pipeline development. Expertise in CICD pipelines, Jenkins, Git Excellent problem-solving skills, attention to detail, and ability to manage complex datasets. Strong communication skills with more »

Azure Data Engineer - ADF, Snowflake - £425pd inside IR35 I'm currently looking for a Data Engineer to join a company that is based in London. They need someone to start in the next few weeks. Location: London (twice a week) Length: 3 months Day Rate … per day. IR35 Status: Inside of IR35 Required experience will include: Expertise in designing and implementing data pipelines using Azure services such as Azure Data Factory , Azure Synapse and Snowflake . … Hands-on experience with SQL database design Deploy applications using Docker and contribute to CI/CD pipelines. Utilise Python for data pipeline development and visualisation. If you are interested in this Data Engineer role please apply with your most recent CV. Alternatively email me more »

JOB SUMMARY Senior Data Engineer DV Cleared Onsite in Malvern £500 - £600 per day - Outside IR35 JOB DESCRIPTION This role requires strong expertise in building and managing data pipelines using the Elastic Stack (Elasticsearch, Logstash, Kibana) and Apache NiFi. The successful candidate will design, implement, and … maintain scalable, secure data solutions, ensuring compliance with strict security standards and regulations. This is a UK based onsite role with the option of compressed hours. The role will include: Design, develop, and maintain secure and scalable data pipelines using the Elastic Stack (Elasticsearch, Logstash, Kibana … and Apache NiFi. Implement data ingestion, transformation, and integration processes, ensuring data quality and security. Collaborate with data architects and security teams to ensure compliance with security policies and data governance standards. Manage and monitor large-scale data flows in more »

Spark, Apache Beam, Google Cloud Dataflow We're working with a forward-thinking consultancy that helps top companies build and scale high-performance data platforms . They take an engineering-first approach , and more than half of their team consists of hands-on engineers. If you love working … with large-scale data processing and cutting-edge cloud technologies, this ones for you. What Youll Be Doing: Building data pipelines and ETL workflows that process huge datasets Designing, optimizing, and maintaining high-throughput reporting solutions Working with Apache Spark for large-scale data … Terraform Deep understanding of Apache Spark for large-scale data processing Knowledge of Apache Beam & Google Cloud Dataflow for data pipeline orchestration A team-first mindset with strong communication skills This is a contract role outside IR35 , so you must be UK-based and have more »

Insurance domain experience Location: Hybrid - London 3 days per week Job Type: Contract - £600 to £700/day Industry: Cyber Security, Cyber Risk, Data Engineering, Insurance Domain About the Role We are seeking a highly skilled Solution Architect with expertise in Azure Cloud, Zero Trust security, and Databricks … an Insurance customer but through a consultancy. In this role, you will play a critical part in designing and implementing cyber security-focused data architectures , leveraging machine learning (ML), cyber data pipelines, and BDAP infrastructure to enhance cyber risk analysis. Key Responsibilities Architect and implement cloud … based solutions in Azure , ensuring alignment with Zero Trust security principles . Design and optimize Databricks Lakehouse architectures for cyber data storage, processing, and analytics. Develop and manage cyber data pipelines , enabling seamless data integration for cyber risk modelling and threat intelligence. Integrate and more »

cloud infrastructure for various products, including Astronomer Airflow and AccelData environments. Facilitate cross-functional integration between vendor products and other systems, such as data lakes, storage, and compute services. Establish best practices for cloud security, scalability, and performance. Manage and configure vendor product deployments, ensuring the setup and … of Airflow workers. Maintain and configure integrations with source control systems (e.g., GitHub, GitLab) for version control. Collaborate with cloud providers (e.g., AWS) for pipeline integration and scaling requirements. Identify, design, and implement internal process improvements, including re-designing infrastructure for greater scalability, optimizing data delivery, and … Work with stakeholders, including design, product, and executive teams, to address platform-related technical issues. Build analytical tools to leverage the data pipeline, providing actionable insights into key business performance metrics, such as operational efficiency and customer acquisition. All profiles will be reviewed against the required skills more »

and develop API-based and event-driven integrations , utilising Azure services. ✅ Oversee the implementation of RESTful APIs, messaging queues (Azure Service Bus), and data pipelines . ✅ Establish and enforce integration best practices , ensuring security, scalability, and maintainability. ✅ Provide technical leadership to development teams, guiding them towards a best more »

and develop API-based and event-driven integrations , utilising Azure services. ? Oversee the implementation of RESTful APIs, messaging queues (Azure Service Bus), and data pipelines . ? Establish and enforce integration best practices , ensuring security, scalability, and maintainability. ? Provide technical leadership to development teams, guiding them towards a best more »

Senior Actuarial Data Scientist (Contract) Location: Hybrid/London Contract Length: 6-12 months (potential for extension) We are hiring a Senior Actuarial Data Scientist to support a leading insurance client in leveraging AI & advanced analytics for pricing, underwriting, and claims optimization. This is an exciting … opportunity to work at the intersection of actuarial science, machine learning, and data-driven decision-making in a dynamic, fast-paced environment. Key Responsibilities: Develop and implement predictive models for risk assessment, pricing, and reserving using AI/ML techniques Utilize actuarial methodologies and data science … tools to enhance claims modeling & loss ratio predictions Work with large structured & unstructured datasets to derive actionable insights Collaborate with actuarial, underwriting, and data teams to integrate AI-driven solutions into business processes Support the development of automated data pipelines for real-time analytics Ensure models more »

is an exciting opportunity to enhance your career and contribute to innovative projects in a collaborative environment. Key Responsibilities: Develop and maintain scalable data pipelines using PySpark to support our data processing needs. Collaborate with cross-functional teams to gather requirements and design efficient ETL processes. … Optimise existing data workflows and troubleshoot issues to ensure high performance and reliability. Implement data processing algorithms and integrate with various data sources, including databases and cloud storage. Work closely with stakeholders to understand business needs and translate them into technical solutions. Document processes … Proven experience as a PySpark Developer or in a similar role, within the investment banking sector. Strong proficiency in PySpark and experience with data manipulation and analysis. Familiarity with big data technologies and frameworks. Solid understanding of data warehousing concepts and ETL processes. Ability more »

require a minimum of 3 days per week on site in central Cambridge. The key experience required for this Bioinformatician position is: CRISPR Data pipelines Python or R Support lab scientists If you do have the relevant skills for this Bioinformatician Engineer contract position, please do apply. more »

platform based in MS Fabric on behalf of a consultancy client of ours. Key Responsibilities: Design and implement Azure Fabric solutions for scalable data architecture. Optimise data pipelines, governance, and security within the Microsoft Azure ecosystem . Collaborate with stakeholders to align technical solutions with business … needs. Requirements: Proven experience as an Azure Data Architect , with expertise in Azure Fabric . Strong knowledge of Data Lake, Synapse, Databricks, Azure Data Factory and Power BI . Ability to work independently in a fast-paced, remote environment. While this position is a more »

the Role: The Python Developer role involves replacing internal systems with Python, requiring proof of project and initial investigation along with system development, data processing, creating data pipelines and focusing on backend ETL tasks. The Python Developer will sit within a small team within Finance Risk … standards for management approval. Writing new Python scripts to handle Informatica logic-type code and convert existing C++ modules into Python code. Holding Data Processing expertise, focusing on data processing tasks, creating data pipelines, and handling backend ETL. What we require from the Python more »

Client: Our client is a global leader in Systems Integration and IT Consultancy. Renowned for their advanced and respected industry-wide Insights and Data Practice, they provide consultancy services, advice, proof-of-values, and industrialised solutions across public, CPG, retail, manufacturing, and utilities sectors. The Role: We are … client's Emerging Tech Team. This role is pivotal in leveraging cutting-edge AI technology and implementing Snowflake solutions to enhance our clients' data platform capabilities. Essential Skills and Experience: Proven track record in global projects implementing AWS, GCP, and Azure platform components on multi-tenant Snowflake DWH. … Expertise in developing enterprise-grade ELT data pipelines within Snowflake. Strong command of data manipulation/wrangling techniques. Adherence to Snowflake best practices including coding practices to DS, unit testing, version control, and code review. Experience working with global teams and Agile delivery methodologies. In-depth more »

investment bank based in London. **Inside IR35 - 3 days a week onsite'' Job Description Seeking a Java Spark Developer with expertise in big data processing, Core Java and Apache spark particularly within finance domain. Candidate should have a strong experience working with financial instruments, market risk and large … scale distributed computing systems. This role involves developing and optimizing data pipelines for risk calculations, trade analytics and regulatory reporting. Key Responsibilities: Develop and optimize scalable Java Spark-based data pipelines for processing and analyzing large scale financial data. Design and implement distributed computing solutions for … risk modeling, pricing and regulatory compliance. Ensure efficient data storage and retrieval using Big Data. Implement best practices for spark performance tuning including partition, caching and memory management. Design and implement distributed computing solutions for risk modeling and regulatory compliance. Maintain high code quality through testing, CI/ more »

days onsite Contract Type: Contract to Perm (inside IR35 via umbrella) Are you a skilled Java Spark Developer with a passion for big data processing? Our client, a leading player in the finance domain, is looking for a talented individual to join their team in Canary Wharf, London. … where your expertise in Core Java and Apache Spark will make a significant impact. Key Responsibilities: Develop and optimise scalable Java Spark-based data pipelines for processing and analysing large-scale financial data. Design and implement distributed computing solutions for risk modelling, pricing, and regulatory compliance. Ensure efficient … data storage and retrieval using Big Data technologies. Implement best practises for Spark performance tuning, including partitioning, caching, and memory management. Maintain high code quality through testing, CI/CD pipelines, and version control (Git, Jenkins). Work on batch processing frameworks for market risk analytics. more »