etc) Spark or other distributed big data systems (e.g. Hadoop, Pig, Hive, etc.) Stream-processing frameworks (e.g. Kafka) Data pipeline orchestration tools (e.g. Airflow, Prefect, Dagster, etc.) Job Requirements: Bachelor's/Master's in Computer Science, a related field, or equivalent work experience Fluency in Portuguese and English Experience More ❯
/Azure/GCP (preferably AWS), Redshift/BigQuery, Apache Spark/EMR, Docker, Python, and SQL. Hands-on expertise with workflow orchestration tools (Prefect, Mage, or Airflow). Experience with MPP architectures and managing large-scale data processing. Familiarity with data visualisation platforms such as Looker/Looker Studio More ❯
or multiple systems. You know how to create repeatable and reusable products. Experience with workflow management tools such as Nextflow, WDL/Cromwell, Airflow, Prefect and Dagster Good understanding of cloud environments (ideally Azure), distributed computing and scaling workflows and pipelines Understanding of common data transformation and storage formats, e.g. More ❯
of common data transformation and storage formats, e.g. Apache Parquet. Good understanding of cloud environments (ideally Azure), and workflow management systems (e.g. Dagster, Airflow, Prefect). Follow best practices like code review, clean code and unit tests. Familiar with version control and Git/GitHub. Understanding of containerisation (e.g. Docker More ❯
industry. Conducted and analysed large scale A/B experiments Experience mentoring team members Experience with workflow orchestration technologies such as Airflow, Dagster or Prefect Experience with technologies such as: Google Cloud Platform, particularly Vertex AI Docker and Kubernetes Perks of joining us: Company pension contributions at 5% Individualised training More ❯
Bath, Somerset, United Kingdom Hybrid / WFH Options
Opus Recruitment Solutions Ltd
skills across: Python Python Frameworks (NumPy, Django, etc) Writing and implementing Code Testing Quality assurance Software tool deployment AWS (Lambda, AWS integrations etc) Metaflow & Prefect Key will be working with Raster Data This will be a 9 month contract Outside of IR35 paying £450 a day. The client is based More ❯
Glastonbury, Somerset, United Kingdom Hybrid / WFH Options
Opus Recruitment Solutions
skills across: Python Python Frameworks (NumPy, Django, etc) Writing and implementing Code Testing Quality assurance Software tool deployment AWS (Lambda, AWS integrations etc) Metaflow & Prefect Key will be working with Raster Data This will be a 9 month contract Outside of IR35 paying 450 a day. The client is based More ❯
Bath, Glastonbury, Somerset, United Kingdom Hybrid / WFH Options
Opus Recruitment Solutions
skills across: Python Python Frameworks (NumPy, Django, etc) Writing and implementing Code Testing Quality assurance Software tool deployment AWS (Lambda, AWS integrations etc) Metaflow & Prefect Key will be working with Raster Data This will be a 9 month contract Outside of IR35 paying £450 a day. The client is based More ❯
skills across: Python Python Frameworks (NumPy, Django, etc) Writing and implementing Code Testing Quality assurance Software tool deployment AWS (Lambda, AWS integrations etc) Metaflow & Prefect Key will be working with Raster Data This will be a 9 month contract Outside of IR35 paying £450 a day. The client is based More ❯
and non-Geospatial domains Experience with Machine Learning (sci-kit learn, tensorflow, metaflow, MLOps) Preferred Experience: Knowledge of Rust Experience with frameworks like Metaflow, Prefect, etc. Experience with geospatial libraries i.e. Raster, Geo-pandas, Vector databases This role would see you work in their South West office c.1 - 2 days More ❯
and non-Geospatial domains Experience with Machine Learning (sci-kit learn, tensorflow, metaflow, MLOps) Preferred Experience: Knowledge of Rust Experience with frameworks like Metaflow, Prefect, etc. Experience with geospatial libraries i.e. Raster, Geo-pandas, Vector databases This role would see you work in their South West office c.1 - 2 days More ❯
Prefect, or Airflow Solid understanding of distributed compute engines such as Spark/Databricks. Confidence using Docker, Kubernetes, and Helm in cloud environments. Experience building software for public cloud environments (Azure preferred, AWS/GCP considered). Our Future Health is the UK's largest health research program, uniting millions More ❯
Design and implement complex data pipelines and architectures. Utilising dbt and Prefect to automate and run your data pipelines. Collaborate with cross-functional teams to align data engineering efforts. Engaging with clients to identify available data and help them achieve business goals. Key Candidate Requirements Knowledge Advanced proficiency in programming More ❯
control Setting up monitoring and alerting frameworks to track model drift, data quality, and inference health Leveraging orchestration tools such as Dagster, Airflow, or Prefect to manage and scale ML workflows Supporting ongoing infrastructure migration or optimisation initiatives (e.g. improving cost efficiency, latency, or reliability) Partnering with product and engineering … on AWS , using services like ECS, EKS, Fargate, Lambda, S3, and more Familiarity with orchestration and workflow scheduling tools such as Dagster , Airflow , or Prefect Knowledge of CI/CD best practices and tools (e.g. GitHub Actions, Jenkins, CodePipeline) Exposure to monitoring and observability tools for ML systems (e.g. Prometheus More ❯
the storing and processing of most of our data. You will work with other Data Engineers using tools such as Python, SQL, Airflow and Prefect to ensure quality and collaborate with other teams across the business. What you'll be working on: Implementing and supporting ETLs and Data Quality monitors … should apply if you have: 3+ years of data engineering, analytics or machine learning experience Advanced skills in Python and SQL Experience with Airflow, Prefect or other task orchestration tools Familiarity with modern data stack - you know the current trends and what tools to use for the job A proactive More ❯
Join us a Data Engineer at Peregrine Resourcing. At Peregrine, we’re always seeking Specialist Talent that have the ideal mix of skills, experience, and attitude, to place with our vast array of clients. From Business Analysts in large government More ❯
Join us a Data Engineer at Peregrine Resourcing. At Peregrine, we’re always seeking Specialist Talent that have the ideal mix of skills, experience, and attitude, to place with our vast array of clients. From Business Analysts in large government More ❯
