15 of 15 Apache Airflow Jobs in Yorkshire

knowledge of optimisation techniques for large-scale data processing Strong proficiency in SQL and Python for handling complex data problems Hands-on experience with Apache Spark (PySpark or Spark SQL) Experience with the Azure data stack Knowledge of workflow orchestration tools like Azure Data Factory or Apache Airflow … cost optimisation strategies for cloud data platforms Experience with data quality frameworks and implementation Experience with data visualisation tools like Power BI or Apache Superset Experience with other cloud data platforms like AWS, GCP or Oracle Experience with modern unified data platforms like Databricks or Microsoft Fabric Experience with ...

writing automation scripts or custom AI agents using node or python. Data Engineering (30%): Experience deploying and maintaining data orchestration toolsets (e.g., dbt, Apache Airflow), ETL/ELT methodologies, real‐time streaming (e.g., AWS Kinesis, Apache Kafka), Vector databases, and RAG architectures. Observability & FinOps: Experience implementing modern ...

Solid experience with modern data engineering tools and patterns, with real depth in several of SQL, Spark, Databricks/Delta Lake, orchestration tools (Dagster, Airflow, Temporal), and dbt. Comfort with modern deployment practices: CI/CD, containerisation (Docker), and cloud-based infrastructure. It's a bonus ...

large scale data. Highly proficient working with Infrastructure as Code (Terraform, Ansible) Highly proficient in Python. Experience in orchestration tools like Dagster, Prefect and Airflow are advantageous. Experience with Spark, Databricks, data lakes/lakehouses. Knowledge of good data modelling and design patterns; in-depth knowledge of relational databases ...

workstreams or managing delivery teams. Working within highly regulated or public sector environments. Experience with technologies such as Databricks, Snowflake, Fabric, Spark, Data Factory, Airflow or dbt would be highly advantageous. Please note that SC Clearance is required for this position Apply now or send your CV directly ...

Solid experience with modern data engineering tools and patterns, with real depth in several of SQL, Spark, Databricks/Delta Lake, orchestration tools (Dagster, Airflow, Temporal), and dbt. Comfort with modern deployment practices: It's a bonus if you've shaped these for a team, not only worked within ...

Feature engineering. Exposure to common data analysis and ML technologies such as on scikit-learn, pandas, NumPy, XGBoost, LightGBM. Exposure to tools like Apache Oozie, Apache Airflow. £70,000 - £105,000 a year Why you’ll love it here: We want to make Prima a happy and empowering ...

Skills and knowledge Demonstrable experience designing, deploying, and operating big data platforms at scale. Hands‐on experience with OSS data ecosystems (e.g., Spark, Trino, Airflow, Apache Superset, and similar technologies). Knowledge of cloud platforms and traditional on‐premise hosting; hands‐on experience with Azure, AWS and/ ...

usage exports, AI vendor billing data, marketplace billing, or the FOCUS specification Experience with Google BigQuery, or with data pipelines and orchestration tooling (Dagster, Airflow) Experience building AI-powered product features (LLM integration, intelligent automation) Exposure to FinOps practices or cloud financial management products BA/BS degree ...

culture of engineering excellence, continuous learning, and cross-pollination of knowledge. Own the technical roadmap for Airalo’s data infrastructure (GCP, BigQuery), orchestration (Dagster, Airflow), and ingestion (Fivetran, custom APIs). Drive data-platform architecture decisions, turning ambiguous business problems into scalable, production-grade technical designs. Bridge the data … background with major cloud platforms (GCP preferred) and cloud-native data warehouses (BigQuery preferred, or Snowflake/Redshift). Strong experience with orchestration tools (Airflow, Dagster), ELT pipelines (Fivetran, dbt), and distributed data processing frameworks (Apache Spark, Flink). Hands-on experience using AI tools to accelerate engineering ...

stakeholder communication skills Desirable: Experience overseeing or supporting data ingestion pipelines, including APIs and event-driven data sources Familiarity with orchestration tools such as Airflow and modern ELT architectures Experience governing or implementing data CI/CD pipelines (e.g. dbt tests, deployment pipelines, automated checks). We currently … design input, reviews, and mentoring Influence how analytics data is built, trusted, and operated at scale Work with a modern stack: AWS, Snowflake, dbt, Airflow, SQL, and Python Join us as we redefine travel experiences and create memories for millions of passengers. At Jet2.com and Jet2holidays, your potential ...

reason about the performance of non-trivial analytical queries Hands-on experience building and operating data pipelines with an orchestration framework. Dagster or Airflow is highly desired ; equivalent experience with Prefect, dbt or a comparable tool is relevant if you're ready to work in Dagster/Airflow ...

reason about the performance of non-trivial analytical queries Hands-on experience building and operating data pipelines with an orchestration framework. Dagster or Airflow is highly desired ; equivalent experience with Prefect, dbt or a comparable tool is relevant if you're ready to work in Dagster/Airflow ...

game production cycle or long-term project; A strong understanding of Photogrammetry workflows and software; Proficiency in Python; Experience with build systems like Jenkins, Airflow, or Celery; An understanding of a 3D asset pipeline and color management; Ability to think creatively and develop innovative solutions to complex problems. Pluses ...

Highly proficient in Python. Highly proficient in version control and Git/GitHub. Experience of workflow management tools, e.g. Nextflow, WDL/Cromwell, Airflow, Prefect, Dagster Understanding of containerisation (e.g. Docker) and deployment (e.g. Kubernetes). Good understanding of cloud environments (ideally Azure), distributed computing and scaling workflows … pipelines Understanding of common data transformation and storage formats, e.g. Apache Parquet. Awareness of data standards such as GA4GH ( ) and FAIR (). Experience with Spark, Databricks, data lakes. Follow best practices like code reviews, clean code and unit tests. Experience working in an agile development team. Why join ...