2 of 2 Apache Airflow Jobs in Yorkshire

days per week in the office. * A collaborative culture and clear opportunities for technical growth. * Exposure to modern tooling including AWS, Redshift, Airflow, Terraform, and dbt. * The chance to shape a data platform and have meaningful influence on engineering standards. How to Apply If you are interested in this ...

Design and implement high-throughput, fault-tolerant ingestion pipelines for both batch and streaming data into BigQuery. Architect data ingestion solutions using GCS, Dataflow (Apache Beam), Pub/Sub, Dataproc, Cloud Composer (Airflow), and BigQuery Storage Write API . Define data loading frameworks , schema evolution strategies, mapping rules … data ingestion and integration frameworks Advanced SQL and BigQuery optimization skills (partitioning, clustering, cost optimization) Hands-on experience with ETL/ELT tools (Airflow, Dataflow, dbt, etc.) Proficiency in Python and/or Java for pipeline devel ...