6 of 6 Apache Airflow Jobs in Wales

modern data architectures, including cloud-based and distributed systems. Strong proficiency in SQL and Python. Experience with data engineering and pipeline technologies such as Apache Airflow and Spark. Experience working with cloud platforms, particularly AWS and Azure. Knowledge of big data technologies and distributed data processing. Understanding ...

required, we are seeking Engineers who have experience with one or more of the following technological areas: Analytics & Platform Frameworks Orchestration like Airflow or AWS Glue Database Technologies Data processing libraries & Schema Management tools like Pandas and Pydantic Storage Formats: Familiarity with data types such as JSON and Parquet ...

related data-access frameworks. Experience with Go, Node.js, or Java environments. Familiarity with API gateways, service mesh technologies, and reverse proxies. Experience with Airflow, Prefect, Dagster, or other workflow orchestration tools. Strong understanding of caching, performance optimization, and observability platforms. Experience with Terraform, Pulumi, Helm, GitHub Actions, GitLab ...

UK. We seek a Data Engineer experienced with cloud-native systems to drive data pipelines, quality, and scalability. You will work with Python, Spark, Airflow, and AWS technologies to deliver real-time and batch solutions. Join a collaborative team focused on high throughput, low latency data delivery and data ...

snewConsumer Bureauwithdata quality,lineage, andauditabilityas primary goals whilstmaintaininghigh throughputand scalability.Thedata platformwill be builtusingAWSinfrastructure such as S3and S3-tables(Iceberg), AuroraServerless, DynamoDB, Lambda,Fargate, ECS, Airflow,Sparkand Kinesis.Theplatformisexpectedto manage overone hundred millionobjectswith a high volume of daily updateshandlingaddition, deletion,andcorrectionof our dataand indexesinanauditable manner.The platform will employ a sophisticated matching … Amigo's refinement sessions all the way to post-deployment observability. Design and build metadata driven,eventbaseddistributeddata processing platformusing technologies such asPython,Spark,Airflow, Iceberg,DynamoDB,AWS Glue, S3. Playan activerole in the design, development, and deployment ofour business-criticalsystem. Building and scalingCreditsafe’sdata pipelinescombining batch and streaming approachesandtargetingsub ...

solution. Design mechanical components in CREO Parametric 3D - Pro/E CAD software. Conduct or direct CFD and FEA analyses to optimize Semiconductor Equipment airflow requirements, and mechanical design. Reduce 3D mechanical designs to 2D drawing and proper application of mechanical tolerancing/GD&T to achieve desired … Experience with 2D drafting and tolerance analysis. Experience in several of these areas is desirable: CREO Parametric 3D MCAD - Pro/E software, CFD airflow analysis; FEA analysis; Design for manufacture (DFM); Design of welded sheet metal structures and frames; Measurement instrumentation, testing and data acquisition; Programming in Python ...