4 of 4 Apache Spark Jobs in Yorkshire

Stock Exchange. With 3,000 employees and 32 offices in 12 countries we’re a business with lots of opportunity for people with talent, spark and lots of ambition. If you want to build a great career with a company that prioritises strong values – such as integrity and courage ...

services that support data engineering teams (e.g., ingestion services, metadata, access patterns, developer enablement). Familiarity with big data ecosystems - particularly data bricks/spark (e.g., data lakes, distributed compute, streaming) and observability practices. Prior exposure to cyber/security engineering domains (useful, not required). Lots of cloud ...

serving - training/inference, focusing on cost efficiency, performance, and resilience. Big data & distributed processing:* Develop and tune large-scale batch/stream workloads (Spark or similar); improve job efficiency, stability, and operational readiness in production. Software engineering:* Strong Python + SQL; apply CI/CD, automated testing, modular ...

Build and manage CI/CD pipelines and Infrastructure as Code using Terraform, GitLab CI, and GitHub. Support big data platforms using Hadoop, Hive, Spark, EMR, Athena, and Redshift. Implement monitoring, logging, security, and compliance controls across cloud environments. Drive automation, containerisation, and platform resilience using Kubernetes, Python … with Terraform and CI/CD tools (GitLab CI, Concourse, GitHub). Strong Linux administration experience. Experience with big data technologies including Hadoop, Hive, Spark, and Amazon EMR (or equivalent). Knowledge of monitoring, logging, and security tools. Desirable Skills Kubernetes/EKS or other container platforms. Python scripting ...