20 of 20 Apache Spark Jobs in Yorkshire

automation to ensure successful project delivery, adhering to client timelines and quality standards. Implement and manage real-time and batch data processing frameworks (e.g., Apache Kafka, Apache Spark, Google Cloud Dataproc) in line with project needs. Build and maintain robust monitoring, logging, and alerting systems for client … languages like Python, Bash, or Go to automate tasks and build necessary tools. Expertise in designing and optimising data pipelines using frameworks like Apache Airflow or equivalent. Demonstrated experience with real-time and batch data processing frameworks, including Apache Kafka, Apache Spark, or Google Cloud Dataflow. ...

control. Excellent collaboration and communication skills, with strong attention to detail and data quality. Desirable: Exposure to AI/ML data preparation, Python or Spark, Data Vault 2.0, data governance, GDPR, and experience working in mobility, logistics, financial services, or tech-enabled environments. What ...

platforms such as AWS, Azure, or GCP. Exposure to modern data tools such as Snowflake, Databricks, or BigQuery. Familiarity with streaming technologies (e.g., Kafka, Spark Streaming, Flink) is an advantage. Experience with orchestration and infrastructure tools such as Airflow, dbt, Prefect, CI/CD pipelines, and Terraform. What ...

comprehensive data analysis and reporting. Implement and uphold best practices in data governance, security, and compliance, using tools like Azure Purview, Unity Catalog, and Apache Atlas to maintain data integrity and trust. Ensure data quality and integrity through meticulous attention to detail and rigorous QA processes, continually refining … ease. Understanding of Data Lakehouse Architecture: You have a strong grasp of the principles and implementation of Data Lakehouse architecture. Hands-On Experience with Spark-Based Solutions: You possess experience with Spark-based platforms like Azure Synapse, Databricks, Microsoft Fabric, or even on-premise Spark clusters, using ...

comprehensive data analysis and reporting. Implement and uphold best practices in data governance, security, and compliance, using tools like Azure Purview, Unity Catalog, and Apache Atlas to maintain data integrity and trust. Ensure data quality and integrity through meticulous attention to detail and rigorous QA processes, continually refining … ease. Understanding of Data Lakehouse Architecture: You have a strong grasp of the principles and implementation of Data Lakehouse architecture. Hands-On Experience with Spark-Based Solutions: You possess experience with Spark-based platforms like Azure Synapse, Databricks, Microsoft Fabric, or even on-premise Spark clusters, using ...

modern cloud engineering practices. Wherever you land, youll be working with some of the biggest datasets in the UK using everything from Spark and statistical methods to domain knowledge and emerging GenAI applications. The work you could be doing Design and deploy machine learning models for fraud detection, credit … risk, customer segmentation, and behavioural analytics using scalable frameworks like TensorFlow, PyTorch, and XGBoost. Engineer robust data pipelines and ML workflows using Apache Spark, Vertex AI, and CI/CD tooling to ensure seamless model delivery and monitoring. Apply advanced techniques in deep learning, natural language processing ...

modern cloud engineering practices. Wherever you land, youll be working with some of the biggest datasets in the UK using everything from Spark and statistical methods to domain knowledge and emerging GenAI applications. The work you could be doing Design and deploy machine learning models for fraud detection, credit … risk, customer segmentation, and behavioural analytics using scalable frameworks like TensorFlow, PyTorch, and XGBoost. Engineer robust data pipelines and ML workflows using Apache Spark, Vertex AI, and CI/CD tooling to ensure seamless model delivery and monitoring. Apply advanced techniques in deep learning, natural language processing ...

compliance. Collaborate with stakeholders and external partners. Skills & Experience: Strong experience with AWS data technologies (e.g., S3, Redshift, Lambda). Proficient in Python, Apache Spark, and SQL. Experience in data warehouse design and data migration projects. Cloud data platform development and deployment. Expertise across data warehouse ...

analytics tools Familiarity with industry common BI and ETL solutions Hands-on experience in a technical role leveraging SQL, Python, Java and/or Spark to build, operate and maintain data analytics solutions Extensive knowledge of and experience with large-scale database technology (e.g. Snowflake, Netezza, Exadata, Teradata, Greenplum ...

analytics tools Familiarity with industry common BI and ETL solutions Hands-on experience in a technical role leveraging SQL, Python, Java and/or Spark to build, operate and maintain data analytics solutions Extensive knowledge of and experience with large-scale database technology (e.g. Snowflake, Netezza, Exadata, Teradata, Greenplum ...

PyTorch) into real customer value Working closely with engineering teams using TypeScript, Node.js, React, and Next.js Partnering with data teams working across AWS, Kafka, Spark, and modern data pipelines Shaping products built on secure, scalable fintech architecture (Event-driven, microservices, serverless) Running discovery experiments and validating concepts at speed ...

PyTorch) into real customer value Working closely with engineering teams using TypeScript, Node.js, React, and Next.js Partnering with data teams working across AWS, Kafka, Spark, and modern data pipelines Shaping products built on secure, scalable fintech architecture (Event-driven, microservices, serverless) Running discovery experiments and validating concepts at speed ...

PyTorch) into real customer value Working closely with engineering teams using TypeScript, Node.js, React, and Next.js Partnering with data teams working across AWS, Kafka, Spark, and modern data pipelines Shaping products built on secure, scalable fintech architecture (Event-driven, microservices, serverless) Running discovery experiments and validating concepts at speed ...

solving, from ideation through experimentation and production. Strong understanding of statistical concepts and ML feature engineering techniques. Experience analyzing large-scale datasets using SQL, Spark, or similar tools. Preferred/Nice to Have Certifications or academic background in Data Science/Machine Learning/AI. Experience with ML frameworks ...

solving, from ideation through experimentation and production. Strong understanding of statistical concepts and ML feature engineering techniques. Experience analyzing large-scale datasets using SQL, Spark, or similar tools. Preferred/Nice to Have Certifications or academic background in Data Science/Machine Learning/AI. Experience with ML frameworks ...

building and managing real-time data pipelines across a track record of multiple initiatives. Expertise in developing data backbones using distributed streaming platforms (Kafka, Spark Streaming, Flink, etc.). Experience working with cloud platforms such as AWS, GCP, or Azure for real-time data ingestion and storage. Programming skills … similar language. Proficiency in database technologies (SQL, NoSQL, time-series databases) and data modelling. Strong understanding of data pipeline orchestration tools (e.g., Apache Airflow, Kubernetes). You thrive when working as part of a team Comfortable in a fast-paced environment Have excellent written and verbal English skills Last ...

building and managing real-time data pipelines across a track record of multiple initiatives. Expertise in developing data backbones using distributed streaming platforms (Kafka, Spark Streaming, Flink, etc.). Experience working with cloud platforms such as AWS, GCP, or Azure for real-time data ingestion and storage. Programming skills … similar language. Proficiency in database technologies (SQL, NoSQL, time-series databases) and data modelling. Strong understanding of data pipeline orchestration tools (e.g., Apache Airflow, Kubernetes). You thrive when working as part of a team Comfortable in a fast-paced environment Have excellent written and verbal English skills Last ...

driven solutions. • Technical Excellence: Promote and enable adoption of Technical Standards and Engineering Effectiveness within development squads. • Technical Experience: Demonstrate expertise in using SQL (Spark, Dremio), Python, GitHub and data orchestration tools (Airflow, Oozie) for data wrangling, building data pipelines and developing analytical interfaces. • AI-Assisted Analytics: Bring experience … practices, AI capabilities, advanced analytics and automation practices. You will need • Experience developing robust data products that are actionable and scalable.• Expertise of using Spark, Dremio, Teradata or other SQL technologies.• Strong knowledge and experience of using business intelligence, ETL frameworks and visualisation tools such as Tableau.• Experience with ...

driven solutions. • Technical Excellence: Promote and enable adoption of Technical Standards and Engineering Effectiveness within development squads. • Technical Experience: Demonstrate expertise in using SQL (Spark, Dremio), Python, GitHub and data orchestration tools (Airflow, Oozie) for data wrangling, building data pipelines and developing analytical interfaces. • AI-Assisted Analytics: Bring experience … practices, AI capabilities, advanced analytics and automation practices. You will need • Experience developing robust data products that are actionable and scalable.• Expertise of using Spark, Dremio, Teradata or other SQL technologies.• Strong knowledge and experience of using business intelligence, ETL frameworks and visualisation tools such as Tableau.• Experience with ...

driven solutions. • Technical Excellence: Promote and enable adoption of Technical Standards and Engineering Effectiveness within development squads. • Technical Experience: Demonstrate expertise in using SQL (Spark, Dremio), Python, GitHub and data orchestration tools (Airflow, Oozie) for data wrangling, building data pipelines and developing analytical interfaces. • AI-Assisted Analytics: Bring experience … practices, AI capabilities, advanced analytics and automation practices. You will need • Experience developing robust data products that are actionable and scalable.• Expertise of using Spark, Dremio, Teradata or other SQL technologies.• Strong knowledge and experience of using business intelligence, ETL frameworks and visualisation tools such as Tableau.• Experience with ...