17 of 17 Apache Spark Jobs in Yorkshire

platforms such as AWS, Azure, or GCP. Exposure to modern data tools such as Snowflake, Databricks, or BigQuery. Familiarity with streaming technologies (e.g., Kafka, Spark Streaming, Flink) is an advantage. Experience with orchestration and infrastructure tools such as Airflow, dbt, Prefect, CI/CD pipelines, and Terraform. What ...

control. Excellent collaboration and communication skills, with strong attention to detail and data quality. Desirable: Exposure to AI/ML data preparation, Python or Spark, Data Vault 2.0, data governance, GDPR, and experience working in mobility, logistics, financial services, or tech-enabled environments. What ...

technology (AWS or Azure). Strong experience with creation of data ingestion pipeline and ET process. Strong knowledge of big data tool such as Spark, Databricks and Python. Strong understanding of common machine learning techniques and frameworks (e.g. mlflow). Strong knowledge of Natural language processing (NPL) concepts. Strong ...

technology (AWS or Azure). Strong experience with creation of data ingestion pipeline and ET process. Strong knowledge of big data tool such as Spark, Databricks and Python. Strong understanding of common machine learning techniques and frameworks (e.g. mlflow). Strong knowledge of Natural language processing (NPL) concepts. Strong ...

modern cloud engineering practices. Wherever you land, youll be working with some of the biggest datasets in the UK using everything from Spark and statistical methods to domain knowledge and emerging GenAI applications. The work you could be doing Design and deploy machine learning models for fraud detection, credit … risk, customer segmentation, and behavioural analytics using scalable frameworks like TensorFlow, PyTorch, and XGBoost. Engineer robust data pipelines and ML workflows using Apache Spark, Vertex AI, and CI/CD tooling to ensure seamless model delivery and monitoring. Apply advanced techniques in deep learning, natural language processing ...

repositories. 7. Document data architecture, processes, and workflows for reference and knowledge sharing. 8. Utilise programming languages (e.g., C#, Python, SQL) and technologies (e.g., Apache Spark, SSIS, .NET) to manipulate and analyse data. 9. Participate in code reviews, version control (e.g., using Git), and other software development best ...

compliance. Collaborate with stakeholders and external partners. Skills & Experience: Strong experience with AWS data technologies (e.g., S3, Redshift, Lambda). Proficient in Python, Apache Spark, and SQL. Experience in data warehouse design and data migration projects. Cloud data platform development and deployment. Expertise across data warehouse ...

Owning the end-to-end data lifecycle — from ingestion and transformation through to analytics and data product delivery. Architecting and operating pipelines using Databricks, Spark, and Delta Lake, ensuring performance, reliability, and cost-efficiency. Working closely with BI developers and analysts to deliver dashboards, extracts, datasets, and APIs that … supporting their development. Qualifications Experience leading or mentoring data engineering teams within a SaaS or product-led environment. Deep hands-on knowledge of Databricks, Apache Spark, and Delta Lake, including large-scale or near real-time workloads. Strong proficiency in Python, SQL, and cloud data services (Azure preferred ...

Owning the end-to-end data lifecycle — from ingestion and transformation through to analytics and data product delivery. Architecting and operating pipelines using Databricks, Spark, and Delta Lake, ensuring performance, reliability, and cost-efficiency. Working closely with BI developers and analysts to deliver dashboards, extracts, datasets, and APIs that … supporting their development. Qualifications Experience leading or mentoring data engineering teams within a SaaS or product-led environment. Deep hands-on knowledge of Databricks, Apache Spark, and Delta Lake, including large-scale or near real-time workloads. Strong proficiency in Python, SQL, and cloud data services (Azure preferred ...

focused organization Knowledge of LLMs, embeddings, and modern ML techniques for text, images, and multimodal data Experience working with big data ecosystems (Spark, Ray, Snowflake, BigQuery, etc.) Familiarity with statistical modeling frameworks such as Bayesian methods or probabilistic programming Why Join Gain exposure to cutting-edge AI research workflows ...

focused organization Knowledge of LLMs, embeddings, and modern ML techniques for text, images, and multimodal data Experience working with big data ecosystems (Spark, Ray, Snowflake, BigQuery, etc.) Familiarity with statistical modeling frameworks such as Bayesian methods or probabilistic programming Why Join Gain exposure to cutting-edge AI research workflows ...

focused organization Knowledge of LLMs, embeddings, and modern ML techniques for text, images, and multimodal data Experience working with big data ecosystems (Spark, Ray, Snowflake, BigQuery, etc.) Familiarity with statistical modeling frameworks such as Bayesian methods or probabilistic programming Why Join Gain exposure to cutting-edge AI research workflows ...

focused organization Knowledge of LLMs, embeddings, and modern ML techniques for text, images, and multimodal data Experience working with big data ecosystems (Spark, Ray, Snowflake, BigQuery, etc.) Familiarity with statistical modeling frameworks such as Bayesian methods or probabilistic programming Why Join Gain exposure to cutting-edge AI research workflows ...

analytics tools Familiarity with industry common BI and ETL solutions Hands-on experience in a technical role leveraging SQL, Python, Java and/or Spark to build, operate and maintain data analytics solutions Extensive knowledge of and experience with large-scale database technology (e.g. Snowflake, Netezza, Exadata, Teradata, Greenplum ...

analytics tools Familiarity with industry common BI and ETL solutions Hands-on experience in a technical role leveraging SQL, Python, Java and/or Spark to build, operate and maintain data analytics solutions Extensive knowledge of and experience with large-scale database technology (e.g. Snowflake, Netezza, Exadata, Teradata, Greenplum ...

your specific data industry experience to keep our curriculum cutting-edge. This involves iterating on materials to reflect modern stacks (e.g. cloud data engineering, Apache Airflow, Kafka). You will collaborate with the Curriculum team to design highly effective educational experiences that give learners the AI and Data skills … data modelling, and managing data warehouses (experience with DBT is a plus). Pipelines & Orchestration: Familiarity with building data pipelines, covering batch processing (e.g., Apache Airflow) and an understanding of stream processing (e.g., Kafka/Spark). Data Statistics & Analytics: A strong grasp of statistical concepts (distributions, regression ...

society's experience with software. Come join one of the fastest-growing startups, supported by best-in-class institutions like Battery Ventures, Salesforce Ventures, Spark Capital and Meritech. You will gain experience in a diverse and exciting set of technologies and clients and have a real impact on Pendo ...