13 of 13 Apache Spark Jobs in Yorkshire

Azure, or Google Cloud) including data lakes, warehouses, and lakehouse solutions. Transform and process structured and unstructured data using modern ETL/ELT frameworks (Apache Spark, Airflow, dbt). Collaborate closely with product managers, analysts, and software developers to ensure seamless integration and high-quality data availability. Develop ...

years (Senior) of data engineering experience Hands-on experience with Palantir Foundry and/or Foundry Data Platform (FDP) Strong proficiency in: Python SQL Spark (PySpark or SparkSQL) Experience building production data pipelines in cloud environments (AWS, Azure, or GCP) Understanding of data modeling, schema design, and metadata management ...

seeking a Lead Data Engineering Consultant with proven experience in leading and developing data engineering platforms. Experience required: Extensive enterprise experience with Hadoop, Spark, and Splunk. Proficiency in object-oriented and functional scripting, particularly in Python. Skilled in handling raw, structured, semi-structured, and unstructured data (SQL and NoSQL ...

seeking a Lead Data Engineering Consultant with proven experience in leading and developing data engineering platforms. Experience required: Extensive enterprise experience with Hadoop, Spark, and Splunk. Proficiency in object-oriented and functional scripting, particularly in Python. Skilled in handling raw, structured, semi-structured, and unstructured data (SQL and NoSQL ...

leading and developing data engineering platforms. The ideal candidate will possess hands-on expertise in the following areas: Extensive enterprise experience with Hadoop, Spark , and Splunk. Proficiency in object-oriented and functional scripting, particularly in Python . Skilled in handling raw, structured, semi-structured, and unstructured data ...

leading and developing data engineering platforms. The ideal candidate will possess hands-on expertise in the following areas: Extensive enterprise experience with Hadoop, Spark, and Splunk. Proficiency in object-oriented and functional scripting, particularly in Python. Skilled in handling raw, structured, semi-structured, and unstructured data (SQL and NoSQL ...

leading and developing data engineering platforms. The ideal candidate will possess hands-on expertise in the following areas: Extensive enterprise experience with Hadoop, Spark, and Splunk. Proficiency in object-oriented and functional scripting, particularly in Python. Skilled in handling raw, structured, semi-structured, and unstructured data (SQL and NoSQL ...

leading and developing data engineering platforms. The ideal candidate will possess hands-on expertise in the following areas: Extensive enterprise experience with Hadoop, Spark, and Splunk. Proficiency in object-oriented and functional scripting, particularly in Python. Skilled in handling raw, structured, semi-structured, and unstructured data (SQL and NoSQL ...

architectures. Familiarity with document processing pipelines, chunking strategies, and semantic search patterns. Familiarity with data pipeline and orchestration tools (e.g., Airflow, dbt, Spark, Kafka, cloud-native ETL tools) and batch vs. streaming patterns. Understanding of data governance (catalog, lineage, security, RBAC, masking, compliance requirements like GDPR/CCPA). … data science and ML workflows (feature engineering, experimentation, model training/deployment, RAG pipeline development, prompt engineering) and tools/languages such as Python, Spark, notebooks, and ML frameworks (e.g., scikit-learn, MLflow, TensorFlow/PyTorch, LangChain, LlamaIndex at a conceptual level). Consulting Skills Skilled at asking ...

DevOps, with the freedom to shape how GenAI and ML are used across the platform. Tech stack: Python PyTorch or TensorFlow scikit-learn Spark Kafka Databricks LangChain and RAG-based GenAI systems Cloud-native Azure environment Important Details: Fully remote across ...

leading and developing data engineering platforms. The ideal candidate will possess hands-on expertise in the following areas: Extensive enterprise experience with Hadoop, Spark , and Splunk. Proficiency in object-oriented and functional scripting, particularly in Python click apply for full job details ...

/ELT pipelines integrating large, complex datasets Work with structured, semi-structured, and unstructured data across SQL and NoSQL technologies Develop solutions using Hadoop, Spark, and Splunk in large-scale environments Write maintainable Python code, applying object-oriented and functional programming principles Implement and maintain CI/CD pipelines … sustainability, and recoverability Essential Skills & Experience Proven experience as a Lead or Senior Data Engineer in enterprise-scale environments Hands-on expertise with Hadoop, Spark, and Splunk Advanced Python development skills Experience designing and optimising high-performance data pipelines Strong understanding of CI/CD, source control, and automated ...

Title: Lead Data Engineer - Hadoop, Spark, Pytthon Location: Sheffield - 3 days per week in the office Salary/Rate: Up to £617 per day inside IR35 Start Date: 02/03/2026 Job Type: Contract until November We are seeking a Lead Data Engineering Consultant with proven experience … leading and developing data engineering platforms. The ideal candidate will possess hands-on expertise in the following areas: Extensive enterprise experience with Hadoop, Spark, and Splunk. Proficiency in object-oriented and functional scripting, particularly in Python. Skilled in handling raw, structured, semi-structured, and unstructured data (SQL and NoSQL ...