1 to 2 of 2 Contract Python Jobs in West Yorkshire

experience with designing, building, and maintaining data pipelines using Apache Spark and Scala. Programming languages: Proficiency in Scala and Spark is essential. Familiarity with Python and SQL is often a plus. Big Data technologies: Understanding of HDFS, Kafka, Hive, and cloud platforms is valuable. Data engineering concepts: Knowledge of data more »

exposed to the latest features and toolsets of Databricks Role Responsibilities: Design and develop scalable data pipelines using Databricks. Implement data processing workflows in Python and SQL. Manage and optimise data storage solutions on AWS. Ensure data quality and integrity across all stages of data processing. Develop and maintain documentation … bundles in Databricks for streamlined data operations. Key Skills: Proven experience as a Senior Data Engineer with a strong focus on Databricks. Proficiency in Python and SQL for data processing and analysis. Spark Python API/PySpark Hands-on experience with AWS services related to data storage and processing (e.g. more »