1 to 2 of 2 Apache Spark Jobs in Yorkshire

Spark Architect/SME Contract Role- 6 months to begin with & its extendable Location: Leeds, UK (min 3 days onsite) Context: Legacy ETL code for example DataStage is being refactored into PySpark using Prophecy low-code no-code and available converters. Converted code is causing failures/performance issues. … Skills: Spark Architecture – component understanding around Spark Data Integration (PySpark, scripting, variable setting etc.), Spark SQL, Spark Explain plans. Spark SME – Be able to analyse Spark code failures through Spark Plans and make correcting recommendations. Spark SME – Be able to review PySpark … and Spark SQL jobs and make performance improvement recommendations. Spark – SME Be able to understand Data Frames/Resilient Distributed Data Sets and understand any memory related problems and make corrective recommendations. Monitoring – Be able to monitor Spark jobs using wider tools such as Grafana to see more »

transformation, and visualization capabilities. - Strong programming skills in Python, SQL, and other relevant languages. - Experience with big data technologies and tools such as Hadoop, Spark, and Kafka. - Familiarity with cloud platforms (AWS, Azure, GCP) and containerization technologies (Docker, Kubernetes). *Soft Skills:* - Excellent problem-solving and analytical skills. - Strong more »