PySpark Jobs with Work from Home Options

PySpark - Spark Python API
UK > Work from Home

The table below provides summary statistics for permanent job vacancies with an option to work from home (WFH) and a requirement for PySpark skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited PySpark over the 6 months to 9 June 2024 with a comparison to the same period in the previous 2 years.

6 months to
9 Jun 2024
Same period 2023 Same period 2022
Rank 404 516 612
Rank change year-on-year +112 +96 -58
Permanent jobs citing PySpark 184 124 281
As % of all permanent jobs with a WFH option 0.52% 0.33% 0.37%
As % of the Libraries, Frameworks & Software Standards category 1.97% 1.00% 0.81%
Number of salaries quoted 127 109 86
10th Percentile £52,500 £52,500 £48,750
25th Percentile £56,875 £65,000 £50,000
Median annual salary (50th Percentile) £70,000 £85,000 £70,000
Median % change year-on-year -17.65% +21.43% +21.74%
75th Percentile £100,000 £108,750 £80,000
90th Percentile £115,000 £160,000 £85,000
UK median annual salary £70,000 £80,000 £70,000
% change year-on-year -12.50% +14.29% +7.69%

All Software Libraries and Frameworks
Work from Home

PySpark is in the Software Libraries and Frameworks category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies with a WFH option and a requirement for technical specification, industry standards, software libraries and framework skills.

Permanent vacancies with a requirement for technical specification, industry standards, software libraries and framework skills 9,345 12,383 34,609
As % of all permanent jobs with a WFH option 26.49% 32.81% 45.84%
Number of salaries quoted 7,699 9,698 17,347
10th Percentile £38,750 £41,250 £38,750
25th Percentile £46,875 £51,250 £46,250
Median annual salary (50th Percentile) £60,000 £65,351 £62,500
Median % change year-on-year -8.19% +4.56% +13.64%
75th Percentile £80,000 £85,000 £82,500
90th Percentile £102,500 £105,000 £95,000
UK median annual salary £60,000 £67,500 £65,000
% change year-on-year -11.11% +3.85% +18.18%

PySpark
Trend for Jobs with a WFH Option

Job vacancies with a work from home option citing PySpark as a proportion of all IT jobs advertised.

PySpark trend for jobs with a WFH option

PySpark
Salary Trend for Jobs with a WFH Option

3-month moving average salary quoted in job vacancies with a work from home option citing PySpark.

PySpark salary trend for jobs with a WFH option

PySpark
Salary Histogram for Jobs with a WFH Option

Salary distribution for jobs with a work from home option citing PySpark over the 6 months to 9 June 2024.

PySpark salary histogram for jobs with a WFH option

PySpark
Co-occurring Skills and Capabilities with WFH Options by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same employment type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 34 (18.48%) Apache Airflow
2 25 (13.59%) Apache
3 23 (12.50%) Apache Spark
4 4 (2.17%) Confluence
4 4 (2.17%) Luigi
Applications
1 7 (3.80%) Microsoft Excel
Business Applications
1 4 (2.17%) Primavera
2 1 (0.54%) SAP ERP
Cloud Services
1 103 (55.98%) Azure
2 44 (23.91%) AWS
3 41 (22.28%) GCP
4 39 (21.20%) Azure Synapse Analytics
5 37 (20.11%) Azure Data Factory
6 27 (14.67%) Power Platform
7 26 (14.13%) Snowflake
8 25 (13.59%) dbt
9 21 (11.41%) Azure DevOps
10 18 (9.78%) Azure Cosmos DB
11 16 (8.70%) GitHub
12 12 (6.52%) Amazon EC2
12 12 (6.52%) Amazon ECS
12 12 (6.52%) Amazon EKS
13 9 (4.89%) Dagster
14 8 (4.35%) Azure AKS
14 8 (4.35%) Azure Machine Learning
14 8 (4.35%) Power Automate
14 8 (4.35%) PowerApps
15 5 (2.72%) AWS Lambda
Communications & Networking
1 2 (1.09%) Network Security
Database & Business Intelligence
1 74 (40.22%) Data Lake
2 45 (24.46%) Tableau
3 39 (21.20%) Data Warehouse
4 27 (14.67%) Power BI
5 26 (14.13%) Big Data
6 22 (11.96%) Amazon Athena
7 21 (11.41%) Azure SQL Database
7 21 (11.41%) Dremio
8 18 (9.78%) SQL Server
9 13 (7.07%) Spark SQL
10 12 (6.52%) Apache Arrow
10 12 (6.52%) Apache Ignite
11 11 (5.98%) Data Mining
11 11 (5.98%) NoSQL
12 10 (5.43%) Delta Lake
13 5 (2.72%) Metadata
13 5 (2.72%) MySQL
13 5 (2.72%) PostgreSQL
14 4 (2.17%) Data Mart
14 4 (2.17%) SQL Server Integration Services
Development Applications
1 58 (31.52%) Git
2 12 (6.52%) MLflow
3 7 (3.80%) Jenkins
4 4 (2.17%) JIRA
5 3 (1.63%) Octopus Deploy
5 3 (1.63%) Team Foundation Server
5 3 (1.63%) TeamCity
6 2 (1.09%) CircleCI
6 2 (1.09%) GitLab
6 2 (1.09%) JUnit
6 2 (1.09%) MSTest
6 2 (1.09%) NUnit
General
1 74 (40.22%) Social Skills
2 39 (21.20%) Finance
3 25 (13.59%) Banking
4 21 (11.41%) Front Office
5 12 (6.52%) Investment Banking
6 11 (5.98%) Analytical Skills
6 11 (5.98%) Legal
7 9 (4.89%) Inclusion and Diversity
8 6 (3.26%) Law
9 5 (2.72%) Influencing Skills
9 5 (2.72%) Publishing
9 5 (2.72%) Retail
10 1 (0.54%) Organisational Skills
10 1 (0.54%) Pharmaceutical
10 1 (0.54%) Public Sector
Job Titles
1 96 (52.17%) Senior
2 94 (51.09%) Data Engineer
3 55 (29.89%) Data Scientist
4 49 (26.63%) Senior Data Scientist
5 40 (21.74%) Senior Data Engineer
6 30 (16.30%) Azure Engineer
7 29 (15.76%) Azure Data Engineer
8 28 (15.22%) Lead
9 23 (12.50%) Python Engineer
10 20 (10.87%) Python Data Engineer
11 13 (7.07%) Lead Data Engineer
12 9 (4.89%) Architect
13 8 (4.35%) Team Leader
14 7 (3.80%) Developer
14 7 (3.80%) Software Engineer
14 7 (3.80%) Technical Leader
15 6 (3.26%) Lead Developer
15 6 (3.26%) Lead Software Engineer
15 6 (3.26%) Software Developer
15 6 (3.26%) Solutions Architect
Libraries, Frameworks & Software Standards
1 76 (41.30%) Pandas
2 45 (24.46%) scikit-learn
2 45 (24.46%) TensorFlow
3 21 (11.41%) NumPy
3 21 (11.41%) SciPy
4 12 (6.52%) gRPC
4 12 (6.52%) Protocol Buffers
5 8 (4.35%) JSON
6 4 (2.17%) AWS CDK
6 4 (2.17%) SQLAlchemy
7 3 (1.63%) SOAP
8 2 (1.09%) MDX
9 1 (0.54%) Kafka
Miscellaneous
1 48 (26.09%) Algorithms
2 12 (6.52%) Derivative
2 12 (6.52%) Equities
2 12 (6.52%) Fixed Income
2 12 (6.52%) Foreign Exchange (FX)
2 12 (6.52%) Hedge funds
3 9 (4.89%) Product Ownership
3 9 (4.89%) Reinsurance
4 7 (3.80%) Data Structures
5 3 (1.63%) Greenfield Project
5 3 (1.63%) Team-Oriented Environment
6 2 (1.09%) Hybrid Cloud
6 2 (1.09%) PropTech
7 1 (0.54%) Blockchain
7 1 (0.54%) Distributed Computing
7 1 (0.54%) Low-Code
7 1 (0.54%) Management Information System
Operating Systems
1 1 (0.54%) Linux
1 1 (0.54%) Ubuntu
Processes & Methodologies
1 82 (44.57%) Machine Learning
2 79 (42.93%) Data Engineering
3 69 (37.50%) Data Science
4 56 (30.43%) Agile
4 56 (30.43%) Analytics
5 54 (29.35%) Artificial Intelligence
5 54 (29.35%) Decision-Making
6 51 (27.72%) Natural Language Processing
6 51 (27.72%) Visualisation
7 50 (27.17%) MLOps
8 49 (26.63%) Mathematics
8 49 (26.63%) Statistics
9 48 (26.09%) Customer Experience
10 46 (25.00%) Predictive Modelling
11 45 (24.46%) Actionable Insight
11 45 (24.46%) Data Munging
11 45 (24.46%) Demand Forecasting
11 45 (24.46%) Propensity Modelling
11 45 (24.46%) Root Cause Analysis
11 45 (24.46%) WFM
Programming Languages
1 160 (86.96%) SQL
2 147 (79.89%) Python
3 21 (11.41%) Java
4 17 (9.24%) T-SQL
5 12 (6.52%) JavaScript
5 12 (6.52%) TypeScript
6 6 (3.26%) Data Analysis Expressions
7 5 (2.72%) C#
7 5 (2.72%) R
8 4 (2.17%) Scala
Qualifications
1 59 (32.07%) Degree
2 48 (26.09%) Master's Degree
3 45 (24.46%) DBS Check
4 11 (5.98%) Computer Science Degree
5 4 (2.17%) CIMA
6 3 (1.63%) PhD
7 2 (1.09%) Microsoft Certification
8 1 (0.54%) Azure Certification
8 1 (0.54%) Postgraduate
8 1 (0.54%) SC Cleared
8 1 (0.54%) Security Cleared
Quality Assurance & Compliance
1 21 (11.41%) Data Quality
2 2 (1.09%) QA
System Software
1 2 (1.09%) Docker
Systems Management
1 18 (9.78%) Terraform
2 14 (7.61%) Kubernetes
Vendors
1 137 (74.46%) Databricks
2 16 (8.70%) Microsoft
3 12 (6.52%) Bloomberg
3 12 (6.52%) Reuters
4 7 (3.80%) Google
5 5 (2.72%) SAP
6 4 (2.17%) Informatica
6 4 (2.17%) SnapLogic
7 2 (1.09%) Oracle
8 1 (0.54%) Marketo
8 1 (0.54%) OpenAI
8 1 (0.54%) Salesforce