Apache Spark Jobs

Apache Spark
UK

The following table provides summary statistics for permanent job vacancies with a requirement for Apache Spark skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited Apache Spark over the 6 months to 5 June 2024 with a comparison to the same period in the previous 2 years.

6 months to
5 Jun 2024
Same period 2023 Same period 2022
Rank 301 298 253
Rank change year-on-year -3 -45 +1
Permanent jobs citing Apache Spark 970 1,121 2,815
As % of all permanent jobs advertised in the UK 0.91% 1.20% 1.65%
As % of the Application Platforms category 9.05% 10.74% 11.79%
Number of salaries quoted 277 559 897
10th Percentile £48,400 £60,000 £50,000
25th Percentile £63,750 £67,500 £58,000
Median annual salary (50th Percentile) £80,000 £87,500 £77,000
Median % change year-on-year -8.57% +13.64% +2.67%
75th Percentile £102,500 £110,000 £92,500
90th Percentile £118,750 £150,000 £108,750
UK excluding London median annual salary £69,724 £70,000 £62,500
% change year-on-year -0.40% +12.00% -

All Application Platform Skills
UK

Apache Spark is in the Application Platforms category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies with a requirement for application platform skills.

Permanent vacancies with a requirement for application platform skills 10,716 10,439 23,877
As % of all permanent jobs advertised in the UK 10.07% 11.18% 14.03%
Number of salaries quoted 7,104 7,282 11,567
10th Percentile £27,500 £31,250 £28,750
25th Percentile £35,000 £40,000 £38,500
Median annual salary (50th Percentile) £50,000 £55,000 £55,000
Median % change year-on-year -9.09% - +4.76%
75th Percentile £70,000 £77,500 £73,750
90th Percentile £98,474 £95,000 £92,500
UK excluding London median annual salary £42,500 £48,500 £45,000
% change year-on-year -12.37% +7.78% -

Apache Spark
Job Vacancy Trend

Job postings citing Apache Spark as a proportion of all IT jobs advertised.

Job vacancy trend for Apache Spark in the UK

Apache Spark
Salary Trend

3-month moving average salary quoted in jobs citing Apache Spark.

Salary trend for Apache Spark in the UK

Apache Spark
Salary Histogram

Salary distribution for jobs citing Apache Spark over the 6 months to 5 June 2024.

Salary histogram for Apache Spark in the UK

Apache Spark
Top 17 Job Locations

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing Apache Spark within the UK over the 6 months to 5 June 2024. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Jobs
UK excluding London +216 704 £69,724 -0.40% 49
Scotland +152 573 £47,500 -20.83% 5
England -228 378 £80,000 -9.86% 122
London -141 244 £90,000 -5.26% 84
Work from Home -156 162 £82,500 -17.50% 67
South West -35 51 £80,000 +6.67% 13
North of England -61 41 £60,000 -19.46% 16
North West -5 36 £60,000 -20.00% 12
Midlands -8 16 £72,750 +3.93% 5
South East -29 14 £52,000 -25.71% 6
East Midlands -12 10 £60,000 - 3
West Midlands -20 6 £73,000 +4.29% 2
East of England -20 5 £101,250 +102.50% 6
Yorkshire -14 4 £53,264 -22.61% 4
Wales +6 3 £72,500 +38.10%
Northern Ireland +20 1 £70,000 -30.00% 1
North East - 1 £75,000 -

Apache Spark
Co-occurring Skills and Capabilities by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same employment type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 284 (29.28%) Apache
2 168 (17.32%) Apache Airflow
3 26 (2.68%) Jupyter
4 16 (1.65%) nginx
5 8 (0.82%) Confluence
6 5 (0.52%) Apache Pulsar
7 3 (0.31%) SAS
8 2 (0.21%) Luigi
9 1 (0.10%) Apache Pig
9 1 (0.10%) Apache Solr
Applications
1 7 (0.72%) Microsoft Excel
2 4 (0.41%) Spreadsheet
3 1 (0.10%) Microsoft Office
Business Applications
1 4 (0.41%) Primavera
Cloud Services
1 664 (68.45%) AWS
2 213 (21.96%) Azure
3 174 (17.94%) GCP
4 103 (10.62%) Serverless
5 96 (9.90%) Amazon EKS
6 95 (9.79%) AWS Lambda
7 71 (7.32%) Amazon EMR
8 69 (7.11%) Amazon SQS
9 60 (6.19%) Amazon S3
10 55 (5.67%) Amazon ECS
11 53 (5.46%) Amazon EC2
12 49 (5.05%) Snowflake
13 47 (4.85%) OpenShift
14 44 (4.54%) AWS CloudFormation
15 40 (4.12%) Power Platform
16 39 (4.02%) Azure Synapse Analytics
17 38 (3.92%) Azure Data Factory
18 28 (2.89%) Amazon Kinesis
19 25 (2.58%) GitHub
20 24 (2.47%) Azure DevOps
Communications & Networking
1 29 (2.99%) AMQP
2 15 (1.55%) DSL
3 12 (1.24%) Wireless
4 5 (0.52%) Network Security
5 4 (0.41%) Firewall
6 2 (0.21%) HTTP
Database & Business Intelligence
1 675 (69.59%) Big Data
2 399 (41.13%) NoSQL
3 395 (40.72%) Hadoop
4 152 (15.67%) Apache Hive
5 99 (10.21%) HBase
6 97 (10.00%) Data Lake
7 84 (8.66%) Data Warehouse
8 53 (5.46%) Electronic Patient Records
9 48 (4.95%) Data Mining
10 46 (4.74%) Amazon Redshift
11 44 (4.54%) Flink
12 43 (4.43%) Elasticsearch
13 42 (4.33%) SQL Server
14 38 (3.92%) BigQuery
14 38 (3.92%) Power BI
15 36 (3.71%) Tableau
16 30 (3.09%) Presto
17 29 (2.99%) Cloudera CDH
18 26 (2.68%) Relational Database
19 22 (2.27%) PostgreSQL
Development Applications
1 135 (13.92%) Jenkins
2 125 (12.89%) Git
3 83 (8.56%) Bitbucket Server
4 16 (1.65%) Atlassian Bamboo
5 12 (1.24%) Spinnaker
6 9 (0.93%) JIRA
7 8 (0.82%) Oracle Forms
7 8 (0.82%) Visual Studio
8 6 (0.62%) CircleCI
9 5 (0.52%) MLflow
10 4 (0.41%) Bitbucket
11 3 (0.31%) Deequ
11 3 (0.31%) Octopus Deploy
12 2 (0.21%) Team Foundation Server
13 1 (0.10%) ERwin
13 1 (0.10%) GitLab
General
1 616 (63.51%) Finance
2 590 (60.82%) Inclusion and Diversity
3 562 (57.94%) Law
4 305 (31.44%) Marketing
5 266 (27.42%) Legal
6 156 (16.08%) Influencing Skills
7 110 (11.34%) Social Skills
8 86 (8.87%) Analytical Skills
9 66 (6.80%) Financial Institution
10 52 (5.36%) Banking
11 29 (2.99%) Aerospace
12 19 (1.96%) Investment Banking
12 19 (1.96%) Retail
13 18 (1.86%) Games
14 17 (1.75%) Presentation Skills
15 14 (1.44%) Electronics
16 12 (1.24%) Public Sector
17 10 (1.03%) Private Banking
18 6 (0.62%) Retail Banking
19 4 (0.41%) Music Industry
Job Titles
1 567 (58.45%) Software Engineer
2 377 (38.87%) Big Data Engineer
3 360 (37.11%) Big Data Software Engineer
4 322 (33.20%) Data Engineer
5 221 (22.78%) Lead
6 215 (22.16%) Senior
7 208 (21.44%) Scala Engineer
7 208 (21.44%) Scala Software Engineer
8 183 (18.87%) Java Engineer
9 182 (18.76%) Java Software Engineer
10 181 (18.66%) AWS Engineer
11 180 (18.56%) Lead Software Engineer
12 169 (17.42%) AWS Software Engineer
13 142 (14.64%) Lead Big Data Engineer
14 131 (13.51%) Python Engineer
15 117 (12.06%) Java Scala Software Engineer
16 110 (11.34%) Python Software Engineer
17 101 (10.41%) Senior Software Engineer
18 97 (10.00%) Lead Java Engineer
19 84 (8.66%) Developer
Libraries, Frameworks & Software Standards
1 204 (21.03%) Kafka
2 168 (17.32%) PySpark
3 108 (11.13%) Spring
4 85 (8.76%) Spring Boot
5 64 (6.60%) React
6 52 (5.36%) RabbitMQ
7 49 (5.05%) AngularJS
8 46 (4.74%) REST
9 33 (3.40%) Node.js
10 32 (3.30%) Apache Beam
11 30 (3.09%) gRPC
11 30 (3.09%) OpenAPI
11 30 (3.09%) Swagger
12 29 (2.99%) Material-UI
13 22 (2.27%) Pandas
14 21 (2.16%) ActiveMQ
14 21 (2.16%) Apache NiFi
15 19 (1.96%) Elastic Stack
16 17 (1.75%) TensorFlow
17 16 (1.65%) YAML
Miscellaneous
1 235 (24.23%) Algorithms
2 220 (22.68%) Cloud Native
3 57 (5.88%) Data Structures
4 40 (4.12%) Legacy Code
5 36 (3.71%) Management Information System
6 30 (3.09%) Public Cloud
7 29 (2.99%) Distributed Systems
8 24 (2.47%) Product Ownership
9 18 (1.86%) Hybrid Cloud
10 14 (1.44%) Greenfield Project
10 14 (1.44%) Legacy Systems
11 10 (1.03%) Enterprise Software
12 8 (0.82%) BBC
12 8 (0.82%) Distributed Applications
13 7 (0.72%) Distributed Computing
14 6 (0.62%) Mainframe
14 6 (0.62%) Onboarding
15 4 (0.41%) IoT
15 4 (0.41%) Renewable Energy
16 3 (0.31%) Virtual Team
Operating Systems
1 99 (10.21%) Unix
2 66 (6.80%) Linux
3 29 (2.99%) CentOS
3 29 (2.99%) Debian
3 29 (2.99%) Red Hat Enterprise Linux
3 29 (2.99%) Ubuntu
Processes & Methodologies
1 685 (70.62%) Software Engineering
2 683 (70.41%) Agile
3 591 (60.93%) CI/CD
4 526 (54.23%) SDLC
5 467 (48.14%) Operational Stability
6 372 (38.35%) Machine Learning
7 312 (32.16%) Artificial Intelligence
8 279 (28.76%) Risk Management
9 260 (26.80%) Visualisation
10 250 (25.77%) Community of Practice
11 218 (22.47%) Performance Tuning
12 201 (20.72%) Continuous Improvement
13 197 (20.31%) Problem-Solving
14 193 (19.90%) Microservices
15 174 (17.94%) Data Engineering
16 167 (17.22%) Decision-Making
17 156 (16.08%) Data Modelling
18 152 (15.67%) Data Pipeline
19 137 (14.12%) ETL
19 137 (14.12%) Open Source
Programming Languages
1 591 (60.93%) SQL
2 573 (59.07%) Java
3 557 (57.42%) Python
4 374 (38.56%) Scala
5 99 (10.21%) PL/SQL
6 41 (4.23%) Go
7 35 (3.61%) JavaScript
8 32 (3.30%) Kotlin
9 31 (3.20%) Bash
10 24 (2.47%) C
11 21 (2.16%) C#
11 21 (2.16%) R
12 19 (1.96%) Shell Script
13 17 (1.75%) C++
14 13 (1.34%) TypeScript
15 9 (0.93%) Clojure
16 8 (0.82%) Data Analysis Expressions
16 8 (0.82%) MATLAB
17 6 (0.62%) Rust
18 3 (0.31%) VB
Qualifications
1 119 (12.27%) Degree
2 57 (5.88%) Security Cleared
3 51 (5.26%) DV Cleared
4 38 (3.92%) SC Cleared
5 33 (3.40%) Computer Science Degree
6 20 (2.06%) Master's Degree
7 11 (1.13%) Microsoft Certification
7 11 (1.13%) PhD
8 6 (0.62%) Azure Certification
8 6 (0.62%) BPSS Clearance
8 6 (0.62%) DBS Check
9 4 (0.41%) CIMA
9 4 (0.41%) Numerate Degree
10 3 (0.31%) ISTQB
11 1 (0.10%) AWS Certification
11 1 (0.10%) AWS Certified Solutions Architect
Quality Assurance & Compliance
1 70 (7.22%) Data Quality
2 31 (3.20%) SLA
3 20 (2.06%) QA
4 4 (0.41%) Accessibility
4 4 (0.41%) Solvency II
5 2 (0.21%) GDPR
6 1 (0.10%) CRD IV
6 1 (0.10%) IFRS
6 1 (0.10%) IFRS 9
System Software
1 109 (11.24%) HDFS
2 65 (6.70%) Docker
3 17 (1.75%) Virtual Machines
4 6 (0.62%) Ceph
5 3 (0.31%) Virtual Servers
6 2 (0.21%) IAG
7 1 (0.10%) Apache Flume
7 1 (0.10%) Slurm Workload Manager
Systems Management
1 196 (20.21%) Kubernetes
2 187 (19.28%) Terraform
3 66 (6.80%) Ansible
4 33 (3.40%) Puppet
5 17 (1.75%) Prometheus
6 16 (1.65%) Grafana
6 16 (1.65%) HAProxy
7 6 (0.62%) Knative
8 4 (0.41%) Kibana
9 3 (0.31%) Progress Chef
10 2 (0.21%) Argo
10 2 (0.21%) Kubeflow
11 1 (0.10%) logstash
11 1 (0.10%) Podman
Vendors
1 248 (25.57%) Databricks
2 110 (11.34%) Google
3 91 (9.38%) Informatica
4 82 (8.45%) Cloudera
5 65 (6.70%) Microsoft
6 31 (3.20%) Oracle
7 29 (2.99%) Red Hat
7 29 (2.99%) SaltStack
8 17 (1.75%) Cognos
9 16 (1.65%) Alteryx
9 16 (1.65%) Teradata
10 15 (1.55%) Sybase
11 14 (1.44%) Dell
12 6 (0.62%) YouTube
13 4 (0.41%) Apple
13 4 (0.41%) Confluent
13 4 (0.41%) Hortonworks
13 4 (0.41%) SAP
13 4 (0.41%) Splunk
14 3 (0.31%) Facebook