Apache Spark Jobs

Apache Spark
UK

The following table provides summary statistics for permanent job vacancies with a requirement for Apache Spark skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited Apache Spark over the 6 months to 1 June 2024 with a comparison to the same period in the previous 2 years.

6 months to
1 Jun 2024
Same period 2023 Same period 2022
Rank 302 287 254
Rank change year-on-year -15 -33 +3
Permanent jobs citing Apache Spark 985 1,164 2,800
As % of all permanent jobs advertised in the UK 0.94% 1.23% 1.66%
As % of the Application Platforms category 9.22% 11.01% 11.79%
Number of salaries quoted 277 569 897
10th Percentile £48,400 £60,000 £50,000
25th Percentile £63,750 £67,500 £58,750
Median annual salary (50th Percentile) £80,000 £87,500 £77,500
Median % change year-on-year -8.57% +12.90% +3.33%
75th Percentile £102,500 £110,000 £92,500
90th Percentile £118,750 £150,000 £108,750
UK excluding London median annual salary £69,447 £70,000 £62,500
% change year-on-year -0.79% +12.00% -

All Application Platform Skills
UK

Apache Spark is in the Application Platforms category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies with a requirement for application platform skills.

Permanent vacancies with a requirement for application platform skills 10,683 10,576 23,740
As % of all permanent jobs advertised in the UK 10.18% 11.17% 14.07%
Number of salaries quoted 7,100 7,362 11,563
10th Percentile £27,500 £31,250 £28,750
25th Percentile £34,500 £40,000 £38,188
Median annual salary (50th Percentile) £50,000 £55,000 £55,000
Median % change year-on-year -9.09% - +4.76%
75th Percentile £70,000 £77,500 £73,750
90th Percentile £97,500 £95,000 £92,500
UK excluding London median annual salary £42,500 £47,800 £45,000
% change year-on-year -11.09% +6.22% -

Apache Spark
Job Vacancy Trend

Job postings citing Apache Spark as a proportion of all IT jobs advertised.

Job vacancy trend for Apache Spark in the UK

Apache Spark
Salary Trend

3-month moving average salary quoted in jobs citing Apache Spark.

Salary trend for Apache Spark in the UK

Apache Spark
Salary Histogram

Salary distribution for jobs citing Apache Spark over the 6 months to 1 June 2024.

Salary histogram for Apache Spark in the UK

Apache Spark
Top 17 Job Locations

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing Apache Spark within the UK over the 6 months to 1 June 2024. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Jobs
UK excluding London +207 718 £69,447 -0.79% 56
Scotland +149 585 £47,500 -20.83% 10
England -244 381 £80,000 -10.86% 132
London -136 245 £90,000 -5.26% 90
Work from Home -164 162 £82,500 -17.50% 65
South West -25 51 £80,000 +6.67% 14
North of England -63 43 £55,000 -24.91% 16
North West -5 36 £60,000 -20.00% 12
Midlands -6 16 £72,750 +3.93% 5
South East -30 14 £52,000 -25.71% 7
East Midlands -9 10 £60,000 - 3
West Midlands -16 6 £73,000 +4.29% 2
Yorkshire -33 6 £51,528 -25.13% 3
East of England -6 5 £101,250 +102.50% 5
Wales +11 3 £72,500 +38.10%
Northern Ireland +20 1 £70,000 -30.00% 1
North East - 1 £75,000 - 1

Apache Spark
Co-occurring Skills and Capabilities by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same employment type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 288 (29.24%) Apache
2 168 (17.06%) Apache Airflow
3 26 (2.64%) Jupyter
4 16 (1.62%) nginx
5 8 (0.81%) Confluence
6 5 (0.51%) Apache Pulsar
6 5 (0.51%) SAS
7 2 (0.20%) Luigi
8 1 (0.10%) Apache Pig
8 1 (0.10%) Apache Solr
Applications
1 7 (0.71%) Microsoft Excel
2 4 (0.41%) Spreadsheet
3 1 (0.10%) Microsoft Office
Business Applications
1 4 (0.41%) Primavera
Cloud Services
1 673 (68.32%) AWS
2 214 (21.73%) Azure
3 172 (17.46%) GCP
4 104 (10.56%) Serverless
5 98 (9.95%) Amazon EKS
6 96 (9.75%) AWS Lambda
7 72 (7.31%) Amazon EMR
8 70 (7.11%) Amazon SQS
9 60 (6.09%) Amazon S3
10 56 (5.69%) Amazon ECS
11 54 (5.48%) Amazon EC2
12 47 (4.77%) OpenShift
12 47 (4.77%) Snowflake
13 44 (4.47%) AWS CloudFormation
14 40 (4.06%) Power Platform
15 38 (3.86%) Azure Data Factory
16 37 (3.76%) Azure Synapse Analytics
17 26 (2.64%) Amazon Kinesis
17 26 (2.64%) GitHub
18 24 (2.44%) Azure DevOps
Communications & Networking
1 29 (2.94%) AMQP
2 16 (1.62%) DSL
3 14 (1.42%) Wireless
4 5 (0.51%) Network Security
5 4 (0.41%) Firewall
6 2 (0.20%) HTTP
Database & Business Intelligence
1 686 (69.64%) Big Data
2 406 (41.22%) NoSQL
3 400 (40.61%) Hadoop
4 156 (15.84%) Apache Hive
5 101 (10.25%) Data Lake
5 101 (10.25%) HBase
6 87 (8.83%) Data Warehouse
7 54 (5.48%) Electronic Patient Records
8 46 (4.67%) Data Mining
9 45 (4.57%) Flink
10 44 (4.47%) Amazon Redshift
11 43 (4.37%) Elasticsearch
12 42 (4.26%) SQL Server
13 38 (3.86%) Power BI
14 37 (3.76%) Tableau
15 35 (3.55%) BigQuery
16 30 (3.05%) Presto
17 29 (2.94%) Cloudera CDH
18 27 (2.74%) Relational Database
19 22 (2.23%) MongoDB
Development Applications
1 135 (13.71%) Jenkins
2 127 (12.89%) Git
3 84 (8.53%) Bitbucket Server
4 16 (1.62%) Atlassian Bamboo
5 12 (1.22%) Spinnaker
6 9 (0.91%) JIRA
7 8 (0.81%) Oracle Forms
7 8 (0.81%) Visual Studio
8 7 (0.71%) CircleCI
9 5 (0.51%) MLflow
10 4 (0.41%) Bitbucket
11 3 (0.30%) Deequ
11 3 (0.30%) Octopus Deploy
12 2 (0.20%) Team Foundation Server
13 1 (0.10%) Ant
13 1 (0.10%) ERwin
13 1 (0.10%) GitLab
13 1 (0.10%) Gradle
13 1 (0.10%) JUnit
13 1 (0.10%) Maven
General
1 633 (64.26%) Finance
2 603 (61.22%) Inclusion and Diversity
3 574 (58.27%) Law
4 312 (31.68%) Marketing
5 270 (27.41%) Legal
6 159 (16.14%) Influencing Skills
7 111 (11.27%) Social Skills
8 90 (9.14%) Analytical Skills
9 68 (6.90%) Financial Institution
10 57 (5.79%) Banking
11 29 (2.94%) Aerospace
12 21 (2.13%) Retail
13 20 (2.03%) Investment Banking
14 19 (1.93%) Presentation Skills
15 18 (1.83%) Games
16 14 (1.42%) Electronics
17 12 (1.22%) Public Sector
18 11 (1.12%) Private Banking
19 7 (0.71%) Retail Banking
20 4 (0.41%) Music Industry
Job Titles
1 575 (58.38%) Software Engineer
2 382 (38.78%) Big Data Engineer
3 365 (37.06%) Big Data Software Engineer
4 327 (33.20%) Data Engineer
5 224 (22.74%) Lead
6 218 (22.13%) Senior
7 212 (21.52%) Scala Engineer
7 212 (21.52%) Scala Software Engineer
8 188 (19.09%) Java Engineer
9 187 (18.98%) Java Software Engineer
10 185 (18.78%) AWS Engineer
11 183 (18.58%) Lead Software Engineer
12 173 (17.56%) AWS Software Engineer
13 144 (14.62%) Lead Big Data Engineer
14 132 (13.40%) Python Engineer
15 119 (12.08%) Java Scala Software Engineer
16 111 (11.27%) Python Software Engineer
17 102 (10.36%) Senior Software Engineer
18 97 (9.85%) Lead Java Engineer
19 86 (8.73%) Developer
Libraries, Frameworks & Software Standards
1 203 (20.61%) Kafka
2 170 (17.26%) PySpark
3 112 (11.37%) Spring
4 87 (8.83%) Spring Boot
5 65 (6.60%) React
6 52 (5.28%) RabbitMQ
7 51 (5.18%) AngularJS
8 47 (4.77%) REST
9 33 (3.35%) Node.js
10 31 (3.15%) Apache Beam
11 30 (3.05%) gRPC
11 30 (3.05%) OpenAPI
11 30 (3.05%) Swagger
12 29 (2.94%) Material-UI
13 23 (2.34%) Pandas
14 21 (2.13%) ActiveMQ
14 21 (2.13%) Apache NiFi
15 19 (1.93%) Elastic Stack
16 17 (1.73%) TensorFlow
17 16 (1.62%) YAML
Miscellaneous
1 240 (24.37%) Algorithms
2 224 (22.74%) Cloud Native
3 59 (5.99%) Data Structures
4 41 (4.16%) Legacy Code
5 38 (3.86%) Management Information System
6 31 (3.15%) Public Cloud
7 30 (3.05%) Distributed Systems
8 25 (2.54%) Product Ownership
9 18 (1.83%) Hybrid Cloud
10 15 (1.52%) Greenfield Project
11 13 (1.32%) Legacy Systems
12 11 (1.12%) Enterprise Software
13 9 (0.91%) Distributed Applications
14 8 (0.81%) BBC
15 7 (0.71%) Distributed Computing
16 6 (0.61%) Mainframe
16 6 (0.61%) Onboarding
17 4 (0.41%) IoT
17 4 (0.41%) Renewable Energy
18 3 (0.30%) Analytical Mindset
Operating Systems
1 101 (10.25%) Unix
2 68 (6.90%) Linux
3 29 (2.94%) CentOS
3 29 (2.94%) Debian
3 29 (2.94%) Red Hat Enterprise Linux
3 29 (2.94%) Ubuntu
Processes & Methodologies
1 697 (70.76%) Agile
1 697 (70.76%) Software Engineering
2 601 (61.02%) CI/CD
3 535 (54.31%) SDLC
4 477 (48.43%) Operational Stability
5 380 (38.58%) Machine Learning
6 320 (32.49%) Artificial Intelligence
7 286 (29.04%) Risk Management
8 263 (26.70%) Visualisation
9 255 (25.89%) Community of Practice
10 223 (22.64%) Performance Tuning
11 207 (21.02%) Continuous Improvement
12 202 (20.51%) Problem-Solving
13 196 (19.90%) Microservices
14 176 (17.87%) Data Engineering
15 172 (17.46%) Decision-Making
16 158 (16.04%) Data Modelling
17 152 (15.43%) Data Pipeline
18 138 (14.01%) Open Source
19 137 (13.91%) ETL
Programming Languages
1 599 (60.81%) SQL
2 586 (59.49%) Java
3 567 (57.56%) Python
4 381 (38.68%) Scala
5 101 (10.25%) PL/SQL
6 41 (4.16%) Go
7 35 (3.55%) JavaScript
8 31 (3.15%) Bash
8 31 (3.15%) Kotlin
9 25 (2.54%) C
10 23 (2.34%) R
11 22 (2.23%) C#
12 20 (2.03%) Shell Script
13 18 (1.83%) C++
14 12 (1.22%) TypeScript
15 8 (0.81%) Clojure
15 8 (0.81%) Data Analysis Expressions
15 8 (0.81%) MATLAB
16 6 (0.61%) Rust
17 4 (0.41%) VB
Qualifications
1 123 (12.49%) Degree
2 57 (5.79%) Security Cleared
3 51 (5.18%) DV Cleared
4 38 (3.86%) SC Cleared
5 34 (3.45%) Computer Science Degree
6 21 (2.13%) Master's Degree
7 11 (1.12%) Microsoft Certification
7 11 (1.12%) PhD
8 6 (0.61%) Azure Certification
8 6 (0.61%) BPSS Clearance
8 6 (0.61%) DBS Check
9 4 (0.41%) CIMA
9 4 (0.41%) Numerate Degree
10 3 (0.30%) ISTQB
11 1 (0.10%) AWS Certification
11 1 (0.10%) AWS Certified Solutions Architect
Quality Assurance & Compliance
1 71 (7.21%) Data Quality
2 32 (3.25%) SLA
3 20 (2.03%) QA
4 4 (0.41%) Accessibility
4 4 (0.41%) Solvency II
5 3 (0.30%) CRD IV
5 3 (0.30%) IFRS
5 3 (0.30%) IFRS 9
6 2 (0.20%) GDPR
System Software
1 112 (11.37%) HDFS
2 66 (6.70%) Docker
3 17 (1.73%) Virtual Machines
4 6 (0.61%) Ceph
5 3 (0.30%) Virtual Servers
6 2 (0.20%) IAG
7 1 (0.10%) Apache Flume
7 1 (0.10%) Slurm Workload Manager
Systems Management
1 196 (19.90%) Kubernetes
2 186 (18.88%) Terraform
3 66 (6.70%) Ansible
4 33 (3.35%) Puppet
5 17 (1.73%) Prometheus
6 16 (1.62%) Grafana
6 16 (1.62%) HAProxy
7 6 (0.61%) Knative
8 4 (0.41%) Kibana
9 3 (0.30%) Progress Chef
10 2 (0.20%) Argo
10 2 (0.20%) Kubeflow
11 1 (0.10%) logstash
11 1 (0.10%) Podman
Vendors
1 253 (25.69%) Databricks
2 110 (11.17%) Google
3 92 (9.34%) Informatica
4 85 (8.63%) Cloudera
5 65 (6.60%) Microsoft
6 32 (3.25%) Oracle
7 29 (2.94%) Red Hat
7 29 (2.94%) SaltStack
8 18 (1.83%) Cognos
9 17 (1.73%) Alteryx
10 16 (1.62%) Sybase
10 16 (1.62%) Teradata
11 14 (1.42%) Dell
12 6 (0.61%) YouTube
13 4 (0.41%) Apple
13 4 (0.41%) Atlassian
13 4 (0.41%) Confluent
13 4 (0.41%) Hortonworks
13 4 (0.41%) SAP
13 4 (0.41%) Splunk