Hadoop Distributed File System (HDFS)
UK

The table below provides summary statistics for permanent job vacancies requiring HDFS skills. It includes a benchmarking guide to the annual salaries offered in vacancies that cited HDFS over the 6 months leading up to 12 June 2024, comparing them to the same period in the previous two years.

6 months to
12 Jun 2024
Same period 2023 Same period 2022
Rank 801 886 1172
Rank change year-on-year +85 +286 -360
Permanent jobs citing HDFS 103 33 91
As % of all permanent jobs advertised in the UK 0.094% 0.036% 0.053%
As % of the System Software category 0.85% 0.23% 0.29%
Number of salaries quoted 2 24 43
10th Percentile - £48,250 £48,750
25th Percentile £29,000 £50,000 £50,000
Median annual salary (50th Percentile) £31,000 £68,823 £70,000
Median % change year-on-year -54.96% -1.68% +14.29%
75th Percentile £33,000 £82,500 £110,000
90th Percentile - £112,065 £124,000
UK excluding London median annual salary - £68,823 £57,500
% change year-on-year - +19.69% +3.60%

All System Software Skills
UK

HDFS falls under the System Software category. For comparison with the information above, the following table provides summary statistics for all permanent job vacancies requiring system software skills.

Permanent vacancies with a requirement for system software skills 12,139 14,261 31,519
As % of all permanent jobs advertised in the UK 11.05% 15.42% 18.24%
Number of salaries quoted 9,285 9,600 15,562
10th Percentile £26,250 £27,750 £28,000
25th Percentile £32,500 £37,500 £39,750
Median annual salary (50th Percentile) £50,000 £55,000 £56,325
Median % change year-on-year -9.09% -2.35% +2.41%
75th Percentile £67,250 £77,500 £80,000
90th Percentile £85,000 £95,000 £95,000
UK excluding London median annual salary £45,000 £47,500 £50,000
% change year-on-year -5.26% -5.00% +5.26%

HDFS
Job Vacancy Trend

Job postings citing HDFS as a proportion of all IT jobs advertised.

Job vacancy trend for HDFS in the UK

HDFS
Salary Trend

3-month moving average salary quoted in jobs citing HDFS.

Salary trend for HDFS in the UK

HDFS
Top 6 Job Locations

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing HDFS within the UK over the 6 months to 12 June 2024. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Jobs
UK excluding London +23 90 - -
Scotland - 86 - -
England +17 17 £31,000 -54.96% 1
London +89 13 £31,000 -48.33% 1
South East - 4 - -
Work from Home +13 1 - - 1

HDFS
Co-occurring Skills and Capabilities by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same employment type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 99 (96.12%) Apache Spark
2 12 (11.65%) Apache Airflow
3 6 (5.83%) Apache
4 4 (3.88%) Apache Pig
5 1 (0.97%) SAS
Cloud Services
1 103 (100.00%) AWS
2 13 (12.62%) Azure
3 7 (6.80%) Amazon EKS
3 7 (6.80%) GCP
4 6 (5.83%) Serverless
5 5 (4.85%) GitHub
5 5 (4.85%) GitHub Actions
6 4 (3.88%) Amazon EMR
6 4 (3.88%) Amazon S3
7 2 (1.94%) AWS Lambda
7 2 (1.94%) Snowflake
8 1 (0.97%) Azure AKS
8 1 (0.97%) Google Kubernetes Engine
8 1 (0.97%) OpenShift
Database & Business Intelligence
1 101 (98.06%) Apache Hive
2 88 (85.44%) HBase
3 85 (82.52%) NoSQL
4 73 (70.87%) Hadoop
5 44 (42.72%) Big Data
6 7 (6.80%) Data Warehouse
7 5 (4.85%) Data Lake
7 5 (4.85%) Flink
7 5 (4.85%) Parquet
8 4 (3.88%) Impala
9 3 (2.91%) Sqoop
10 2 (1.94%) Amazon Athena
10 2 (1.94%) Amazon Redshift
10 2 (1.94%) PowerCenter
11 1 (0.97%) Apache Cassandra
11 1 (0.97%) Metadata
11 1 (0.97%) MongoDB
11 1 (0.97%) Neo4j
11 1 (0.97%) SAS Data Integration Studio
11 1 (0.97%) SQL Server Integration Services
Development Applications
1 84 (81.55%) Git
1 84 (81.55%) Jenkins
2 79 (76.70%) Bitbucket Server
3 5 (4.85%) CircleCI
4 4 (3.88%) Checkstyle
4 4 (3.88%) JUnit
4 4 (3.88%) SonarQube
5 1 (0.97%) ERwin
General
1 91 (88.35%) Finance
1 91 (88.35%) Inclusion and Diversity
2 86 (83.50%) Law
3 74 (71.84%) Legal
4 12 (11.65%) Marketing
5 8 (7.77%) Banking
6 5 (4.85%) Analytical Skills
6 5 (4.85%) Retail Banking
7 4 (3.88%) Social Skills
8 2 (1.94%) Telecoms
Job Titles
1 86 (83.50%) Software Engineer
2 69 (66.99%) AWS Engineer
3 67 (65.05%) AWS Software Engineer
4 55 (53.40%) Developer
5 49 (47.57%) Database Developer
5 49 (47.57%) Database Engineer
5 49 (47.57%) Developer/Software Engineer
6 37 (35.92%) AWS Database Engineer
6 37 (35.92%) AWS Developer
6 37 (35.92%) Data Engineer
7 30 (29.13%) Big Data Engineer
7 30 (29.13%) Big Data Software Engineer
8 8 (7.77%) Java Engineer
9 7 (6.80%) Java Software Engineer
10 6 (5.83%) Lead
11 5 (4.85%) Python Engineer
11 5 (4.85%) Senior
11 5 (4.85%) Senior Data Engineer
12 4 (3.88%) Lead Developer
12 4 (3.88%) Technical Leader
Libraries, Frameworks & Software Standards
1 12 (11.65%) Kafka
2 11 (10.68%) Spring
3 7 (6.80%) JPA
3 7 (6.80%) React
3 7 (6.80%) REST
4 5 (4.85%) Apache Avro
4 5 (4.85%) Apache Beam
4 5 (4.85%) PySpark
4 5 (4.85%) Spark Streaming
5 4 (3.88%) J2EE
5 4 (3.88%) Spring Boot
6 2 (1.94%) Apache NiFi
6 2 (1.94%) Elastic Stack
Miscellaneous
1 79 (76.70%) Algorithms
2 5 (4.85%) Distributed Applications
2 5 (4.85%) Distributed Systems
2 5 (4.85%) Greenfield Project
3 4 (3.88%) Client/Server
3 4 (3.88%) Product Ownership
3 4 (3.88%) Self-Motivation
4 1 (0.97%) Data Fabric
4 1 (0.97%) Hybrid Cloud
4 1 (0.97%) Private Cloud
Operating Systems
1 86 (83.50%) Unix
2 2 (1.94%) Linux
Processes & Methodologies
1 97 (94.17%) SDLC
2 95 (92.23%) Agile
2 95 (92.23%) CI/CD
3 93 (90.29%) Machine Learning
4 91 (88.35%) Software Engineering
5 88 (85.44%) Artificial Intelligence
6 86 (83.50%) Operational Stability
7 84 (81.55%) Continuous Improvement
7 84 (81.55%) Open Source
8 80 (77.67%) Visualisation
9 79 (76.70%) Batch Processing
9 79 (76.70%) Community of Practice
9 79 (76.70%) TDD
9 79 (76.70%) Test Automation
10 74 (71.84%) Risk Management
11 61 (59.22%) Microservices
12 29 (28.16%) Wealth Management
13 12 (11.65%) ETL
14 10 (9.71%) Data Modelling
15 8 (7.77%) Computer Science
Programming Languages
1 97 (94.17%) SQL
2 96 (93.20%) Python
3 79 (76.70%) PL/SQL
4 70 (67.96%) Java
5 8 (7.77%) Scala
6 2 (1.94%) C#
Qualifications
1 4 (3.88%) Degree
2 2 (1.94%) Computer Science Degree
2 2 (1.94%) Master's Degree
3 1 (0.97%) AWS Certification
3 1 (0.97%) AWS Certified Solutions Architect
Quality Assurance & Compliance
1 4 (3.88%) QA
2 3 (2.91%) Data Quality
System Software
1 3 (2.91%) Docker
Systems Management
1 79 (76.70%) Terraform
2 2 (1.94%) Kubernetes
3 1 (0.97%) Apache Ranger
3 1 (0.97%) Hadoop YARN
Vendors
1 91 (88.35%) Databricks
2 84 (81.55%) Informatica
3 65 (63.11%) Cloudera
4 4 (3.88%) Google
5 3 (2.91%) Microsoft
6 2 (1.94%) Apple
6 2 (1.94%) Facebook
6 2 (1.94%) Intel
7 1 (0.97%) Ab Initio