Data Lakes
UK

The following table provides summary statistics for permanent job vacancies with a requirement for Data Lake skills. Included is a benchmarking guide to the salaries offered in vacancies that have cited Data Lake over the 6 months to 20 May 2024 with a comparison to the same period in the previous 2 years.

6 months to
20 May 2024
Same period 2023 Same period 2022
Rank 262 247 300
Rank change year-on-year -15 +53 +68
Permanent jobs citing Data Lake 1,078 1,508 2,259
As % of all permanent jobs advertised in the UK 1.08% 1.52% 1.39%
As % of the Database & Business Intelligence category 5.90% 7.05% 4.95%
Number of salaries quoted 820 1,019 1,129
10th Percentile £42,500 £50,000 £46,250
25th Percentile £59,750 £58,750 £54,000
Median annual salary (50th Percentile) £85,000 £80,000 £70,000
Median % change year-on-year +6.25% +14.29% -
75th Percentile £122,500 £100,000 £87,500
90th Percentile £148,750 £130,000 £102,500
UK excluding London median annual salary £60,000 £60,000 £60,000

All Database and Business Intelligence Skills
UK

Data Lake is in the Databases and Business Intelligence category. The following table is for comparison with the above and provides summary statistics for all permanent job vacancies with a requirement for database or business intelligence skills.

Permanent vacancies with a requirement for database or business intelligence skills 18,271 21,401 45,622
As % of all permanent jobs advertised in the UK 18.29% 21.52% 28.02%
Number of salaries quoted 12,519 13,364 21,166
10th Percentile £35,000 £37,500 £36,250
25th Percentile £42,725 £47,500 £45,000
Median annual salary (50th Percentile) £57,500 £65,000 £60,000
Median % change year-on-year -11.54% +8.33% +9.09%
75th Percentile £77,500 £83,250 £80,000
90th Percentile £97,500 £103,750 £95,000
UK excluding London median annual salary £52,500 £55,000 £52,500
% change year-on-year -4.55% +4.76% +10.53%

Data Lake
Job Vacancy Trend

Job postings citing Data Lake as a proportion of all IT jobs advertised.

Job vacancy trend for Data Lake in the UK

Data Lake
Salary Trend

3-month moving average salary quoted in jobs citing Data Lake.

Salary trend for Data Lake in the UK

Data Lake
Salary Histogram

Salary distribution for jobs citing Data Lake over the 6 months to 20 May 2024.

Salary histogram for Data Lake in the UK

Data Lake
Top 18 Job Locations

The table below looks at the demand and provides a guide to the median salaries quoted in IT jobs citing Data Lake within the UK over the 6 months to 20 May 2024. The 'Rank Change' column provides an indication of the change in demand within each location based on the same 6 month period last year.

Location Rank Change
on Same Period
Last Year
Matching
Permanent
IT Job Ads
Median Salary
Past 6 Months
Median Salary
% Change
on Same Period
Last Year
Live
Jobs
England -15 934 £87,500 +9.38% 231
Work from Home +43 636 £100,000 +24.54% 118
London +18 549 £112,500 +25.00% 140
UK excluding London -71 471 £60,000 - 112
North of England -9 125 £55,000 -4.35% 27
South East -34 99 £63,500 -9.29% 41
Scotland -41 85 £66,500 +2.31% 5
Midlands -16 76 £57,500 +7.98% 14
North West -23 69 £45,000 -18.18% 19
South West -63 55 £70,576 +4.56% 15
West Midlands +11 50 £57,500 +7.98% 10
Yorkshire +51 40 £67,500 +3.85% 7
East Midlands -26 26 £47,500 -13.64% 4
North East -1 16 £75,000 +36.36% 1
East of England -12 15 £57,500 - 7
Wales -2 8 £46,131 -20.46% 1
Northern Ireland +23 6 - - 3
Isle of Man - 2 £82,500 -

Data Lake
Co-occurring Skills and Capabilities by Category

The follow tables expand on the table above by listing co-occurrences grouped by category. The same employment type, locality and period is covered with up to 20 co-occurrences shown in each of the following categories:

Application Platforms
1 292 (27.09%) Apache Airflow
2 255 (23.65%) Apache
3 108 (10.02%) Apache Spark
4 15 (1.39%) Confluence
5 7 (0.65%) SharePoint
6 5 (0.46%) Microsoft Exchange
7 2 (0.19%) IBM Domino
7 2 (0.19%) Jupyter
8 1 (0.093%) Apache Pig
8 1 (0.093%) Apache Synapse
8 1 (0.093%) IIS
8 1 (0.093%) SAS
Applications
1 19 (1.76%) Microsoft Excel
2 18 (1.67%) Microsoft Office
3 5 (0.46%) Microsoft PowerPoint
4 1 (0.093%) MS Visio
Business Applications
1 21 (1.95%) Dynamics CRM
2 9 (0.83%) Dynamics NAV
3 6 (0.56%) Dynamics AX
3 6 (0.56%) Primavera
4 3 (0.28%) SAP S/4HANA
5 2 (0.19%) Magento
6 1 (0.093%) HFM
6 1 (0.093%) Salesforce CRM
Cloud Services
1 636 (59.00%) Azure
2 540 (50.09%) AWS
3 357 (33.12%) Snowflake
4 288 (26.72%) Power Platform
5 287 (26.62%) dbt
6 269 (24.95%) GCP
7 192 (17.81%) Azure Data Factory
8 155 (14.38%) Azure Synapse Analytics
9 57 (5.29%) Azure Cosmos DB
10 55 (5.10%) Matillion
11 53 (4.92%) Azure DevOps
12 45 (4.17%) PowerApps
12 45 (4.17%) Serverless
13 42 (3.90%) Dynamics 365
14 31 (2.88%) SaaS
15 27 (2.50%) Power Automate
16 23 (2.13%) GitHub
16 23 (2.13%) PaaS
17 22 (2.04%) Cloud Computing
17 22 (2.04%) IaaS
Communications & Networking
1 18 (1.67%) DSL
1 18 (1.67%) Wireless
2 11 (1.02%) Network Security
3 5 (0.46%) DNS
4 4 (0.37%) Firewall
5 3 (0.28%) NetScaler
6 1 (0.093%) HTTP
6 1 (0.093%) SAN
Database & Business Intelligence
1 405 (37.57%) Data Warehouse
2 279 (25.88%) Dremio
2 279 (25.88%) Power BI
3 276 (25.60%) Apache Arrow
4 273 (25.32%) Amazon Athena
5 231 (21.43%) Apache Ignite
6 192 (17.81%) Big Data
7 175 (16.23%) SQL Server
8 77 (7.14%) Azure SQL Database
9 75 (6.96%) Tableau
10 73 (6.77%) NoSQL
11 68 (6.31%) Azure SQL Data Warehouse
12 65 (6.03%) Metadata
13 61 (5.66%) SQL Server Integration Services
14 50 (4.64%) Star Schema
15 46 (4.27%) Qlik Sense
16 44 (4.08%) Elasticsearch
17 43 (3.99%) Hadoop
18 41 (3.80%) Relational Database
19 37 (3.43%) SQL Server Analysis Services
Development Applications
1 39 (3.62%) GitLab
2 25 (2.32%) JIRA
3 23 (2.13%) Git
4 17 (1.58%) Jenkins
5 16 (1.48%) CircleCI
6 10 (0.93%) MLflow
6 10 (0.93%) Selenium
7 9 (0.83%) JUnit
7 9 (0.83%) Oracle Forms
8 8 (0.74%) Cucumber
8 8 (0.74%) SQL Server Management Studio (SSMS)
9 6 (0.56%) RStudio
9 6 (0.56%) Visual Studio
10 4 (0.37%) Zephyr
11 3 (0.28%) ER Studio
11 3 (0.28%) ERwin
11 3 (0.28%) Octopus Deploy
12 2 (0.19%) Bitbucket
13 1 (0.093%) Gradle
13 1 (0.093%) Visual Studio Team System
General
1 563 (52.23%) Finance
2 304 (28.20%) Banking
3 243 (22.54%) Investment Banking
4 222 (20.59%) Front Office
5 177 (16.42%) Social Skills
6 145 (13.45%) Analytical Skills
7 101 (9.37%) Inclusion and Diversity
8 88 (8.16%) Law
9 67 (6.22%) Legal
10 56 (5.19%) Marketing
11 53 (4.92%) Presentation Skills
12 36 (3.34%) Retail
13 34 (3.15%) Financial Institution
14 23 (2.13%) Retail Banking
15 18 (1.67%) Billing
16 17 (1.58%) Public Sector
17 15 (1.39%) Aerospace
18 14 (1.30%) Back Office
18 14 (1.30%) Telecoms
19 13 (1.21%) Influencing Skills
Job Titles
1 352 (32.65%) Data Engineer
2 257 (23.84%) Senior
3 217 (20.13%) Architect
4 178 (16.51%) Lead
5 173 (16.05%) Software Engineer
6 136 (12.62%) AWS Engineer
7 120 (11.13%) Developer
8 105 (9.74%) Data Architect
9 99 (9.18%) Python Engineer
10 85 (7.88%) Senior Data Engineer
11 82 (7.61%) Solutions Architect
12 78 (7.24%) Azure Engineer
12 78 (7.24%) Python Software Engineer
13 77 (7.14%) Java Engineer
13 77 (7.14%) Java Software Engineer
14 76 (7.05%) AWS Software Engineer
15 72 (6.68%) Azure Data Engineer
15 72 (6.68%) Senior Software Engineer
16 71 (6.59%) Consultant
17 62 (5.75%) Data Consultant
Libraries, Frameworks & Software Standards
1 272 (25.23%) gRPC
1 272 (25.23%) Protocol Buffers
2 233 (21.61%) Pandas
3 218 (20.22%) NumPy
3 218 (20.22%) SciPy
4 111 (10.30%) PySpark
5 53 (4.92%) Kafka
6 18 (1.67%) Apache Beam
7 16 (1.48%) JSON
8 15 (1.39%) OpenID
9 14 (1.30%) .NET
9 14 (1.30%) RESTful
9 14 (1.30%) XML
10 9 (0.83%) MDX
11 8 (0.74%) Apache Avro
11 8 (0.74%) Spark Streaming
12 6 (0.56%) REST
12 6 (0.56%) Web Services
13 4 (0.37%) SOAP
13 4 (0.37%) YAML
Miscellaneous
1 276 (25.60%) Hedge funds
2 222 (20.59%) Derivative
2 222 (20.59%) Equities
2 222 (20.59%) Fixed Income
2 222 (20.59%) Foreign Exchange (FX)
3 79 (7.33%) Algorithms
4 62 (5.75%) Data Structures
5 55 (5.10%) Clustering
6 50 (4.64%) Cloud Native
7 47 (4.36%) Product Ownership
8 44 (4.08%) Self-Motivation
9 42 (3.90%) Onboarding
10 35 (3.25%) Management Information System
11 30 (2.78%) Pricing Data
12 28 (2.60%) Greenfield Project
13 24 (2.23%) Public Cloud
14 23 (2.13%) Enterprise Software
15 16 (1.48%) Blockchain
15 16 (1.48%) Legacy Applications
16 15 (1.39%) NHS
Operating Systems
1 26 (2.41%) Linux
2 23 (2.13%) Unix
3 7 (0.65%) Windows
4 3 (0.28%) Windows Server
5 1 (0.093%) AIX
5 1 (0.093%) Android
5 1 (0.093%) Solaris
5 1 (0.093%) VMS
Processes & Methodologies
1 508 (47.12%) Agile
2 320 (29.68%) Asset Management
3 317 (29.41%) Business Intelligence
4 313 (29.04%) Software Engineering
5 310 (28.76%) Analytics
6 264 (24.49%) ETL
7 255 (23.65%) Data Modelling
8 247 (22.91%) Data Pipeline
9 243 (22.54%) Investment Management
10 237 (21.99%) Data Engineering
11 233 (21.61%) Machine Learning
12 232 (21.52%) Artificial Intelligence
13 180 (16.70%) CI/CD
14 176 (16.33%) DevOps
15 171 (15.86%) Roadmaps
16 162 (15.03%) Visualisation
17 150 (13.91%) Data Architecture
18 142 (13.17%) Data Strategy
19 137 (12.71%) Data Science
20 114 (10.58%) Continuous Improvement
Programming Languages
1 600 (55.66%) Python
2 492 (45.64%) SQL
3 387 (35.90%) Java
4 278 (25.79%) JavaScript
5 220 (20.41%) TypeScript
6 98 (9.09%) T-SQL
7 64 (5.94%) Go
8 56 (5.19%) Scala
9 47 (4.36%) Data Analysis Expressions
10 31 (2.88%) C#
11 26 (2.41%) R
12 21 (1.95%) PL/SQL
13 11 (1.02%) Kotlin
13 11 (1.02%) U-SQL
14 9 (0.83%) PowerShell
15 4 (0.37%) C
16 3 (0.28%) Bash
17 2 (0.19%) C++
17 2 (0.19%) VBA
18 1 (0.093%) Shell Script
Qualifications
1 89 (8.26%) Azure Certification
1 89 (8.26%) Degree
2 66 (6.12%) AWS Certification
3 43 (3.99%) Microsoft Certification
4 22 (2.04%) Computer Science Degree
5 13 (1.21%) Master's Degree
5 13 (1.21%) SC Cleared
5 13 (1.21%) Security Cleared
6 11 (1.02%) DBS Check
7 8 (0.74%) DV Cleared
8 6 (0.56%) CIMA
9 3 (0.28%) MCP
9 3 (0.28%) MCSE
10 2 (0.19%) MCSA
10 2 (0.19%) Postgraduate
11 1 (0.093%) AWS Certified Solutions Architect
11 1 (0.093%) IBM Certification
11 1 (0.093%) LPIC
11 1 (0.093%) Numerate Degree
Quality Assurance & Compliance
1 228 (21.15%) Data Quality
2 54 (5.01%) GDPR
3 20 (1.86%) SLA
4 13 (1.21%) QA
5 8 (0.74%) PSD2
6 6 (0.56%) Accessibility
6 6 (0.56%) California Consumer Privacy Act
7 3 (0.28%) PMO
8 1 (0.093%) HIPAA
8 1 (0.093%) ISO/IEC 27001
System Software
1 25 (2.32%) Docker
2 8 (0.74%) HDFS
3 7 (0.65%) Active Directory
4 1 (0.093%) Apache Flume
4 1 (0.093%) Virtual Machines
4 1 (0.093%) VMware Infrastructure
Systems Management
1 116 (10.76%) Kubernetes
2 82 (7.61%) Terraform
3 54 (5.01%) Prometheus
4 3 (0.28%) HP Fortify
5 2 (0.19%) Argo
5 2 (0.19%) Progress Chef
6 1 (0.093%) Ansible
Vendors
1 290 (26.90%) Databricks
2 272 (25.23%) Microsoft
3 71 (6.59%) Alteryx
4 55 (5.10%) Google
5 46 (4.27%) Oracle
6 22 (2.04%) Cognos
7 18 (1.67%) Sybase
8 15 (1.39%) SAP
9 13 (1.21%) Teradata
10 11 (1.02%) Cloudera
11 8 (0.74%) Duck Creek
12 7 (0.65%) Adobe
12 7 (0.65%) Informatica
13 6 (0.56%) Apple
13 6 (0.56%) Talend
14 4 (0.37%) OpenAI
14 4 (0.37%) SnapLogic
15 3 (0.28%) CheckPoint
15 3 (0.28%) Citrix
15 3 (0.28%) Veracode