1 to 25 of 116 Data Lake Jobs in London

will develop superior digital and technology concepts and architecture solutions as well as support technical implementations actively and on site, applying your sound Data/Cloud technical know-how, your understanding of business contexts, along with your analytical and conceptual skills. As you acquire experience, you will be … technical project management, digital delivery, or technology consulting environment • Platform implementation experience (Apache Hadoop - Kafka - Storm and Spark, Elasticsearch and others) • Experience around data integration & migration, data governance, data mining, data visualisation, database modelling in an agile delivery-based environment • Experience with … code reviews & performance of technical proofs of concepts • Analysis of complex IT Application landscapes, optimization of development processes and Solution Architecture • Data lake experience on AWS and Azure • General knowledge of database technologies and trade-offs • University degree with above-average academic performance in a mathematical-scientific more »

Synapri collaborates with a large transport company, who is seeking a Senior Data & Analytics Developer for a 12 months contract. You will be involved in the design, development and configuration of applications, components and tools according to the technical plans set by the Development Technical Lead. Knowledge and … experience required: In-depth working knowledge of ETL/ELT, data warehousing/business intelligence methodologies including dealing with big data, cloud technology and unstructured data Knowledge of star schema structure & design, detailed understanding of Kimball & Inmon and … hybrid data warehouse design Detailed knowledge using cloud data products like Data Factory, Event Hubs, Data Lake, Synapse, Azure SQL server. Detailed knowledge in developing in Databricks and experience in coding with PySpark. Spark SQL ETL coding standards: ensuring that code more »

for designing, developing, and maintaining data pipelines and systems in Azure Cloud environments, with a strong emphasis on Azure Data Lake, Azure Data Factory, and Microservices architecture. Key Responsibilities: Develop and maintain data pipelines in Azure Cloud environments, leveraging Azure Data Lake, Azure Data Factory, and Microservices architecture. Design and implement scalable and efficient data processing solutions using technologies such as ServiceBus, REST API, App Service, SQL Data Warehouse, and Data Mart. Utilize Vector Databases, Cosmos DB, Redis, and Elasticsearch … engineering. Bachelor’s Degree or higher in Computer Science or a related field. Proficiency in Azure Cloud environments, Azure Data Lake, Azure Data Factory, Microservices architecture. Experience with Vector Databases, Cosmos DB, Redis, Elasticsearch. Strong programming skills in Python, Spark, Databricks, Pyspark, SQL more »

Data Analyst Location: Hybrid - London Package: Negotiable + Benefits Data and Data services is a key strategic enabler in Brown and Brown Europe as we look to integrate acquired business, enable them to provide great customer service and grow through the provision of superior … data services The Data Analyst for Data Operations will be responsible for managing stakeholders, gathering, collating and analysing data issues and translating them into data and reporting needs, using technical expertise to ensure data accuracy and high-quality … communication and collaboration skills Experience with data warehousing Experience in the Microsoft stack - Azure Data factory, Azure Data Lake Storage, Azure SQL Server, Azure Power Platform, Power BI. Working early some mornings, on a scheduled basis to monitor overnight operational batches and resolve more »

Data Engineer Grade: Senior Consultant – Senior Manager Location: London Employment Type: Full Time The Opportunity: A Tier One Technology Consultancy seeks a highly ambitious and technically astute AWS Data Engineer to help design, build and deploy cloud-based data solutions - transforming raw data … in data cleaning, wrangling, visualization, and reporting. • Specialised in AWS cloud technologies for ETL, data warehouse, and data lake design. • Hands-on experience with AWS services like EMR, Glue, RedShift, Kinesis, Lambda, DynamoDB. • Capable of processing large volumes of structured and unstructured data … AWS cloud, including Platform-as-a-Service, server-less, and container technologies. • Diverse skills in ETL, data warehouse, and Data Lake design. • Familiarity with various tools, cloud technologies, and approaches. • Expertise in AWS or equivalent open-source tools. • Proficient in defining the end-to-end more »

Are you passionate about leveraging cutting-edge data technologies to drive business success? Do you thrive in dynamic environments where innovation and collaboration are valued? If so, we have an exciting opportunity for you! Position: Lead Data Engineer Responsibilities: Lead Data Management and Architecture … Spearhead the design and implementation of robust data solutions across the organization, ensuring scalability and efficiency. Next Gen Data Platform Leadership: Act as the primary data engineer supporting the development of our next-generation data platform, collaborating closely with third-party vendors … Skills and Experience: Proficiency in Microsoft Azure Data Services, including Azure SQL Database, Azure Data Factory, Azure Data Lake Storage, and Azure HDInsight. Advanced skills in SQL programming, with the ability to design, optimize, and maintain complex queries and databases. Strong experience in more »

Introduction Axtria is a global provider of cloud software and data analytics to the Life Sciences industry. We help Life Sciences companies transform the product commercialization journey to drive sales growth and improve healthcare outcomes for patients. We are acutely aware that our work impacts millions of patients … in the development of disaster recovery and business continuity plans. Preferred Qualifications: 3-5 years working in Azure cloud environments, Azure Data Lake, Azure Data Factory Bachelor’s or Master’s degree in Computer Science, Information Technology, or a related field. Experience working in healthcare … or pharmaceutical industry. Proficiency in Azure Cloud environments, Azure Data Lake, Azure Data Factory, Microservices architecture. Knowledge of Azure DevOps, CI/CD pipelines, GitHub, Kubernetes (AKS). Additional certifications in relevant areas such as Azure Security, Azure Networking, or Azure AI/ML. Experience more »

the UK, USA and Australia, their dynamic technology team – the driving force behind their success – is looking for a talented and creative Senior Data Engineer to develop and implement new Greenfield software solutions for seamless data accessibility. The Role: Designing and implementing data solutions … constructing new infrastructure to centralize data, and delivering actionable insights across the business. Responsibilities include collaborating with key stakeholders, gathering requirements, designing solutions, and leading the development lifecycle. Key Duties: Cultivate relationships with key stakeholders to design and implement data solutions aligned with business requirements. Lead … Essential Requirements: Proven track record in designing and building data infrastructure and ETL pipelines. Proficiency in Azure Platform, including Data Lake, Data Factory, Synapse, Logic Apps, and Function Apps etc. SQL Server, including Store Procedures, T-SQL, or similar enterprise database toolset. Familiarity more »

infrastructure investments. Be part of a dynamic team driving growth and maximizing value in the alternative asset management industry. Key Responsibilities: Support the Data Team Lead in system updates. Develop, modify, and maintain software components. Resolve data/software issues and suggest technical solutions. Collaborate with … experience in Azure data systems. Proficiency in Data Warehouse technologies and SQL. Familiarity with Azure services like Data Lake and Data Factory. Essential Experience: Supporting traditional Data Warehouses. Building Azure data pipelines and data lakes. more »

I am looking for a Machine Learning (ML) and Data Engineer to join my clients growing Data & AI team and contribute to Artificial Intelligence technology solutions for their public sector and enterprise clients. This role is offering a generous salary of up to £70K! As a … ML and Data Engineer, you will collaborate with a team of data scientists, data architects, software engineers, business analysts and delivery managers to deliver innovative solutions that leverage the power of cloud-based AI. This is an exciting opportunity for a driven individual to … Microsoft Azure AI services. Experience with Azure data and analytics services, such as Azure Data Factory, Azure Data Lake, Azure Synapse Analytics, Azure SQL Database, etc.. Experience with Azure DevOps, GitHub, or other tools for version control, testing and deployment. This is an more »

Senior Data Engineer (No Sponsorship Available) I am seeking an experienced Data Engineer who has experience managing green field projects and implementing modern data platforms that connect business systems and data sources to drive insights through data analytics. The ideal … candidate will have experience using Azure Platforms for consolidating data sources and making them accessible to the wider business to support growth and business strategy. Must have strong communication skills and be comfortable liaising with key stakeholders and other business units to understand business needs and gather requirements. … designing and building data infrastructure and ETL pipelines Experience with Azure Platform including Data Factory, Synapse, and Data Lake DB development experience with SQL Software development experience in relevant languages like Python or C# Experienced with T-SQL and RESTful APIs Experience with more »

Data Engineer (one day a week in the central London office) Sports Analytics This role is a unique opportunity to combine technical challenges with creativity in a collaborative, high-standard work environment. By joining this team, you’ll not only be part of a creative and open work … a detail-oriented innovator and team player, we’re excited to hear from you. Responsibilities Design, construct, install, test, and maintain highly scalable data management systems. Ensure systems meet requirements and industry practices for data quality and integrity. Integrate data management technologies and software … tools into existing structures. Create data tools to support sport data modelling, prediction and analytics. Work with data and analytics experts to strive for greater functionality in our data systems. Essential Skills Proven experience as a Data Engineer, with expertise more »

Data Architect Salary: £90,000 - £110,000 Location: Remote/Hybrid Data Idols are working with a fast-growing technology company who are disrupting the software engineering industry. They are looking for a Data Architect with strong experience in data engineering and … experience in delivering innovative solutions. The Opportunity In this role as a Data Architect, you will play a pivotal role in ensuring the efficiency of their data architecture while heading the migration from legacy systems to modern cloud solutions. You will design and build scalable database and data lake solutions, implement CICD pipelines and automate the deployment of data products into production environments. Skills and Experience Proficiency in AWS Great communication more »

About Tredence: - Tredence focuses on last mile delivery of insights into actions by uniting its strengths in business analytics, data science, and software engineering. The largest companies across industries are engaging with Tredence and deploying its prediction and optimization solutions at scale –empowering end users to improve decision … methodology; think strategically and execute methodically. Develop and manage capacity and growth projection forecasts of the environment within budgets. Create and maintain optimal data pipeline architecture, assemble large, complex data sets that meet functional/non-functional business requirements. Drive optimization, testing and tooling to improve … in Data bricks Experience managing distributed teams preferred. Strong technical experience in large distributed systems, Data Warehousing, Data Lake at scale Project management skills: financial/budget management, scheduling and resource management experience with medium and large-scale projects Comfortable working with ambiguity more »

Job Description: I am currently looking to hire for a Lead/Senior Data Engineer to lead modernisation efforts for a client of ours who are a leading company at the top of their game in their respective field. Duties and responsibilities: Develop and refine ETL and data integration solutions leveraging a variety of technologies and data sources. Maintain and enhance the Data Engineering Architecture. Gather and process raw data from various sources. Investigate methods to improve data quality and reliability. Identify and implement internal process enhancements, such … as automating manual tasks and optimising data delivery. Utilise cloud-based infrastructure (Azure) for hosting data solutions and applications. Collaborate with architects, data analysts, and data scientists to achieve business objectives. Requirements: Demonstrated expertise in developing and maintaining ETL/ELT processes more »

have created is unique and difficult to replicate, with our ‘best of both’ approach combining deep human expertise with advanced digital, tech and data capabilities. This combination not only allows us to deliver excellent customer experiences, but to do so efficiently and at scale. We are proud of … of a business that cares too Want to continue to grow professionally and be the best version of yourself Job Description The Lead Data Engineer is a key role to ensure Shawbrook unlocks the value contained in its data across the organisation in a well-governed … and table formats (Delta etc) This is an incredibly exciting role that rarely becomes available. You will own a well-designed data lake serving a 50+ user community who run '000s of queries across TBs of data each month with a mandate to take it more »

specific business challenges. Implement and maintain scalable and efficient AI systems, ensuring they meet business requirements and performance benchmarks. Collaborate with business analysts, data scientists, and IT teams to integrate AI solutions into existing systems and workflows, enhancing their capabilities and impact. Stay abreast of advancements in AI … efforts. Conduct data analysis and feature engineering to prepare data for use in AI models, utilizing Azure Data Lake and other data storage solutions. Develop robust testing and validation processes to ensure the accuracy and reliability of AI models and solutions. … Open AI Service, Azure Cognitive Services, and Azure Machine Learning. Familiarity with Azure Databricks is a plus. Solid background in machine learning algorithms, data pre-processing, feature engineering, and model evaluation. Experience with deep learning frameworks like TensorFlow or PyTorch is desirable. Proficiency in handling large datasets, experience more »

Data Engineer Location: London, UK Hybrid: 3x a week in office Salary: £63,000 About the Client Our client is a top-tier professional services firm known for its exceptional workplace culture and some of the most sought-after office spaces in London. Role Overview Our client is … in search of a skilled Data Engineer to manage and engineer the data warehouse. The opportunity will be amazing for a Data Engineer who is happy to work in a small team. Incredible opportunity to mentor/lead a small Data Engineering … team within a year! Main Responsibilities Build scalable data pipelines on a daily basis Code on a daily basis with Azure SQL and python Develop and maintain the logical and technical architecture to fulfil the client’s information, insights, and reporting needs. Stay informed on the latest developments more »

Data Consultant (Azure) £40,000-£50,000 salary Hybrid (London) Azure, Power BI, Purview, Python, ETLs, Consulting One of my long-standing consultancy clients, one of the UK's leading Microsoft Solutions Partners for over 20 years and experts in digital transformation with Azure, have just reached out … with an urgent hire for a Data Consultant to join the team. **This is a permanent position ( no contractors will be considered ), and you must have full right to work in the UK to apply** The ideal candidate will be working on projects using products in within the … MS Data stack, including Power BI, Synapse, Data Lakes, and Fabric. This is a customer-facing role, so you'll be regularly liaising with clients to understand their business needs and offer technical solutions. Essentials: Experience in a consultancy . Experience with Data Quality more »

Here at Vallum, we are looking for a Power BI Developer to assist the Head of Data & Reporting and the Project Controls Manager. In this role, you will coordinate Project Controls activities related to the development, design, manufacture, construction, and commissioning of two EPRs. Your efforts will ensure … on licence management and support. Principal Accountabilities: Report Development : Develop and implement Power BI reports and dashboards for the project and external clients. Data Organisation : Structure data to ensure compliance with WBS, CBS, and OBS for consistent reporting. Trend Analysis : Use data to predict … trends and perform statistical analysis, visualising data in user-friendly formats. Digital Reporting : Implement digital reports and dashboards using data from the Data Lake. Support Monthly Reports : Provide intelligent information and analysis for monthly report production. Training and Support : Develop training materials, conduct sessions more »

Lead Data Engineer £80,000 - £100,000 DOE Mainly remote, some travel may be required Blue Wolf Digital are partnered with a boutique consulting firm who are looking to grow significantly this year due to some greenfield project wins. I'm looking for an experienced Lead Data … touch. Key Responsibilities: Design, develop, and maintain cutting-edge Databricks or Snowflake Lakehouse solutions from scratch Collaborate with cross-functional teams to understand data requirements and translate them into efficient ETL processes. Work with Azure services to ensure data storage, security, and scalability. Develop and optimize … SQL queries for data extraction and analysis. Stay updated on industry trends and regulatory changes within the insurance sector to inform data strategies and solutions. Experience: Proven experience in building and maintaining data lakes or data warehouses using Databricks on Azure. Strong more »

Head of Integrations 🏡 London - Hybrid 🔵 Permanent 💸 Salary - £65K Extensive experience of implementing an Enterprise Service Bus, Data lakes and design patterns. Experience with Azure Previous experience of managing a team in a complex environment Experience of automating tasks using Azure Dev-Ops, Terraform, C# or Python. Substantial experience more »

Lead Data Engineer - up to £100K - Hybrid London Our client a global brand based out of lovely offices in West London are looking for a Lead Data Engineer to work on a business-critical, brand new, innovative high availability application which will be used by thousands … who is looking to take their next step in their career and wants to take on more responsibility. Technical Skillset: Experience as a data engineer developing and maintenance of ETL/ELT processes Good experience in Data Modeling within a cloud … based data platform Strong experience with SQL Server Azure data engineering stack, including Azure Synapse and Azure Data Lake Python, PySpark and T-SQL In return you will be offered a competitive salary and benefits package, remote working options and an opportunity to more »

Java Solutions Architect (Architecture Programmer Developer Java Python Software Engineer Data Enterprise Engineering Developer Programmer AWS GCP Python Athena Glue Airflow Ignite JavaScript Agile Pandas NumPy SciPy Spark Dremio Apache Iceburg Iceberg PySpark MWAA Arrow DBT gRPC protobuf Snowflake TypeScript Manager Finance Trading Front Office Investment Banking Asset … external sources such as Bloomberg and Reuters and it's distribution across investment technology. The data enters an AWS data lake which, over the next 12 months, will be migrated to a data mesh architecture. You will be leading one of the teams … in the area that is responsible for this core data lake and its migration to a data mesh. You will be working with Java, Python, AWS (EC2, ECS, EKS, Glue), SQL, Spark, MWAA/Airflow, Iceberg and DBT. You will have line-management responsibility for more »

Python Solutions Architect (Architecture Programmer Developer Java Python Software Engineer Data Enterprise Engineering Developer Programmer AWS GCP Python Athena Glue Airflow Ignite JavaScript Agile Pandas NumPy SciPy Spark Dremio Apache Iceburg Iceberg PySpark MWAA Arrow DBT gRPC protobuf Snowflake TypeScript Manager Finance Trading Front Office Investment Banking Asset … external sources such as Bloomberg and Reuters and it's distribution across investment technology. The data enters an AWS data lake which, over the next 12 months, will be migrated to a data mesh architecture. You will be leading one of the teams … in the area that is responsible for this core data lake and its migration to a data mesh. You will be working with Java, Python, AWS (EC2, ECS, EKS, Glue), SQL, Spark, MWAA/Airflow, Iceberg and DBT. You will have line-management responsibility for more »