1 to 25 of 480 Big Data Jobs

Active SC Clearance Required Job Overview: The candidate will be joining a team who are experienced data scientists, but with less experience in Microsoft Azure tools (e.g. Microsoft Machine Learning and Azure AI). They should help with the upskilling of the team in these tools as well as … machine learning and AI, including good knowledge of cloud architecture and networking for Azure-specific solutions. · Experience in Infrastructure as Code deployments · Experience in data pipelines for machine learning workflows. · Experience in deploying machine learning models and large language models. · Experience in establishing CI/CD pipelines. · Proficiency in … programming with Python plus any additional programming languages. · Strong foundations in data science methods and best practices. · Experience with open-source machine learning libraries and deep learning frameworks. · Experience with at least one big data framework. · Record of having been part of a team that has deployed more »

scalable software and systems, architect low latency infrastructure solutions, proactively guard against cyber threats, and leverage machine learning alongside financial engineering to continuously turn data into action. Engineering, which is comprised of our Technology Division and global strategist groups, is at the critical center of our business. Our dynamic … the limit of digital possibilities? Start here. Who We Look For Goldman Sachs Engineers are innovators and problem-solvers, building solutions in risk management, big data, mobile and more. We look for creative collaborators who evolve, adapt to change and thrive in a fast-paced global environment. The … entrepreneurial spirit of a tech start-up. Our goal is to provide the best client experience. Through the use of modern technologies centered on data and analytics, we provide customers with powerful tools that are grounded in value, transparency and simplicity to improve cash flow management efficiency. The Team more »

managing diverse teams and clients. RiverSafe is an incredibly innovative consultancy with a range of offerings which delivers fast time to value for Security, Big Data Analytics, and Operational Intelligence use cases. Key Responsibilities As a Principal Consultant in RiverSafe, you will be working within our Cyber Defence … and Big Data practice and will have exposure to a variety of security technologies delivering security implementations or remediation programmes. Highly motivated and experienced in managing teams and engagement with clients, you will be a team player who recognises the value in working and building the experience levels … to address risks across AV, patching, secure build, vulnerability scanning & remediation, logging, and monitoring, & threat management. Security around emerging technology platforms – primarily Cloud security & Big Data Analytics. Breach and incident management, design and implementation of breach and major incident management practices. Experience in working within agile development environments more »

Ensembl project has witnessed a dramatic growth in the breadth of species and depth of genomic annotations, which are collectively integrated into the Ensembl data stores. With major global biodiversity initiatives and increasing efforts to apply genomics in agriculture to improve productivity and sustainability, coupled with the continued evolution … of sequencing technologies, the rate of data growth is expected to continue. To accommodate these burgeoning volumes of genomic data, a new generation of Ensembl browser has been developed, supported by a complex backend infrastructure. We are excited to launch a Team Leader EMBL Faculty position to head … EBI . Your role Your primary responsibility will be to lead a multidisciplinary team of approximately 20 staff members, including software developers, bioinformaticians and data curators to build coordinated data infrastructures to service different scientific communities that leverage the underlying Ensembl infrastructure. This may include omic data more »

Type: Permanent Posted On: 03/12/2023 Head of Data Engineering required by an AI business who is changing an industry. You’ll be leading the build out of a real time streaming platform from scratch, that will become the organisation’s all in one data platform and golden source for data. This proprietary data platform has scope for a lasting impact worldwide. What we offer Lucrative Stock options Hybrid working in London office Our growing success comes from providing next generation technological solutions in an accessible and easy to operate way. We are … a Data-Driven company using Knowledge Graphs, with AI at the heart of our software. This is powered by mammoth quantities of data and is changing the way that software and data platforms are built. Role Head of the Data Engineering function but still cutting your more »

converters. Converted code is causing failures/performance issues. Your responsibilities; As a Spark Scala Engineer you will be working for the GDT (Global Data Technology) Team, you will be responsible for: Designing, building, and maintaining data pipelines using Apache Spark and Scala Working on an Enterprise scale … and Cloud Services in one of the Clouds (GCP). Mandatory Skills; At least 8+ Years of IT Experience with designing, building, and maintaining data pipelines . At least 4+ Years of experience with designing, building, and maintaining data pipelines using Apache Spark and Scala. Programming languages: Proficiency … in Scala and Spark is essential. Familiarity with Python and SQL is often a plus. Big Data technologies: Understanding of HDFS,... more »

Data Architect/Cloud Data Architect (GCP) Google Cloud/GCP, Machine Learning/Generative AI Remote-based or Hybrid should you prefer (London) £100k-£140k basic salary + bonus Benefits: 25 days holiday + bank holidays Private Healthcare Discounted Gym Membership (Nuffield + Hussle) Life Insurance, Income … Protection, Employee Assistance Program Pension Scheme (max 10%) We are working with an AI/Data Consultancy with core clients within the Retail, Financial Services, Gaming, SaaS sectors. The successful Cloud Architect will work with an array of technologies with clients from several industries & sectors meaning you will find … clients, devising innovative solutions to address their needs, defining project parameters, and supervising the execution of these projects. Role of the Cloud Architect/Data Architect: Utilising advanced data processing and cloud technologies effectively to stimulate innovation. Engaging with clients to create top-notch solutions tailored to their more »

and troubleshooting of the SOC’s technology stack (hardware and software). The engineer will also assist with the continued development and maintenance of data pipelines and signature updates and the professional development of the system engineering team. Tasks: • Perform system administration on specific cyber defence applications and systems … defence network tools in response to new or observed threats within the network environment or enclave. • Manage the compilation, cataloguing, distribution, and retrieval of data from a range of enterprise networks and data sources. • Implement data management standards, requirements, and specifications. • Develop data standards, policies, and … procedures. • Analyse data sources to provide actionable recommendations and facilitate data-gathering methods. • To share knowledge, skills and experience, create and improve documentation, and train new members of the data engineering team. Knowledge: • Knowledge of big data technologies and ecosystems (e.g., NiFi). • Knowledge of more »

standards, accessibility, and development best practice Bonus Skills • Building microservices in C# • Experience building solutions that integrate with LLMs • Knowledge of Machine Learning and Big Data • Using the GOV.UK Design System Who you'll be working for Global digital transformation consulting firm Winner at the Glassdoor Employee's more »

DevOps Engineer -Big Data We are looking for a fully remote DevOps Engineer to join a blockchain intelligence team helping regulators and law-enforcement fight crypto criminals with real-time analysis. This team is a spin off from a Blockchain as a service giant. They aim to aid … in the fight against crypto money laundering at scale, by helping both financial institutions, and government organizations solve investigations in this space through a data-driven approach. A big focus for the tech team (~40 people) this year is to modernize their stack, with the DevOps team focusing … on migrating legacy infrastructure towards AWS, IaC, and containerized architecture. As the team analyses 30TB of Data daily, previous DevOps experience with Big Data is essential. We are seeking a DevOps Engineer with the following: Experience maintaining and scaling AWS services Big data experience with more »

Vertex Search is representing a fast-growing financial services firm that is now looking to make a Senior Data Engineering hire, the first role of its kind into the business with huge scope for impact and influence. The nascent data engineering team will collaborate with internal teams around … the business with a focus on data engineering, pipelines, tooling and services. Key Responsibilities: Data Engineering: Design and develop data services across various business functions Data Pipelines: Design and develop data pipelines for extracting, transforming, and loading data into appropriate storage systems. Data Tools and Services: Develop data tools and services to discover, understand, and utilize data effectively. Best Practice: Advise on and contribute to the development of data governance and management practices. Collaborate across teams to understand requirements and deliver data solutions. Skills and Experience: Required: 7+ more »

new client is looking for a ServiceNow consultant who will be accountable for enriching, managing, and ensuring the quality of organisational and product usage data, with a focus on efficient query writing. They will enforce data governance policies and best practices, as well as develop documentation on data assets. Role includes: Instrument data collection on the ServiceNow platform Writing complex queries Maintain an in-depth understanding of data flow within the business Partner across multiple data teams Identify data integrity issues and conduct root cause analyses Develop workflows, processes, SLAs, and documentation for … product usage data governance Must have experience: 2 - 3 years Writing complex queries 2 - 3 years Experience with Glide Skills: big data analytics, database management, or an equivalent analyst position 2 - 3 years + developing on the ServiceNow platform 2 - 3 years Glide experience Proficiency in PL more »

Role Are you looking for an exciting opportunity to make a difference? Join our team and contribute to the future of fusion energy. As Data Engineer , you will play a central role within hardware/software development projects spanning the Authority's programmes, helping to develop HPC (High Performance … Computing) infrastructure towards ITER and the era of Big-Data and Exascale computation. This role has a bias towards High Throughput Computing (HTC) workflows and cloud-based computing in support of experiments (e.g. MAST-U). Additional Responsibilities: Collaborate with national and international partners across universities, research labs … and industrial organisations to deliver projects focused on data delivery, data object storage, and data-centric high-performance computing. Engage in approximately 3-4 projects annually, contributing to innovation and growth of UKAEA's capabilities in this domain by leveraging external resources. Work within a small, focused more »

client is dedicated to revolutionizing the world of connected mobility, aiming to make transportation more personal, convenient, enjoyable, and safe through the power of big data and a customer-focused approach. Their mission is to create and enable technologies that delight users and enhance their experience with their more »

A Commercial Insurance client of mine is seeking a Data Analyst to join their growing Data function. The perfect candidate will harness their enthusiasm for big data and analytics to deliver actionable insights across various business domains. This role involves performing regular and special analyses to … support business needs. Responsibilities: Grasp the daily challenges our business encounters that data can illuminate. Gather and scrutinize data pertinent to business challenges. Create clear visual representations to simplify complex data sets for straightforward interpretation. Qualifications: Ideally 2-5 years of experience in Data Analysis. Proficient more »

Role: Data Architect Location: London, UK Define all architecture patterns (Data Mesh, Data Fabric, EventHub etc) Define each architecture pattern Data & Analytics Platform Functional Blueprint, Logical Layers and Each layer Capabilities Identify All Ingestion Patterns (Real Time, Batch, Structured, Unstructured (Images/PDF) for each data & analytics platform Analyze existing Data & Analytics Platform Architecture Playbook, Identify Gaps and Enrich Playbook Guided Tooling Framework (Scorecard based approach) Potential Engineering Frameworks (Structured/Semi-Structure) Define Intake Frameworks (Unstructured - OCR/Intake) Data Governance Capabilities/Framework/Integration with rest of Ecosystems Design & Recommend … Data Traceability/Lineage/DQ/Reconciliation and Observability Platform Understanding of Cloud (GCP, AWS) Services Offering Deep Understanding of Banking Regulations and solution required to adopt them in Hybrid Cloud Ecosystem Data & Analytics Platform Governance: FinOps, Regulatory Conduct MVP/POC Qualifying Criteria Must have 12+ more »

Data Engineer Grade: Senior Consultant – Senior Manager Location: London Employment Type: Full Time The Opportunity: A Tier One Technology Consultancy seeks a highly ambitious and technically astute AWS Data Engineer to help design, build and deploy cloud-based data solutions - transforming raw data into actionable insights … industry verticals: Consumer & Manufacturing, Defence & Security, Energy & Utilities, Financial Services, Government & Public Services, Healthcare, Life Sciences, and Transport. Essential Skills & Experience: • Design and deploy data pipelines using Java, Python, Scala, Spark, and SQL in big data architecture. • Execute tasks involving scripting, API data extraction, and SQL … queries. • Proficient in data cleaning, wrangling, visualization, and reporting. • Specialised in AWS cloud technologies for ETL, data warehouse, and data lake design. • Hands-on experience with AWS services like EMR, Glue, RedShift, Kinesis, Lambda, DynamoDB. • Capable of processing large volumes of structured and unstructured data on more »

services, leveraging React for the front-end and integrating with back-end services. Optimise User Experience : Ensure high performance, stability, and security of our data collection and processing platforms, enhancing user interfaces and interactions. Deploy and Manage Infrastructure : Oversee the deployment and management of applications and databases, primarily on … AWS. Collaborate and Innovate : Work closely with cross-functional teams to design, develop, and deliver data-driven, client-centric platforms and applications. Stay Technologically Advanced : Continuously explore and integrate the latest technologies to keep our solutions at the forefront of the industry. Tools and Technologies You'll work with … MS SQL for database management. Cloud Platform : AWS for cloud infrastructure. Programming Languages : JavaScript for front-end development and Java for back-end processes. Big Data Technologies : Hadoop, Spark, or Kafka for handling large-scale data processing. What We Need from You Essential Skills: Technical Proficiency : Expertise more »

Are you passionate about data quality and governance? We’re looking for a Data Governance Engineer to elevate our client’s data practices and ensure the integrity of the product usage data. The Offer: Annual Salary up to £65,000 doe 6-month contract – PAYE – with a … option – candidates must be present in the county with appropriate work visa – no work permits provided). What You’ll Need: 5+ years in big data analytics or database management. 1-3 years developing on the ServiceNow platform. Experience in GQL – Glide language puts you on top of … all candidates. Proficiency in PL/SQL and an additional programming language. Strong problem-solving skills and ability to design impactful data models. Expertise in rolling out analytics dashboards (Power BI, Tableau Server). Ability to translate business needs into technical requirements. Good to have: Knowledge of ServiceNow products. more »

and helping them outperform their competition through speed and innovation. Apexon brings together distinct core competencies – in AI, analytics, app development, cloud, commerce, CX, data, DevOps, IoT, mobile, quality engineering and UX, and our deep expertise in BFSI, healthcare, and life sciences – to help businesses capitalize on the unlimited … would be great if you have: Bachelor's degree/master's degree in computer science, Computer Engineering, or other STEM field Experience with big data, enterprise integration patterns, micro-services Understanding of cloud services and cloud architecture Cloud security best practice Django and/or Python skills more »

infrastructure and applications. Responsibilities: Utilize recent AWS experience to design, implement, and manage AWS services such as EC2, ASG, RDS, and OpenSearch for streaming big data. Implement Infrastructure as Code (IAC) principles using tools like AWS CloudFormation or Terraform to automate infrastructure provisioning and management. Develop and maintain deployment more »

Cloud Solution Architect London Permanent Are you passionate about leveraging cloud and data technologies to drive business success? Do you thrive in a dynamic, international environment where your technical expertise can make a real impact? If so, we have an exciting opportunity for you! We are currently seeking a … cloud platforms and infrastructure-as-code technologies. Experience with containerization, serverless computing, and SOA architecture. Familiarity with relational and non-relational databases. Experience with BigData technologies is a plus. Excellent written and verbal communication skills. Ability to work well under pressure and adapt quickly to change. Strong team player with more »

Technical Product Owner - Solution Engineer – Martech - Adobe Experience Platform – Adobe Campaign - Big Data – Customer Data Platform ## 12 - 24 months; Inside IR35 ## Hybrid working I am looking for an Adobe Solutions Engineer/Architect/Product Owner experienced in the development of Customer Data Platforms … that the martech stack aligns with business objectives. Skills and Qualifications Essential: AEP CDP Proficiency: Hands-on experience with Adobe Experience Platform CDP, including data modeling, segmentation, activation, and analytics. Adobe Campaign Experience: Strong background in developing and executing email marketing campaigns using Adobe Campaign, with a deep understanding … of its data integration capabilities. Data Engineering Skills: Proficiency in SQL and data manipulation techniques, with experience in data cleansing, transformation, and integration. Preferred: Adobe Ecosystem Familiarity: Working knowledge of Adobe Analytics, Adobe Target, Adobe Experience Manager (AEM), and Adobe Launch. Tag Management Experience: Experience with more »

working with vendor and private cloud teams to update infrastructure and OS for applications, monitoring system performance and reliability, and troubleshooting operating system and data issues. You will also participate in project-related stand-up meetings and provide necessary documentation support. You will need to be based in the … under limited direction, handling multiple assignments simultaneously Strong verbal and written communication skills Thorough understanding of the Agile Software Development Lifecycle (SDLC) Knowledge of Big Data applications like Hadoop, Spark, and Kafka is a plus Proven Experience: in application development, technology, or a related field; equivalent work experience more »

Are you passionate about leveraging cutting-edge data technologies to drive business success? Do you thrive in dynamic environments where innovation and collaboration are valued? If so, we have an exciting opportunity for you! Position: Lead Data Engineer Responsibilities: Lead Data Management and Architecture: Spearhead the design … and implementation of robust data solutions across the organization, ensuring scalability and efficiency. Next Gen Data Platform Leadership: Act as the primary data engineer supporting the development of our next-generation data platform, collaborating closely with third-party vendors to build a highly scalable and robust … data hub. Develop and Optimize Data Pipelines: Design, construct, install, test, and maintain highly scalable data management systems, ensuring they meet business requirements and industry best practices. Data Integration and Storage Solutions: Implement data storage solutions and manage large-scale processing systems, overseeing the integration more »