scalable and re-usable frameworks for ingestion and transformation of large data sets Integrate the end-to-end data pipeline to take data from source systems to target data repositories ensuring the quality and consistency of data is maintained at all times Work with other members of the project … using AWS or Azure such as: Azure SQL Data Warehouse, Azure Data Lake, AWS S3,AWS RDS,AWS Lambda or similar Have experience with OpenSource big data products i.e. Hadoop Hive, Pig, Impala or similar Have experience with OpenSource non-relational or NoSQL data more »
the design of these services through both Alpha and Omega, and the opportunity to step up to a portfolio level. We code in the open (see our code here: ) and our primary tech stack is Python/Django, Nodejs/React, running on AWS. DBT optimises the architecture for … on technical plans. Introducing practices such as TDD, continuous integration, continuous delivery, and DevOps to teams. Working with cloud infrastructure and services. Knowledge of open-source principles and technologies. Designing secure systems and awareness of emerging security issues. Communicating with both technical and non-technical audiences. It is … in a new window).People working with government assets must complete baseline personnel security standard (opens in new window) checks.Nationality requirementsThis job is broadly open to the following groups:UK nationalsnationals of the Republic of Irelandnationals of Commonwealth countries who have the right to work in the UKnationals of more »
a simple manner Preferred Qualifications Financial Services or Pharmaceutical industry experience Experience in distributed system design Experience with Pure/Alloy Working knowledge of open-source tools such as AWS lambda, Prometheus Spark, Hadoop orSnowflake knowledge would be a plus. Additional Information Location: This role can be delivered more »
the agenda and review other available data to augment existing data to find new commercial insights. Promote innovative methods, such as machine learning and opensource coding languages, to support the increased use of external data and enhanced insights from the data lake, actively sharing best practices. Provide more »
Darlington, County Durham, North East, United Kingdom Hybrid / WFH Options
Inspire People
on technical plans. * Introducing practices such as TDD, continuous integration, continuous delivery, and DevOps to teams. * Working with cloud infrastructure and services. * Knowledge of open-source principles and technologies. * Designing secure systems and awareness of emerging security issues. * Communicating with both technical and non-technical audiences. In return more »