Senior Data Engineer

Senior Data Engineer

Location: Remote, Oxford (Once per Month)

Salary: £60,000 - £85,000 + Benefits

Overview:

We are looking for a Senior Data Engineer to play a key role in evolving our data infrastructure, ensuring enhanced reliability and scalability. Reporting to the Data & Analytics Director, you will be integral in driving forward our data capabilities, as we move towards a larger data mesh set-up managed from a central hub of excellence. Over the past three years, we have built a lakehouse architecture with automated reporting and self-service capabilities. With an expanding scope, we now need a Senior Data Engineer to mature and scale our solution efficiently, focusing on DataOps/DevOps, data ingestion, and processing within Databricks.

Key Responsibilities:

• Enhance and stabilize our data infrastructure for increased reliability.
• Manage data ingestion pipelines (currently using ADF and Python).
• Ensure high-quality raw datasets to enable Data Analysts in creating reliable data models.
• Deploy and manage data-related tools on Kubernetes, including Airflow, Superset, and RStudio Connect.
• Support Data Analytics workflows by managing DBT, implementing deployment rules, and handling DevOps processes.

Skills & Experience:

• Proven Data Engineering experience, particularly with Azure, Databricks, and Delta Lake.
• Expertise in Kubernetes, Infrastructure as Code, and Terraform.
• Strong knowledge of Azure DevOps.
• Preference for simplicity and transparency in solutions, avoiding unnecessary complexity.
• Innovative and proactive approach, driving forward best practices and delivering effective solutions.

What We Offer:

• Enhanced holiday package to support work-life balance.
• Flexible hours and hybrid working options.
• Access to our modern head office in Oxford, Oxfordshire-a great environment for collaboration and teamwork.
• Comprehensive benefits package, including private medical insurance, competitive pension, cycle-to-work scheme, volunteering days, death-in-service insurance, and sick pay.
• Social events to foster team connection and maintain a fun, engaging workplace culture.
• Career development support through a detailed performance management system.

In Technology Group Ltd is acting as an Employment Agency in relation to this vacancy.