Senior Data Engineer

The Senior Data Engineer role will plan and drive forward the design and implementation of data pipelines, data stores, and APIs to acquire and prepare data for downstream applications, such as analytics, data insights, and data-driven tools. You will be a hands-on data engineer, who will contribute to the wider teams' outputs by ensuring data standards and architectures are applied, and by coaching colleagues on approaches to problem-solving and quality of deliverables. Additionally, you will be responsible for integrating data with various applications to ensure seamless data flow and accessibility.

The data domain will cover RSSB, rail industry, and national data sources in the areas of health, safety, quality, operational performance, and sustainability. It will also cover internal business datasets, ensuring that data is effectively integrated with applications to support business processes and decision-making.

Responsibilities 

• Ensure data made available for downstream applications is reliable, accessible, and in the appropriate format through active engagement with Product Management and key stakeholders.
• Plan and drive forward the development of data solutions, ensuring a balance between functional and non-functional requirements.
• Advise on best approaches to solving novel data problems, leveraging Python-based solutions.
• Ensure that data standards and architectures are applied to achieve well-engineered outcomes, and that solutions are reliable and perform optimally.
• Conduct complex data quality checks and monitoring, applying necessary remediation to maintain data integrity.
• Undertake necessary maintenance and optimisation activities to ensure the performance and reliability of data pipelines, APIs, and applications.
• Develop both quick prototypes and high-quality products, judging which approach is needed based on the context.
• Design, code, test, and document complex data engineering scripts and applications, applying software engineering best practices with a focus on SQL and Python.
• Collaborate effectively with cross-functional teams, including Product Management and key stakeholders, to understand business requirements and translate them into technical solutions.
• Communicate technical concepts and solutions effectively to both technical and non-technical audiences.
• Mentor and coach colleagues on approaches to problem-solving and quality of deliverables, fostering a collaborative and innovative team environment.

   

  Qualifications / Experience 

  Essential

  • BSc in a scientific, computer science, or related discipline.
  • Extensive experience with data analysis, data modelling, and quality assurance techniques.
  • Proficiency in using SQL and Python to solve a wide array of data challenges.
  • Experience developing data pipelines in Python using orchestrators such as Airflow.
  • Experience making data available in data science and data visualization tools such as Databricks and Power BI/ Fabric.
  • Experience with data extraction from a variety of systems, including familiarity with web-oriented architecture.
  • Ability to analyse and document technical solutions effectively.
  • Communicate effectively and share thoughts and ideas through methods appropriate to the audience.
  • Adapt and respond effectively when embracing new opportunities, change and in navigating uncertainty.
  • Actively contribute as part of a team and work towards achieving team goals and outcomes.
  • Take responsibility and demonstrate accountability in completing tasks and achieving objectives, actively seeking to resolve problems and identify opportunities.
  • Committed to customer service and placing customer satisfaction at the heart of our success to ensure we deliver against our shared goals.
  • Can work collaboratively within an evolving industry, gaining stakeholder confidence through understanding their goals and motivations and demonstrating credibility as an expert.
  • Make timely, informed decisions taking account of the benefits and constraints involved.

    Desirable

    • Experience developing and deploying APIs in Python using frameworks such as FastAPI.
    • Familiarity with cloud platforms such as AWS, Azure, or GCP.
    • Knowledge of containerization and orchestration tools like Docker and Kubernetes.
    • Experience with version control systems such as Git.
    • Understanding of Agile methodologies and experience working in Agile teams.

       

      We are committed to investing in our staff and offer formal and informal learning opportunities for development.

       

      We value  diversity and equal opportunities in employment and are committed to creating a workplace which is inclusive to everyone. As a member of the  Disability Confident Scheme, we encourage candidates with disabilities who meet the minimum criteria,  to apply for our jobs.  If you have applied under the Disability Confident Scheme, please let us know in advance by emailing Click here to contact this recruiter Find out more about Diversity and Inclusion at RSSB: Rail Safety and Standards Board Careers - VERCIDA

       

      If you require any  reasonable adjustments with respect to our selection process including information in an alternative format, please contact us at Click here to contact this recruiter

       

      We understand the importance of  work-life balance and we offer our staff the flexibility to work within our core hours and the option to vary their location between both the office and home. If you are looking for further flexibility, speak to us at interview stage so that we can consider your request.

       

      We value our staff and we offer a competitive  benefits package to ensure our staff can achieve their best throughout their journey with us. This includes 30 days annual leave (plus bank holidays); a holiday buy and sell scheme; private medical and dental cover; a season ticket loan and travel subsidy; access to a cycle to work scheme; volunteer leave; a performance related bonus and pension.