Key responsibilities: Develop robust architectures and designs for big data platform and applications within the Apache Hadoop ecosystem. Implement and deploy big data platform and solutions on-premises and in hybrid cloud environments. Read, understand, and modify open-source code to implement bug fixes and perform upgrades. Ensure all … applications. Your Profile Key Skills/Knowledge/Experience: Proven experience in architecting, designing, building, and deploying big data platforms and applications using the Apache Hadoop ecosystem in hybrid cloud and private cloud scenarios. Experience with hybrid cloud big data platform designs and deployments, especially in AWS, Azure, or … Google Cloud Platform. Experience in large-scale data platform builds and application migrations. Expert knowledge of Apache Hadoop ecosystem and associated Apache projects (eg, HDFS, Hive, HBase,... more »
Required Skills: * Native on Linux command line. * Python web development - specifically experience with aiohttp * JavaScript/CSS/HTML/SSL * MySQL * Debian/Apache/NGINX server admin experience * Evidence of engaging with open source projects communities. * Competent at using Git for source control. * Passionate about open source more »
month and possibility of extension Looking for an enthusiastic Spark Scala Engineer who will be responsible for designing, building, and maintaining data pipelines using Apache Spark and Scala. This includes tasks like: Extracting data from various sources (databases, APIs, files) Transforming and cleaning the data Loading the data into … Big Data technologies: You'll likely work with various Big Data technologies alongside Spark, including: Hadoop Distributed File System (HDFS) for storing large datasets Apache Kafka for Real Time data streaming Apache Hive for data warehousing on top of HDFS Cloud platforms like AWS, Azure, or GCP for more »
Modelling. Experience with at least one or more of these programming languages: Python, Scala/Java Experience with distributed data and computing tools, mainly Apache Spark & Kafka Understanding of critical path approaches, how to iterate to build value, engaging with stakeholders actively at all stages. Able to deal with more »
Flask, Tornado or Django, Docker Experience working with ETL pipelines is desirable e.g. Luigi, Airflow or Argo Experience with big data technologies, such as Apache Spark, Hadoop, Kafka, etc. Data acquisition and development of data sets and improving data quality Preparing data for predictive and prescriptive modelling Hands on more »
and Saltstack CI/CD: Jenkins, GitLab CI/CD Data/Messaging: Amazon Aurora (Postgres), ElastiCache (Redis), AmazonMQ (RabbitMQ) API: Tyk API Gateway, Apache Monitoring/Logging: Datadog and SumoLogic Security: IAM (Identity Access Management), Security Groups, mTLS Other: VPC and general networking In return, they would be more »
London, England, United Kingdom Hybrid / WFH Options
McGregor Boyall
Work with Hadoop, Spark, and other platforms for large-scale data processing. Real-Time Data Streaming: Develop and manage pipelines using CDC, Kafka, and Apache Spark. Database Management: Handle SQL databases like Oracle, MySQL, or PostgreSQL. Data Governance: Ensure data quality, security, and compliance with best practices. Ideal Candidate more »
. • Troubleshooting networks issues (tcpdump/Wireshark). • Scripting capabilities (SH/Bash/Python/Perl). • Configuration of common services (DNS/Apache/NGINX/Postfix/Squid/SSH/iptables). • Understanding of clustering services, enabling High Availability failover. • Experience with enterprise hardware and more »
data components such as Azure Data Factory, Azure SQL DB, Azure Data Lake, etc. Strong Python and SQL skills for data manipulation Experience with Apache Spark and/or Databricks. Familiarity with BI visualization tools like Power BI Experience in managing end-to-end analytics pipelines (batch and streaming … such as Azure Data Engineer Associate are desirable. Knowledge of data ingestion methods for real-time and batch processing Proficiency in PySpark and debugging Apache Spark workloads. What’s in it for you? Annual bonus scheme – up to 10% Excellent pension scheme Flexible working Enhanced family friendly policies Ongoing more »
Senior QA Automation Tester - Cypress Job Title: QA Automation Engineer Location: South West London Salary: £60,000 per year + Benefits Job type: Permanent Role: Hybrid A long established Fintech company based in London, renowned for its cutting-edge payment more »
Data Engineer to join our team and build robust data pipelines. Responsibilities: Build, and maintain scalable data pipelines. Maintain the pipeline through Databricks and Apache Beam Seamless data integration and quality. Data storage solutions using GCP (will still be considered with other cloud experience) Best practices in data management more »
within the EU Fusion programme and connections to international HPC communities, showcasing contributions made to the field. Experience in workflow management systems such as Apache Airflow. Familiarity with Research Data Management methodologies,modern database technologies including SQL, NoSQL and Graph Databases, and parallel file access technologies such as MPI more »
pipelines Know your way around Unix based operating system Experience working with any major cloud provider (AWS, GCP, Azure) Fluency in English Experience using Apache Airflow Experience using Docker Experience using Apache Spark Benefits: Salary £40-50K per annum dependant on skills and experience 25 Days annual more »
Linux environments. Knowledge of data modeling, database design, and query optimization techniques. Experience with real-time data processing, streaming, and analytics technologies (e.g., Kafka, Apache Flink). Familiarity with financial markets, trading systems, and quantitative analysis is a plus. Excellent problem-solving, analytical, and communication skills, with the ability more »
Role : PHP Developer 🗺️ Location : Dudley - 2 days a week in office 💰 Salary : £35,000 - £45,000 ✅ Skills : PHP, Azure, Apache, CentOS, T-SQL My client is expanding and looking for a PHP Developer to join their growing team in the Dudley area. You'll be responsible, for the development … features Troubleshooting web applications Maintaining clear documentation of all work Qualifications 3+ years of experience with hands-on PHP programming Experience with IDE, Azure, Apache, CentOS, T-SQL What's in it for you: Starting salary up to £45,000 Continuous training and development Flexible working Please note : the more »
GoLang. - Significant experience with Hadoop, Spark and other distributed processing platforms and frameworks. - Experience working with Open table/storage formats like delta lake, apache iceberg or apache hudi. - Experience of developing and managing real time data streaming pipelines using Change data capture (CDC), Kafka and Apachemore »
Recent and proven experience of using Red Hat Linux (or others Unix flavours) including scripting in a commercial environment Experience supporting applications (Java, .NET, Apache, IIS); Desirable: Knowledge of Microsoft Windows Server. Experience with working within financial industry; Experience with working within an ITIL framework; Experience with working with more »
Southampton, Hampshire, South East, United Kingdom
Ordnance Survey Limited
to join Ordnance Survey's Data Derivation team in developing our next generation geospatial data. Using big data technologies (primarily Databricks, an implementation of Apache Spark) you will develop, support, and maintain automated processes that enrich and transform Ordnance Survey's large-scale data. Working in a forward-thinking … testing approaches and best practice Providing technical coaching and mentoring Advocating software engineering industry best practice Interpreting detailed specifications and translating into simple solutions Apache Spark/Databricks experience (desirable) Experience of working in Microsoft Azure (desirable) Experience of working with the Scala language (desirable) The rewards We want more »
Telford, Shropshire, West Midlands, United Kingdom
RECRUIT123 LIMITED
Caching (Redis) It's an advantage if you also have experience with: JavaScript frameworks such as Vue/React or similar Linux server management Apache or Nginx DNS, SSL & Domain management What the role involves: Being part of a development team that is responsible for all aspects of ongoing more »
Leatherhead, Surrey, South East, United Kingdom Hybrid / WFH Options
RINA
experiencing a real breadth and variety of project work on some of the most technically advanced platforms in UK Defence, including unmanned air systems, Apache, Wildcat, Chinook and Typhoon, to name a few. The successful candidate support and deliver Systems Engineering and Assurance projects, for technically interlinked assets and more »
for meetings, teaching and webinars using Microsoft Teams or Zoom; Administration of Microsoft's SQL Server or MySQL; Lecture capture; Configuring Microsoft IIS or Apache; Content Management Systems e.g. Drupal; Crestron, Extron or Q-SYS control systems for AV; Similar responsibilities in the higher education sector. To Apply If more »
Derby, Derbyshire, East Midlands, United Kingdom Hybrid / WFH Options
Devtank Ltd
Required Skills: * Native on Linux command line. * Python web development - specifically experience with aiohttp * JavaScript/CSS/HTML/SSL * MySQL * Debian/Apache/NGINX server admin experience * Evidence of engaging with open source projects communities. * Competent at using Git for source control. * Passionate about open source more »
Employment Type: Permanent, Part Time, Work From Home
Birmingham, West Midlands, United Kingdom Hybrid / WFH Options
becg
and Linux terminal tools Experience writing tests in PHP Unit or similar Experience with Ubuntu or other Linux distributions Experience in configuring Nginx or Apache for security Strong problem-solving skills and ability to troubleshoot and debug code Excellent communication skills with the ability to relay technical information to more »
Cambridge, Cambridgeshire, United Kingdom Hybrid / WFH Options
Pure Resourcing Solutions Limited
updates, allocation of required disk and memory resources), backups, configuration and security of networks and Linux systems - Application support: configuration of web hosting services (ApacheHTTP servers), database maintenance and ideally also CI pipelines definition (we use Bitbucket) - Business operations support: Google workspace Drive and Device management. Other duties … principles, standards, and best practices. Experience with ISO 27001 is highly advantageous. - Knowledge of web hosting technologies highly advantageous, including any of the following- Apache, Nginx, MySQL, MongoDB, Django and PWAs, their rolling updates, red/black deployments and roll-backs. - Experience with virtualisation technologies such as Docker and more »
Employment Type: Permanent
Salary: £50000 - £70000/annum Hybrid working - scale-up business