business requirements and drive decision-making. Your Skills and Experience Demonstrable experience using AWS Glue, AWS Lambda, Amazon Kinesis, AmazonEMR , Amazon Athena, Amazon DynamoDB, Amazon Cloudwatch, Amazon SNS and AWS Step Functions. Strong experience with modern more »
business requirements and drive decision-making. Your Skills and Experience Demonstrable experience using AWS Glue, AWS Lambda, Amazon Kinesis, AmazonEMR , Amazon Athena, Amazon DynamoDB, Amazon Cloudwatch, Amazon SNS and AWS Step Functions. Strong experience with modern more »
into AWS data stores for both batch and streaming data ingestion. • AWS Expertise: Utilize your expertise in AWS services such as AmazonEMR , S3, AWS Glue, Amazon Redshift, AWS Lambda, and more to build and optimize data solutions. • Data Modeling: Design and implement data models … to use the delta features to identify the changed records, optimization and housekeeping on Iceberg tables in the data lake. Must have: AWS, ETL, EMR, GLUE, Spark/Scala, Java, Python Good to have : Cloudera – Spark, Hive, Impala, HDFS , Informatica PowerCenter, Informatica DQ/DG, Snowflake Erwin Qualifications : • Bachelor more »
M17, Trafford Park, Trafford, Greater Manchester, United Kingdom Hybrid / WFH Options
Ikhoi Recruitment
problems in a challenging and fun environment using some of the latest Big Data open-source technologies like Apache Spark as well as Amazon Web Service technologies including ElasticMapReduce Athena and Lambda to develop scalable data solutions. The role is hybrid. Great benefits 25 days paid holiday more »
Employment Type: Permanent
Salary: £60000 - £65000/annum Bonus, Pension, Hybrid and more
into AWS data stores for both batch and streaming data ingestion. AWS Expertise: Utilize your expertise in AWS services such as AmazonEMR , S3, AWS Glue, Amazon Redshift, AWS Lambda, and more to build and optimize data solutions. Data Modeling: Design and implement data models … deliver high-quality data solutions. Automation: Implement automation processes and best practices to streamline data workflows and reduce manual interventions. Must have: AWS, ETL, EMR, GLUE, Spark/Scala, Java, Python, Good to have: Cloudera – Spark, Hive, Impala, HDFS , Informatica PowerCenter, Informatica DQ/DG, Snowflake Erwin Qualifications: Bachelor more »
AWS Certified Data Analytics – Specialty certification. Extensive experience in designing and implementing data solutions on AWS using services such as AWS Redshift, Glue, Athena, EMR, S3, RDS, and DynamoDB. Proficiency in data modelling, ETL/ELT pipelines, and data integration using tools such as AWS Glue or Apache Spark. more »
have the following: Experience in a startup environment where you’ve helped build infrastructure from the ground up. Familiarity with AWS services like S3, EMR, and technologies like Terraform and Docker. Know the ins and outs of current big data frameworks like Spark or Flink, but this is not more »
City of London, London, United Kingdom Hybrid / WFH Options
83zero Limited
across a wide range of AWS services with the ability to demonstrate working on large engagements Experience of AWS tools (e.g Athena, Redshift, Glue, EMR) Java, Scala, Python, Spark, SQL Experience of developing enterprise grade ETL/ELT data pipelines. Deep understanding of data manipulation/wrangling techniques Demonstrable … applying Data Engineering best practices (coding practices to DS, unit testing, version control, code review). Big Data Eco-Systems, Cloudera/Hortonworks, AWS EMR, GCP DataProc or GCP Cloud Data Fusion. NoSQL Databases. Dynamo DB/Neo4j/Elastic, Google Cloud Datastore. Snowflake Data Warehouse/Platform Streaming more »
Manchester, North West, United Kingdom Hybrid / WFH Options
83zero Limited
across a wide range of AWS services with the ability to demonstrate working on large engagements Experience of AWS tools (e.g Athena, Redshift, Glue, EMR) Java, Scala, Python, Spark, SQL Experience of developing enterprise grade ETL/ELT data pipelines. Deep understanding of data manipulation/wrangling techniques Demonstrable … applying Data Engineering best practices (coding practices to DS, unit testing, version control, code review). Big Data Eco-Systems, Cloudera/Hortonworks, AWS EMR, GCP DataProc or GCP Cloud Data Fusion. NoSQL Databases. Dynamo DB/Neo4j/Elastic, Google Cloud Datastore. Snowflake Data Warehouse/Platform Streaming more »
London, Farringdon Without, United Kingdom Hybrid / WFH Options
83zero Ltd
across a wide range of AWS services with the ability to demonstrate working on large engagements Experience of AWS tools (e.g Athena, Redshift, Glue, EMR) Java, Scala, Python, Spark, SQL Experience of developing enterprise grade ETL/ELT data pipelines. Deep understanding of data manipulation/wrangling techniques Demonstrable … applying Data Engineering best practices (coding practices to DS, unit testing, version control, code review). Big Data Eco-Systems, Cloudera/Hortonworks, AWS EMR, GCP DataProc or GCP Cloud Data Fusion. NoSQL Databases. Dynamo DB/Neo4j/Elastic, Google Cloud Datastore. Snowflake Data Warehouse/Platform Streaming more »
across a wide range of AWS services with the ability to demonstrate working on large engagements Experience of AWS tools (e.g Athena, Redshift, Glue, EMR) Java, Scala, Python, Spark, SQL Experience of developing enterprise grade ETL/ELT data pipelines. Deep understanding of data manipulation/wrangling techniques Demonstrable … applying Data Engineering best practices (coding practices to DS, unit testing, version control, code review). Big Data Eco-Systems, Cloudera/Hortonworks, AWS EMR, GCP DataProc or GCP Cloud Data Fusion. NoSQL Databases. Dynamo DB/Neo4j/Elastic, Google Cloud Datastore. Snowflake Data Warehouse/Platform Streaming more »
across a wide range of AWS services with the ability to demonstrate working on large engagements Experience of AWS tools (e.g Athena, Redshift, Glue, EMR) Java, Scala, Python, Spark, SQL Experience of developing enterprise grade ETL/ELT data pipelines. Deep understanding of data manipulation/wrangling techniques Demonstrable … applying Data Engineering best practices (coding practices to DS, unit testing, version control, code review). Big Data Eco-Systems, Cloudera/Hortonworks, AWS EMR, GCP DataProc or GCP Cloud Data Fusion. NoSQL Databases. Dynamo DB/Neo4j/Elastic, Google Cloud Datastore. Snowflake Data Warehouse/Platform Streaming more »
architecture frameworks such as TOGAF, Zachman, etc. Established experience with cloud services as AWS S3, EC2, RDS, Lambda, Cloud Formation, Terraform, Kinesis, Data Pipelines, EMR, Step Functions, Lambda, VPC, IAM, and Security Groups. Experience with API Management, API Gateway, etc. Experience with microservices and containerization using Dockers, ECS, and more »
or bioinformaticsExperience in regulated environmentsFamiliarity with workflow technologies like Apache Airflow or NextFlowProficiency in handling large, complex data-sets using tools like Hadoop/EMR or SparkIf you’re an experienced technologist who thrives on fostering collaboration to deliver positive outcomes, we’d love to hear from you. Join more »
Proven knowledge of Generative AI and hands-on experience of building applications with large foundation models, experiences related to AWS services such as SageMaker, EMR, S3, DynamoDB and EC2, hands-on experience of building ML solutions on AWS. Strong communication skills, with attention to detail and ability to convey more »
working with the latest cloud technologies and provides significant ownership in delivering impactful solutions. Technology Stack Our client client uses AWS with managed Kubernetes, EMR, RDS, OpenSearch, MSK, and EC2. Infrastructure is managed through Terraform, with automation handled via GitHub Actions. Responsibilities Design and manage scalable cloud infrastructure to more »
working with the latest cloud technologies and provides significant ownership in delivering impactful solutions. Technology Stack Our client client uses AWS with managed Kubernetes, EMR, RDS, OpenSearch, MSK, and EC2. Infrastructure is managed through Terraform, with automation handled via GitHub Actions. Responsibilities Design and manage scalable cloud infrastructure to more »
Manchester, North West, United Kingdom Hybrid / WFH Options
Circle Group
Data Engineer - Manchester Key words: Data Engineer, Python, AWS, Apache Spark, SQL, Big Data My client is a well-established software house in Manchester who are going through a big growth phase and as part of a large recruitment drive more »
Manchester, North West, United Kingdom Hybrid / WFH Options
Formula Recruitment Limited
Technology: AWS serverless, Terraform Python, Spark Location: Manchester (Hybrid) A leading technology provider in the market research industry is seeking a skilled Infrastructure Engineer to join their dynamic Data Engineering team. This is an exciting opportunity for those looking to more »
TypeScript, NestJS and ReactJS Experience building platforms using event-based architecture and CQRS Familiarity with AWS Services such as Aurora, MSK Kafka, ECS, and EMR Experience writing automated unit and integration tests Excellent practical experience with Docker Strong understanding of SOLID principles and design patterns Hands-on experience with more »
TypeScript, NestJS and ReactJS Experience building platforms using event-based architecture and CQRS Familiarity with AWS Services such as Aurora, MSK Kafka, ECS, and EMR Experience writing automated unit and integration tests Excellent practical experience with Docker Strong understanding of SOLID principles and design patterns Hands-on experience with more »
and minimal latency. * Security: Implement and uphold robust data security protocols to safeguard our most valuable asset-information. Key Knowledge/Experience Spark/EMR: Big data processing frameworks AWS S3: Data storage solutions AWS Redshift: Data warehousing and analytics AWS Lambda: Serverless computing for ETL processes AWS Glue … integration AWS RDS: Managed relational databases AWS Lake Formation: Data lake management and governance Python/SQL: Programming for data processing and querying Amazon Kinesis: Real-time data streaming dbt (Data Build Tool): Data transformations in warehousing Data Governance Tools: Understanding of AWS Data Catalog or Alation Data more »
with AWS services, particularly S3, Redshift, Lambda, Glue, and RDS. Data Modeling: Exceptional skills in data modeling, ETL processes, and data warehousing. Spark/EMR: Big data processing frameworks. AWS S3: Data storage solutions. AWS Redshift: Data warehousing and analytics. AWS Lambda: Serverless computing for ETL processes. AWS Glue … integration. AWS RDS: Managed relational databases. AWS Lake Formation: Data lake management and governance. Python/SQL: Programming for data processing and querying. Amazon Kinesis: Real-time data streaming. dbt (Data Build Tool): Data transformations in warehousing. Data Governance Tools: Understanding of AWS Data Catalog or Alation. Data more »
Manchester Area, United Kingdom Hybrid / WFH Options
Formula Recruitment
Data Infrastructure Engineer Salary: Up to £65,000 Technology: AWS serverless, Terraform Python, Spark Location: Manchester (Hybrid) A leading technology provider in the market research industry is seeking a skilled Infrastructure Engineer to join their dynamic Data Engineering team. This more »
