/CD pipelines. Big Data & Data Engineering: Strong background in processing large datasets and building data pipelines using platforms like Apache Spark , Databricks , ApacheFlink , or similar big data tools. Experience with batch and stream processing. Security: In-depth knowledge of security practices in cloud environments, including identity management More ❯
advanced analytics infrastructure. Familiarity with infrastructure-as-code (IaC) tools such as Terraform or CloudFormation. Experience with modern data engineering technologies (e.g., Kafka, Spark, Flink, etc.). Why join YouLend? Award-Winning Workplace: YouLend has been recognised as one of the “Best Places to Work 2024” by the Sunday More ❯
Chantilly, Virginia, United States Hybrid / WFH Options
Aerospace Corporation
teams toward software development best practices Experience in SQL, NoSQL, Cypher and other big data querying languages Experience with big data frameworks (Hadoop, Spark, Flink etc.) Experience with ML lifecycle management tools (MLflow, Kubeflow, etc.) Familiarity with data pipelining and streaming technologies (Apache Kafka, Apache Nifi, etc.) Demonstrated contributions More ❯
various applications. MLOps and Automation : Proficiency in Docker, Kubernetes, MLflow, Kubeflow , and Seldon for scaling AI operations. Real-Time Processing : Hands-on experience with Flink, Kafka, and event stream processing for dynamic data workflows. Soft Skills: Strong problem-solving and critical-thinking skills. Ability to collaborate effectively across teams. More ❯
record of building and managing real-time data pipelines across multiple initiatives. Expertise in developing data backbones using distributed streaming platforms (Kafka, Spark Streaming, Flink, etc.). Experience working with cloud platforms such as AWS, GCP, or Azure for real-time data ingestion and storage. Programming skills in Python More ❯
Arlington, Virginia, United States Hybrid / WFH Options
Full Visibility LLC
orchestration (e.g., Apache Airflow, Luigi, Prefect) Strong programming skills in Python, SQL, or Scala Experience with open-source data processing tools (e.g., Kafka, Spark, Flink, Hadoop) Familiarity with database technologies (PostgreSQL, MySQL, or NoSQL solutions) Ability to work in a fast-paced environment with large-scale datasets Preferred: • Experience More ❯
managing real-time data pipelines across a track record of multiple initiatives. Expertise in developing data backbones using distributed streaming platforms (Kafka, Spark Streaming, Flink, etc.). Experience working with cloud platforms such as AWS, GCP, or Azure for real-time data ingestion and storage. Ability to optimise and More ❯
programming languages. Expertise in cloud (GCP, AWS), containerisation, and infrastructure-as-code (Docker, Kubernetes, Terraform). Experience building data platforms using Spark, Trino, Presto, Flink, or similar, with a focus on data quality, SQL performance tuning, and data warehousing principles. Understanding of modern data architecture with experience implementing data More ❯
oriented programming (OOP) principles & concepts Familiarity with advanced SQL techniques Familiarity with data visualization tools such as Tableau or Power BI Familiarity with ApacheFlink or Apache Storm Understanding of DevOps practices and tools for (CI/CD) pipelines. Awareness of data security best practices and compliance requirements (e.g. More ❯
systems, with proficiency in networking, multi-threading and implementation of REST APIs Experience with the Spring framework, messaging frameworks (Kafka, RabbitMQ), streaming analytics (ApacheFlink, Spark), management of containerized applications (Kubernetes). Experience with Enabling tools (Git, Maven, Jira), DevOps (Bamboo, Jenkins, GitLab Cl/Pipelines), Continuous Monitoring (ELK More ❯
utilizing Databricks or Apache Spark Experience designing and maintaining Data Lakes or Data Lakehouses Experience with big data tools such as Spark, NiFi, Kafka, Flink, or others at multi-petabyte scale Expertise in designing and maintaining ETL/ELT data pipelines utilizing storage/serialization formats/schemas such More ❯
of the field will drive your journey.Requirements: - Expert in batch, distributed data processing and near real-time streaming data pipelines with technologies like Kafka, Flink, Spark etc. Experience in Databricks is a plus.- Experience in Data Lake/Big Data Analytics platform implementation with cloud based solution; AWS preferred. More ❯
of the field will drive your journey.Requirements: - Expert in batch, distributed data processing and near real-time streaming data pipelines with technologies like Kafka, Flink, Spark etc. Experience in Databricks is a plus.- Experience in Data Lake/Big Data Analytics platform implementation with cloud based solution; AWS preferred. More ❯
as Looker, Tableau, or Apache Superset. Understanding of software engineering best practices and their application to data processing systems. Knowledge of Python, Java or Flink is a plus, but not a necessity. Strong attention to detail. Ability to work autonomously in a fast-paced and dynamic environment. Strong communication More ❯
engineering experience Experience with data modeling, warehousing and building ETL pipelines Bachelor's degree Knowledge of batch and streaming data architectures like Kafka, Kinesis, Flink, Storm, Beam Knowledge of distributed systems as it pertains to data storage and computing Experience programming with at least one modern language such as More ❯
Key Technologies We Use (not necessarily required for the role): Google Cloud, Google Cloud Composer, BigQuery, Spark, Solr, Elasticsearch, Druid, PostgreSQL, ScyllaDB, Redis, Kafka, Flink, Docker, Kubernetes, Kibana, Jenkins, Prometheus, Grafana, Github, C++, Python, Scala, Compiler Explorer What Blis Can Offer: We want you to be well and thrive More ❯
tools such as Git/GitLab, NPM, Jira Experience with designing, publishing and consuming GraphQLAPI using Node.js and stream processing frameworks such as ApacheFlink/Kafka or SparkStream is preferred Experience integrating with relational, key-value, document or graph databases Experience with secure coding practices (OWASP) and accessibility More ❯
development experience using SQL. Hands-on experience with MPP databases such as Redshift, BigQuery, or Snowflake, and modern transformation/query engines like Spark, Flink, Trino. Familiarity with workflow management tools (e.g., Airflow) and/or dbt for transformations. Comprehensive understanding of modern data platforms, including data governance and More ❯
and Redis. In-depth knowledge of ETL/ELT pipelines, data transformation, and storage optimization. Skilled in working with big data frameworks like Spark, Flink, and Druid. Hands-on experience with both bare metal and AWS environments. Strong programming skills in Python, Java, and other relevant languages. Proficiency in More ❯
including RDS, DynamoDB, S3 and Glue ETL Experience designing and implementing event based stream processing solutions using technologies such as Kafka, Kinesis, Spark and Flink Ability to design and implement high availability, multi-region data replication for mission critical applications Experience designing and implementing data management solutions that enable More ❯
including RDS, DynamoDB, S3 and Glue ETL Experience designing and implementing event based stream processing solutions using technologies such as Kafka, Kinesis, Spark and Flink Ability to design and implement high availability, multi-region data replication for mission critical applications Experience designing and implementing data management solutions that enable More ❯
including RDS, DynamoDB, S3 and Glue ETL Experience designing and implementing event based stream processing solutions using technologies such as Kafka, Kinesis, Spark and Flink Ability to design and implement high availability, multi-region data replication for mission critical applications Experience designing and implementing data management solutions that enable More ❯
solutions. Exposure to some of the following technologies: Python, AWS Redshift, AWS Athena/Apache Presto, Big Data technologies (e.g S3, Hadoop, Hive, Spark, Flink, Kafka etc), NoSQL systems like Cassandra, DBT is nice to have. What you'll get: Full responsibility for projects from day one, a collaborative More ❯
experience in cross-functional teams and able to communicate effectively about technical and operational challenges. Preferred Qualifications: Proficiency with scalable data frameworks (Spark, Kafka, Flink) Proven Expertise with Infrastructure as Code and Cloud best practices Proficiency with monitoring and logging tools (e.g., Prometheus, Grafana) Working at Lila Sciences, you More ❯
experience in cross-functional teams and able to communicate effectively about technical and operational challenges. Preferred Qualifications: Proficiency with scalable data frameworks (Spark, Kafka, Flink) Proven Expertise with Infrastructure as Code and Cloud best practices Proficiency with monitoring and logging tools (e.g., Prometheus, Grafana) Working at Lila Sciences, you More ❯