Flink Manager

Weekday AI
$10,000,000 - $40,000,000

About The Position

As a Flink Manager, you will be responsible for driving real-time streaming initiatives, managing high-performing engineering teams, and architecting scalable data solutions that power mission-critical business applications. You will collaborate closely with product, analytics, infrastructure, and leadership teams to translate business requirements into resilient streaming architectures.

Requirements

  • 8–12 years of overall experience in data engineering, distributed systems, or backend engineering.
  • Strong hands-on expertise in Apache Flink (minimum 3–5 years preferred).
  • Deep understanding of stream processing concepts including event-time semantics, watermarking, state backends, and exactly-once guarantees.
  • Proven experience designing and managing high-throughput, low-latency Flink pipelines.
  • Strong programming skills in Java or Scala for Flink development.
  • Experience with distributed messaging systems (e.g., Kafka) and data storage systems.
  • Familiarity with cluster resource management frameworks (YARN, Kubernetes, or similar).
  • Strong debugging, troubleshooting, and performance tuning skills specific to Flink workloads.
  • Experience leading engineering teams and managing cross-functional stakeholders.

Nice To Haves

  • Experience deploying Flink in cloud environments (AWS, Azure, or GCP).
  • Knowledge of batch and stream unification architectures.
  • Exposure to big data ecosystems including Hadoop or Spark.
  • Understanding of CI/CD practices for streaming applications.
  • Experience implementing real-time analytics, fraud detection, personalization, or IoT streaming use cases.

Responsibilities

  • Lead the design, development, and deployment of real-time data processing pipelines using Apache Flink.
  • Architect scalable, fault-tolerant, and low-latency streaming systems leveraging Flink’s DataStream and Table APIs.
  • Manage end-to-end lifecycle of Flink applications including development, testing, optimization, monitoring, and production support.
  • Drive best practices for state management, checkpointing, windowing, event-time processing, and fault tolerance in Flink.
  • Optimize Flink job performance, resource allocation, and cluster tuning for high-throughput environments.
  • Oversee integration of Flink with distributed systems such as Kafka, data lakes, warehouses, and microservices.
  • Lead and mentor a team of data engineers, conducting code reviews and enforcing engineering excellence standards.
  • Collaborate with DevOps teams to manage Flink deployments on cloud and on-prem environments.
  • Establish monitoring, logging, and alerting mechanisms for streaming pipelines to ensure high availability and reliability.
  • Contribute to roadmap planning for streaming and real-time analytics platforms.
© 2024 Teal Labs, Inc
Privacy PolicyTerms of Service