Senior DevOps Engineer

MarketAxessNew York, NY
4hHybrid

About The Position

We are seeking a hands-on Senior DevOps Engineer to join our Data and Analytics Platform team, driving the design and implementation of our next-generation cloud-based, streaming Data Mesh platform. This role is pivotal for enabling teams to create scalable, data-driven products and advancing our application modernization strategy.

Requirements

  • 8+ years of experience in DevOps/Platform Engineering roles within a data operations team
  • Cloud & Infrastructure: Extensive experience with AWS, particularly EKS and EKS Anywhere
  • Deep expertise in Kubernetes and container orchestration
  • Strong background in Infrastructure as Code using Terraform and Terragrunt
  • Experience with GitOps practices using ArgoCD
  • Knowledge of distributed systems and event-driven architectures
  • Strong knowledge of logging, monitoring, APM tracing and alerting tools (ELK, Datadog, Prometheus, Grafana, etc.)
  • Development & Engineering: Proficient in Java development and JVM optimization
  • Experience with build tools (Maven/Gradle) and dependency management
  • Strong background in unit testing frameworks and testing practices
  • Expertise in Git version control and collaborative development
  • Experience with Apache Kafka and Kafka Streams
  • Background in monitoring and performance optimization for Java applications
  • CI/CD & Automation: Experience designing and managing Jenkins/Github/Gitlab pipelines
  • Strong scripting skills (Python, Bash)
  • Understanding of DevOps practices and automation principles

Nice To Haves

  • Experience with data mesh or similar distributed data architectures
  • Relevant cloud certifications
  • Experience building developer platforms or portals
  • Background in DataOps practices

Responsibilities

  • Design and Implement Cloud Infrastructure: Architect and deploy scalable cloud infrastructure using Infrastructure as Code (IaC) principles to support our Data Mesh platform. Experience with major cloud providers (AWS/Azure) and modern IaC tools (Terraform, CloudFormation) is essential.
  • Build Automation Pipelines: Develop and maintain CI/CD pipelines for data products, ensuring reliable deployment, testing, and monitoring of data infrastructure and applications. Experience with tools like Jenkins, GitLab CI, or GitHub Actions is required.
  • Implement Platform Observability: Design and deploy comprehensive monitoring, logging, and alerting solutions across the data platform. Experience with tools like Datadog, Prometheus, Grafana, ELK stack, or similar technologies is crucial.
  • Enable Self-Service Infrastructure: Develop automated provisioning systems for data infrastructure
  • Create and maintain infrastructure templates and patterns
  • Implement GitOps workflows for infrastructure management
  • Ability to leverage GenAI tools (e.g., GitHub Copilot or equivalent) to accelerate development and improve code quality.
  • Skill in applying AI-driven practices such as prompt engineering and AI-assisted code reviews to boost productivity and reduce delivery time while mitigating risks.
  • Build developer portals and self-service tooling
  • Security and Compliance Automation: Implement security controls and compliance checks as code
  • Automate security scanning and vulnerability management
  • Design and implement IAM and access management solutions
  • Maintain audit trails and compliance reporting automation
  • Performance Optimization: Monitor and optimize platform performance
  • Implement auto-scaling solutions design for high availability and disaster recovery
  • Optimize cost through infrastructure efficiency
  • Lead Platform Re-Architecture: Transform our existing data platform architecture to align with Data Mesh principles
  • Design and implement our Apache Iceberg adoption strategy, enabling ACID transactions and time travel capabilities
  • Create standardized patterns for teams to effectively adopt and optimize Iceberg tables
  • Drive Release Excellence: Establish automated release processes that balance agility with platform stability
  • Build deployment frameworks with comprehensive validation, monitoring, and rollback capabilities
  • Implement coordinated release strategies across distributed domains
  • Enable Technical Success: Design reference architectures and implementation patterns for domain teams
  • Lead technical design reviews and architecture governance
  • Provide technical guidance during the transition to Data Mesh architecture

Benefits

  • Hybrid Environment: Our employees enjoy a mix of working in the office and from home
  • Free Food: We provide free lunch for employees when they are working in the office. Plus, our offices are stocked with snacks
  • Paid Time Off: Competitive PTO package including vacation and personal days, sick leave and charity days
  • Generous Parental Leave: Up to 20 weeks fully paid leave
  • 401(k): Dollar-for-dollar employer match up to $17,500
  • Employee Stock Purchase Plan: Employees can purchase MarketAxess common stock at a discount
  • Wellness Stipend: We provide employees with up to $1K annually towards gym memberships, home office equipment and more
  • Onsite Healthcare: We offer convenient access to world-class care through Mount Sinai at our Hudson Yards location
  • Tuition Assistance and Professional Development: Benefit from live and on-demand learning, role-specific training, employee-led Lunch and Learns and guest speakers
  • Core benefits: Highly competitive medical, dental, and vision programs
© 2024 Teal Labs, Inc
Privacy PolicyTerms of Service