Sr. Software Engineer, Observability - Slack

Slack•Seattle, WA

1d•$172,500 - $285,800•Hybrid

About The Position

Salesforce is seeking a Sr. Software Engineer, Observability to join the Monitoring Infrastructure team within the Service Delivery Platform & Reliability group at Slack. This team develops platforms and tools that generate telemetry, provide insights, and enhance observability in Slack's production services, with a strong emphasis on performance and reliability. Responsibilities include developing management tools for distributed applications and infrastructure, maintaining datasets for performance and system analytics, and building interfaces and backend systems to analyze user and system behavior. The role involves working with open-source observability/monitoring technologies like Astra and Prometheus, cloud providers such as AWS, and developing software using Go, Python, or Java. The engineer will focus on log pipelines, collaborate with other engineering, product development, and customer experience teams to provide insights for decision-making and ensure a positive user experience. The role also includes building and maintaining distributed services that process millions of data points per second, with self-healing and scaling capabilities. The team values inclusivity and empathy for colleagues and customers.

Requirements

Strong communication skills, with the ability to explain complex technical concepts to designers, support, and other engineers.
Enjoyment in onboarding new team members, mentoring, and teaching others.
Proficiency in unit tests, code review, design documentation, debugging, and problem-solving.
Deep curiosity about how systems work internally.
Motivation driven by helping others succeed.
Eagerness and ability to fix issues when they arise.
Interest in improving efficiency or enhancing coworker experience.
Commitment to improving system performance through data-informed decisions.
Deep understanding of security concepts and ability to implement them to protect users and the company.
Experience with Go, Python, or Java.
Experience with open-source observability/monitoring technologies like Astra and Prometheus.
Experience with cloud providers such as AWS.

Nice To Haves

Passion for data visualization, graphing, and maximizing signal versus noise.
Experience with Elasticsearch, Logstash, and Kibana.

Responsibilities

Build, maintain, and ensure timely delivery of high-volume event log pipelines.
Create libraries, tools, and automation to ensure critical event data reaches the correct destination.
Foster a culture of Observability at Slack by identifying problem areas and consulting on improving system visibility.
Prototype tooling interfaces or develop new features for engineering use cases.
Enhance auto-remediation in logging infrastructure to prevent recurring failures.
Educate engineers and customer experience agents on using tools to introspect their systems.
Collaborate with the Platform team to integrate observability solutions, enabling service teams to make data-driven decisions on operations, cost optimization, and scaling.
Participate in the Monitoring Infrastructure on-call rotation, triaging and addressing production issues.