Sr Staff Software Engineer (Distributed Systems | Data Ingestion | Storage | Analytics)

ServiceNow•San Diego, CA

11h

About The Position

The Sr Staff Software Engineer will provide senior technical leadership for ServiceNow’s Health Log Analytics (HLA), a high-volume operational-log data platform that supports the company’s AIOps suite and large enterprise customers. HLA covers the data path from collection and ingestion through routing, enrichment, durable storage, retention, and cost-aware querying. Its ingestion builds on OpenTelemetry, and its log storage is Rust-based. The role partners with engineering and product leadership to shape HLA’s roadmap and architecture across ingestion, storage, and query. The engineer will work with customers, turn feedback into product direction, communicate and demonstrate architectural ideas through prototypes, establish reliability and efficiency standards, guide complex transitions, and define boundaries and contracts across teams. The role also promotes evidence-based use of AI, sets a strong quality and incident-learning example, and mentors senior engineers and technical leads.

Requirements

Experience leveraging AI or critically assessing how to integrate it into work processes, decision-making, or problem-solving, including AI-powered tools, automated workflows, AI-driven insights, or AI’s impact on infrastructure.
At least 12 years building and operating production distributed systems, with substantial ownership of large-scale data or platform systems; equivalent depth through sustained technical leadership is also accepted.
Broad grounding in data ingestion, storage, query, and AI on top of data, with deep expertise in at least one area.
Experience with ingestion tools such as Kafka, Pulsar, or Kinesis, and with columnar stores, time-series databases, Elasticsearch/OpenSearch, or object-storage designs.
A record of influencing the direction of teams outside one’s ownership through credibility and clear thinking.
Evidence-based judgment about AI, including understanding where it is appropriate and building safeguards for correctness, safety, and cost; experience shipping with AI or a strong, well-reasoned point of view.
Good judgment about tradeoffs involving consistency, latency, cost, and operational burden.
Comfort with Kubernetes and cloud infrastructure at scale.
Fluency in at least one of Go, Java, Scala, Rust, or Python, with the ability to prototype, conduct deep reviews, and write code when needed.
Clear communication skills, including the ability to explain complex designs to both new hires and executives.
Experience working remotely and leading remote teams.
Humility, care for people, and a collaborative approach that shares work and credit.
Ability to operate when problems are not fully defined and help define them.
Ability to turn small, messy problems into robust solutions and care for the people who operate what is built.

Nice To Haves

Background in observability, log analytics, OpenTelemetry, or AIOps.
Experience at an AI-native company or building agents and LLM-based features.
Experience with multitenant SaaS and per-tenant isolation.

Responsibilities

Shape the HLA roadmap in partnership with engineering, product, and leadership, balancing vision with practical constraints.
Work directly with customers and translate their changing needs and feedback into actionable product and roadmap input.
Set architectural direction across ingestion, storage, and query, including across teams outside the role’s direct ownership.
Document technical viewpoints and build prototypes that help others evaluate and support proposed approaches.
Challenge unsupported AI ideas and designs or timelines that do not fit current constraints, while recognizing when constraints should be pushed.
Lead deep design reviews that improve technical quality and help participants develop.
Define reliability and efficiency standards for the HLA data path, including completeness, durability, latency, availability, tenant isolation, and unit cost.
Lead technical approaches to difficult transitions such as storage migrations, schema changes, replay, and upgrades with little or no downtime.
Set architectural boundaries and contracts for teams working on collection, ingestion, storage, anomaly detection, query, and AIOps experiences.
Evaluate AI ideas against customer value, data quality, reliability, and cost, and run small experiments to determine what to scale.
Set an example in software quality, on-call practices, and learning from incidents.
Mentor senior engineers and tech leads and help develop future technical leaders.
Partner with engineering and product leadership and learn openly.