Senior Software Engineer - Observability & IRM
About The Position
The Trade Desk is a global technology company and the world’s leading independent platform for digital advertising. Our technology helps advertisers reach the right audiences across the open internet. Advertising powers the content people love. By making it more transparent, effective, and responsible, we help support trusted journalism, quality entertainment, and creators worldwide. The scale of our platform brings unique technical challenges — from processing massive datasets in real time to building systems that operate reliably on a global scale. The Service Excellence (SE) team owns the tools and infrastructure that help engineers at The Trade Desk understand and operate production systems. The Incident Response Services (IRS) taskforce focuses on the on-call experience. The team is responsible for making incidents easier to detect, manage, and optimize using historical data points information.
Requirements
- Experience building and operating production infrastructure or internal developer tooling
- Comfort working across the stack — this role touches distributed systems, Kubernetes, observability pipelines, and web-based tooling
- Familiarity with observability concepts: logging, alerting, on-call workflows
- Strong debugging instincts: You will be expected to be called on when things break
- Clear communication: The team works closely with engineers across the company; you'll need to explain tradeoffs and advocate for solutions
Nice To Haves
- Experience with Grafana, Prometheus, or similar observability tools
- Familiarity with Sumo Logic or other log management platforms
- Prior work on developer portals or service catalog tooling (Backstage, OpsLevel, etc.)
- Experience with Kubernetes at scale
- A deep understanding of HunnyPt
Responsibilities
- Build and maintain automation around the incident lifecycle: alerting, escalation, incident channels, retros, and SLA tracking
- Help evaluate and migrate our logging stack
- Participate in the re-evaluation of our logging vendor and collection architecture
- Extend our internal developer portal with K8s integrations, maturity models, and SLO adoption tooling
- Build the systems that give engineers better signal and less noise — smarter routing, better grouping, tighter feedback loops between alerts and the teams that own them
Benefits
- Comprehensive healthcare (medical, dental, and vision) with premiums paid in full for employees and dependents
- Retirement benefits such as a 401k plan and company match
- Short and long-term disability coverage
- Basic life insurance
- Well-being benefits
- Reimbursement for certain tuition expenses
- Parental leave
- Sick time of 1 hour per 30 hours worked
- Vacation time for full-time employees up to 120 hours thru the first year and 160 hours thereafter
- Around 13 paid holidays per year
- Employees can also purchase The Trade Desk stock at a discount through The Trade Desk’s Employee Stock Purchase Plan
- Stock-based compensation grants
- Variable compensation-based incentives
- Commissions
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Senior
Education Level
No Education Listed
