Principal Observability Automation Engineer

Q2 Software • Austin, TX

5d • Hybrid

About The Position

Most engineering roles hand you a slice. A product owner sets the “what,” a manager sets the “how,” and you build your piece inside someone else’s sprint. This is not that role. We don’t build the product our customers log into — we build the factory that builds it, and the systems that detect the moment anything starts to break. When our tooling works, the entire organization finds and fixes customer problems faster. When it doesn’t, everyone feels it. That is the leverage of this role, and it is why the work sits at the center of how we run reliability. You’ll be handed real business problems — often ambiguous, sometimes dropped on us overnight — and trusted to solve them end to end: shadow the people living the problem, prototype, write the architecture, socialize it with senior and executive leaders, then build it and operationalize it so the team can run it. You’ll work full-stack, fly independently, and own what you ship. You’ll build things that matter and get used across the entire company. If optimizing a single bolt to a tenth of a millimeter for the next four months sounds like a good year, this isn’t your role. If you’d rather build the thing that goes to Mars, keep reading.

Requirements

• Bachelor’s degree in Computer Science or related field with 15 years of experience in software engineering, systems automation, or observability tooling; or an advanced degree with 12+ years of experience; or equivalent related work experience
• Expertise in C# (preferred), Go, and scripting languages (Python, Bash, PowerShell)
• Deep understanding of RESTful APIs, microservices architecture, and distributed systems
• Proficient in SQL and NoSQL databases (MS SQL Server, Postgres)
• Strong experience with monitoring and analytics platforms (Grafana, Splunk)
• Hands-on experience with CI/CD tools and practices
• Advanced knowledge of Linux (Red Hat/CentOS) and Windows Server administration
• Experience with containerization (Docker), orchestration, and infrastructure automation (Terraform)
• Proven ability to lead complex technical projects and influence cross-functional teams.
• Excellent communication and leadership skills
• This position requires fluent written and oral communication in English.
• Applicants must be authorized to work for any employer in the U.S. We are unable to sponsor or take over sponsorship of an employment Visa at this time.

Nice To Haves

• Experience with NGINX, IIS, and public cloud datacenter operations preferred

Responsibilities

• Translate business into solutions — turn ambiguous business needs into architected, detectable, automatable systems — from whiteboard to prototype to production.
• Extend our agentic AI platform — develop and grow our proprietary, cloud-hosted platform — including locally-hosted models — that powers observability and automated issue detection across the organization.
• Architect at enterprise scale — design 24/7, cross-region-resilient, business-critical tooling used by teams across the company.
• Work across the organization — partner with other principal engineers, support, security, and platform teams to map how systems integrate (CRM / case data, endpoint and security telemetry, cloud infrastructure) and design for the real environment.
• Modernize high-risk pathways — re-architect legacy access and operations pathways into secure, controlled, self-service tooling with the right guardrails built in.
• Multiply the team — mentor and up-level a growing bench of junior and mid-level engineers: decompose big problems, hand off well-scoped pieces, and hold the quality bar so what ships is simple, correct, and aimed at the business need.
• Own it end to end — including light, event-driven on-call for what you build. This is not a rotation — you’re reachable when something you own breaks, which is rare (typically a handful of times a year).
• Architect and implement advanced observability and automation frameworks across cloud and on-prem environments (AWS, Azure, GCP, VMware)
• Lead the development of full-stack applications and APIs that enable automated data collection, correlation, and visualization
• Define and enforce best practices for system design, code quality, security, and performance
• Collaborate with engineering, DevOps, and product teams to identify automation opportunities and deliver scalable solutions
• Drive continuous improvement in monitoring, alerting, and incident response systems
• Mentor senior and mid-level engineers, fostering a culture of technical excellence and innovation
• Own the lifecycle of observability tools and platforms, including CI/CD pipelines, configuration management, and infrastructure as code
• Ensure compliance with security, availability, confidentiality, and privacy policies
• Participate in strategic planning, roadmap development, and technical decision-making

Benefits

• Health & Wellness
• Hybrid Work Opportunities
• Flexible Time Off
• Career Development & Mentoring Programs
• Health & Wellness Benefits, including competitive health insurance offerings and generous paid parental leave for eligible new parents
• Community Volunteering & Company Philanthropy Programs
• Employee Peer Recognition Programs – “You Earned it”

What This Job Offers

• Job Type: Full-time
• Career Level: Principal