Lead Infrastructure Engineer-Observability

Truist Financial Corporation•Atlanta, GA

42d•Onsite

About The Position

We are seeking a highly skilled and forward-thinking lead observability engineer to architect, implement, and evolve enterprise-grade observability capabilities across the Truist technology landscape. This role will drive the design and adoption of a modern, scalable observability platform rooted in OpenTelemetry (Otel) and enriched by complementary technologies including Prometheus, Grafana, Jaeger, and commercial APM solutions. You will lead the strategy for metrics, traces, and synthetic monitoring - enabling end-to-end visbility, accelerated incident response, and a frictionless developer experience. In this role, you'll champion a shift from reactive monitoring to proactive, intelligence-driven observability. You'll lead efforts to standardize telemetry pipelines, embed observability into CI/CD workflows, and integrate signal-based insights into reliability, performance, and business outcomes. Success in this position means reducing mean-time-to-detect (MTTD), accelerating root cause analysis, and creating a resilient, insight-rich environment that empowers engineering teams to deliver with confidence. For this opportunity, Truist will not sponsor an applicant for work visa status or employment authorization, nor will we offer any immigration-related support for this position (including, but not limited to H-1B, F-1 OPT, F-1 STEM OPT, F-1 CPT, J-1, TN-1 or TN-2, E-3, O-1, or future sponsorship for U.S. lawful permanent residence status.) This position is office-centric 5 days a week in either our Atlanta or Charlotte Office

Requirements

Bachelor's degree and five years of experience in development or application support or an equivalent combination of education and work experience.
In-depth knowledge in information systems and ability to identify, apply, and implement best practices.
Understanding of key business processes and competitive strategies related to the IT function.
Ability to plan and manage projects.
Ability to solve complex problems by applying best practices.
Ability to provide direction and mentor less experienced teammates.
Ability to interpret and convey complex, difficult, or sensitive information.

Nice To Haves

Bachelor's degree and six years of experience or an equivalent combination of education and work experience.
Expertise with OpenTelemetry (Otel), including custom instrumentation, collector configuration, and pipeline design for traces, metrics, and logs.
Hands-on experience with observability tooling, such as Prometheus, Grafana, Jaegar, Loki, Elastic, Splunk, and/or Dynatrace in enterprise-grade environments.
Strong background in distributed systems, cloud-native architectures, and K8s, with the ability to identify observability gaps across service meshes, APIs, and event-driven platforms.
Proficiency in scripting or development languages (e.g. Python, Go, Bash, or Java) to automate telemetry integration, create custom exporters, and contributed to platform tooling.
Proven track record of driving enterprise adoption of observability standards and practices, including influencing telemetry strategies across engineering, SRE, and platform teams.

Responsibilities

Performs problem tracking, diagnosis and root-cause analysis, replication, troubleshooting, and resolution for complex issues. In this capacity, performs programming and debugging activities.
Responds to issues in a timely manner by receiving and investigating incidents or service tickets.
Analyzes and observes trends with technical issues and develops recommendations for long- term improvements.
Documents all relevant end-user interactions and steps taken to resolve incidents.
Has occasional contact with end-users.
Communicates status of issue resolution to internal customers.
May engage and manage outside vendors.
Applies in-depth knowledge of application support and an understanding of best practices.
Typically leads moderately complex projects and participates in larger, more complex initiatives.
Solves complex technical and operational problems.
Acts as a resource for teammates with less experience.
May have people management responsibilities for a small team.

Benefits

All regular teammates (not temporary or contingent workers) working 20 hours or more per week are eligible for benefits, though eligibility for specific benefits may be determined by the division of Truist offering the position.
Truist offers medical, dental, vision, life insurance, disability, accidental death and dismemberment, tax-preferred savings accounts, and a 401k plan to teammates.
Teammates also receive no less than 10 days of vacation (prorated based on date of hire and by full-time or part-time status) during their first year of employment, along with 10 sick days (also prorated), and paid holidays.
For more details on Truist's generous benefit plans, please visit our Benefits site.
Depending on the position and division, this job may also be eligible for Truist's defined benefit pension plan, restricted stock units, and/or a deferred compensation plan.
As you advance through the hiring process, you will also learn more about the specific benefits available for any non-temporary position for which you apply, based on full-time or part-time status, position, and division of work.

Stand Out From the Crowd

Upload your resume and get instant feedback on how well it matches this job.

Upload and Match Resume