Overwatch- Observability and Evaluation Engineer
Delviom•Charlotte, NC
•Onsite
About The Position
Implements telemetry, traces, dashboards, evaluation suites, service objectives, runbooks, and readiness evidence for priority agent releases. Focuses on LLM and agent evaluation, tracing and telemetry, metrics and dashboards, SLOs, test automation, prompt and model performance analysis, Python, and production operations.
Requirements
- LLM and agent evaluation
- Tracing and telemetry
- Metrics and dashboards
- SLOs
- Test automation
- Prompt and model performance analysis
- Python
- Production operations
Responsibilities
- Implement telemetry, traces, dashboards, evaluation suites, service objectives, runbooks, and readiness evidence for priority agent releases.
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
Upload and Match Resume
What This Job Offers
Job Type
Full-time
Career Level
Senior
Education Level
No Education Listed
