Observability and Evaluation Engineer
Symphony Solutions•Charlotte, NC
•$50 - $60•Onsite
About The Position
The Observability and Evaluation Engineer implements telemetry, traces, dashboards, evaluation suites, service objectives, runbooks, and readiness evidence for priority agent releases. The role focuses on LLM and agent evaluation, tracing and telemetry, metrics and dashboards, SLOs, test automation, prompt and model performance analysis, and production operations.
Requirements
- 12+ years of experience.
- Python.
Responsibilities
- Implement telemetry and tracing for priority agent releases.
- Develop dashboards, evaluation suites, service objectives, runbooks, and release-readiness evidence.
- Perform LLM and agent evaluation.
- Automate tests and analyze prompt and model performance.
- Support production operations.
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
Upload and Match Resume
What This Job Offers
Job Type
Full-time
Career Level
Senior
Education Level
No Education Listed
