Senior Data Scientist II

RELX•Raleigh, NC

3d•$104,900 - $174,700

About The Position

We are looking for a Senior Data Scientist II with deep expertise in Generative AI, Retrieval-Augmented Generation (RAG), and agentic AI systems, combined with strong software-engineering fundamentals and demonstrated ownership of production applications. This role will focus on improving LLM-powered drafting and retrieval solutions through advanced search, embeddings, reranking, evaluation, and production-grade ML components. The successful candidate must be able to independently design, refactor, test, review, deploy, and support clean, reliable Python applications. This includes separating agent responsibilities, designing for failure, applying sound algorithmic reasoning, and establishing appropriate logging, monitoring, testing, and operational controls. The ideal candidate has advanced Python proficiency, experience with OpenSearch or Solr, success working in monorepo environments, and a strong record of cross-functional delivery.

Requirements

Advanced Python proficiency demonstrated through independently designing, implementing, debugging, testing, reviewing, and refactoring production applications.
Strong command of Python fundamentals, standard data structures, common algorithms, object-oriented and functional design principles, type annotations, and time and space complexity analysis.
Demonstrated ability to transform prototype or experimental code into modular, maintainable, observable, and production-ready systems.
Strong understanding of software design principles, including separation of concerns, dependency injection, interface design, configuration management, and effective abstraction.
Experience implementing automated unit, integration, regression, and end-to-end testing using tools such as pytest, including appropriate mocking of external services.
Experience designing resilient distributed applications that account for timeouts, retries, rate limits, partial failures, malformed responses, idempotency, and graceful degradation.
Experience with production observability, including structured logging, metrics, tracing, alerting, and incident troubleshooting.
Demonstrated ability to conduct rigorous code reviews and identify correctness, maintainability, performance, security, testing, and operational risks.
Experience taking technical ownership of applications across their lifecycle, from design and experimentation through deployment, monitoring, incident response, and ongoing improvement.
Strong understanding of production LLM concerns, including structured output validation, context management, model and tool failures, prompt versioning, token and cost controls, security, and evaluation.

Nice To Haves

Experience with Python quality tooling such as pytest, ruff, mypy, profiling tools, and automated CI quality gates.
Experience defining typed schemas and validating LLM inputs and outputs using tools such as Pydantic.
Experience building evaluation frameworks for agentic systems, including task-completion, retrieval-quality, groundedness, hallucination, latency, reliability, and cost metrics.
Experience implementing model fallbacks, tool-use controls, guardrails, human-in-the-loop workflows, and auditability for AI applications.
Experience supporting production services and participating in incident response, root-cause analysis, and post-incident remediation.

Responsibilities

Architect modular agentic applications with clear separation among retrieval, prompt construction, model invocation, tool execution, state and history management, orchestration, validation, and response formatting.
Independently refactor complex or legacy Python code to improve correctness, readability, modularity, extensibility, testability, and runtime performance.
Own production readiness for AI components, including input validation, exception handling, timeout management, retries with backoff, fallback behavior, configuration management, and secure handling of credentials.
Establish observability for LLM and retrieval workflows through structured logging, metrics, distributed tracing, alerting, and actionable error reporting.
Design clear interfaces and data contracts between retrieval, orchestration, model, and downstream application components.
Write comprehensive unit, integration, regression, and end-to-end tests, including tests for failure modes, malformed model responses, empty retrieval results, and unavailable dependencies.
Review Python and agentic application code, identify architectural and operational risks, and provide actionable feedback aligned with production engineering standards.
Diagnose and optimize latency, memory usage, retrieval performance, token consumption, model cost, and application scalability.
Apply appropriate data structures, algorithms, and computational-complexity analysis when designing and optimizing solutions.
Participate in production deployments, incident investigation, root-cause analysis, remediation, and continuous reliability improvements.