Sr Data Scientist Hybrid role
About The Position
The Sr Data Scientist will work in a hybrid role based in San Francisco, California. The position requires strong Python skills and production experience, with a focus on generative AI and large language models (LLMs). The role involves developing retrieval-augmented generation (RAG) pipelines and retrieval strategies, designing prompts and agentic workflows, and orchestrating tools. It also involves evaluating LLM quality and monitoring systems for quality, latency, cost, and drift, while handling security and privacy requirements such as personally identifiable information (PII) and role-based access control (RBAC).
Requirements
- 10+ years of experience as a data scientist.
- Strong Python skills with production experience.
- Experience with generative AI and large language models (LLMs).
- Experience with RAG pipelines and retrieval strategies.
- Experience with prompt engineering, function calling, and prompt versioning.
- Experience with LangChain, LlamaIndex, or equivalent.
- Experience with agentic workflows and tool orchestration.
- Experience with LLM evaluation, including ground truth, accuracy, hallucination measurement, faithfulness, and grounding.
- Experience with observability for quality, latency, cost, and drift monitoring.
- Experience with cloud LLM platforms such as Azure OpenAI, AWS Bedrock, or Vertex AI.
- Knowledge of security and privacy, including PII handling and RBAC.
Nice To Haves
- Experience with Airflow or Dagster workflow orchestration.
- Machine learning and data science fundamentals.
- Experience with human-reviewed LLM-as-judge evaluation.
- Experience with Jira and Confluence.
Responsibilities
- Build or work with RAG pipelines and retrieval strategies.
- Use prompt engineering, function calling, and prompt versioning.
- Develop agentic workflows and orchestrate tools.
- Evaluate LLMs for accuracy, hallucinations, faithfulness, and grounding using ground truth.
- Monitor LLM quality, latency, cost, and drift.
- Handle security and privacy requirements, including PII and RBAC.
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Senior
Education Level
No Education Listed
