Scientific Evals
About The Position
Edison Scientific builds and deploys AI scientist agents to accelerate science and the development of new medicines. We are an ambitious team run by scientists and engineers from leading institutions across biology, physics, chemistry, and AI. We are seeking an ambitious, scientifically grounded person to join our team focused on developing rigorous benchmarks and training datasets that advance AI capabilities in biology. This role sits at the intersection of biology, data curation, and machine learning, and is ideal for someone with deep scientific training who is excited to shape how frontier AI systems learn to do science. This role is on-site at our San Francisco office in the Dogpatch neighborhood. Our office is a converted warehouse with high ceilings, open space, and a team that genuinely believes in what they're building. This position is part of the Evals team.
Requirements
- Graduate-level training in biology, biochemistry, computational biology, or a related field, with hands-on research experience.
- Working knowledge of machine learning concepts, particularly deep learning and large language models.
- Comfortable with Python and building workflows for data processing, analysis, and experimentation.
- Possess strong scientific taste and able to identify what distinguishes expert-level reasoning from surface-level pattern matching.
- Detail-oriented and willing to take on high-value but occasionally tedious work.
- Energized by ambiguous, open-ended problems that require creativity, collaboration, and first-principles thinking to solve.
- Organized and communicative, able to manage multiple workstreams and coordinate across teams.
Nice To Haves
- Prior experience creating evaluation datasets, annotation guidelines, or working on human-in-the-loop data pipelines.
- Experience with bioinformatics pipelines, biological databases, or sequence analysis tools.
- Hands-on experience fine-tuning or evaluating large language models, or familiarity with RLHF and preference-based training.
- Publications or research experience in areas relevant to AI for science.
Responsibilities
- Design benchmarks that capture the complexity of real biological research, drawing on your domain expertise to identify what makes scientific reasoning hard. This will include open-ended scientific benchmarks and building on prior work like LAB-Bench and BixBench.
- Curate and vet biological datasets to ensure scientific rigor.
- Analyze model outputs, identify failure modes, and contribute to iterative improvements in both datasets and evaluation criteria.
- Collaborate with AI/ML researchers to translate scientific intuition into training signal, helping AI systems learn not just facts but how scientists think.
- Coordinate operations and manage workflows, including working with domain experts, tracking task progress, and maintaining documentation.
Benefits
- Competitive salary and equity
- Full healthcare coverage; we pay 100% of premiums for you and your dependents
- Support for growing families, including a yearly new parent stipend and fertility coverage through Carrot
- Mental health support through Rula, our in-network therapist and psychiatrist network with fast availability
- 12 weeks of paid parental leave for maternity, paternity, and adoption
- Pet care support with a yearly employer-funded stipend for your animal companions
- Commuter benefits so you can pay for transit and parking with pre-tax dollars
- 401(k) company matching
- $300 health and wellness benefit quarterly
- Lunch is on us every day you're in the office, and dinner is on us when you're working late
- Regular team off-sites and company events
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Senior
