ML Engineer, LLM's

Knowtex•San Francisco, CA

34d•Hybrid

About The Position

Knowtex is hiring an ML Engineer / Researcher focused on large language models to develop and optimize the models behind its clinical documentation and structured clinical reasoning products. The role uses Knowtex’s proprietary clinical data from real-world encounters across hundreds of specialties to improve model quality, cost, latency, and controllability. This is a research-heavy role with a direct path to production. The engineer will design experiments, build datasets and evaluation systems, fine-tune and post-train models, and collaborate with engineering and clinical teams to deploy successful approaches at scale. The role also contributes centrally to defining Knowtex’s long-term LLM strategy. The work includes generating clinical documentation such as SOAP notes and specialty-specific formats; supporting structured clinical tasks such as medication extraction, orders, coding, and visit summaries; evaluating and training open-weight models; improving inference efficiency; serving models in production; and developing rigorous evaluation frameworks. The position is hybrid, with in-person work in Knowtex’s San Francisco office Monday through Wednesday. Applicants must be authorized to work in the U.S.; visa sponsorship is unavailable.

Requirements

At least 4 years of experience in machine learning research or ML engineering, with deep expertise in large language models.
Hands-on experience fine-tuning or post-training open-weight LLMs, such as with SFT, distillation, preference optimization, or reinforcement learning.
Strong Python and PyTorch skills.
Deep understanding of modern transformer architectures and LLM training techniques.
Strong experimental methodology and ability to independently design and execute research projects.
Experience working with large-scale datasets and distributed training environments.
Strong understanding of LLM evaluation and benchmarking.
Ability to translate research results into production systems.
Bachelor's, Master's, or PhD in Computer Science, Machine Learning, or a related technical field, or equivalent research experience.
Must be authorized to work in the U.S.; visa sponsorship is not available.

Nice To Haves

Experience building LLM evaluation systems, including LLM-as-judge, human preference, and task-specific benchmarks.
Experience serving and optimizing open-weight models at scale, such as vLLM, TensorRT-LLM, quantization, or speculative decoding.
Experience with structured generation, tool use, or agentic systems.
Experience in healthcare AI or clinical NLP.
Familiarity with clinical documentation workflows and medical terminology.
Knowledge of coding systems such as ICD-10, CPT, E&M, or SNOMED.
Publications at leading ML or NLP conferences.
Experience deploying ML systems in HIPAA-compliant or regulated environments.
Experience in fast-moving startups where researchers own projects from experimentation through production.

Responsibilities

Develop and optimize models that generate high-quality clinical documentation, including SOAP notes and specialty-specific note formats.
Build models for downstream clinical tasks, including medication extraction, orders, ICD-10 coding, E&M coding, patient visit summaries, and other structured clinical artifacts.
Apply structured generation, tool use, and agentic approaches to produce reliable, controllable clinical outputs.
Evaluate open-weight and proprietary models and determine when fine-tuning, distillation, structured generation, or task-specific models can outperform general-purpose API-based approaches.
Fine-tune and post-train open-weight LLMs on proprietary clinical datasets using SFT, distillation, preference optimization, and reinforcement learning.
Research ways to reduce inference cost and latency while maintaining or improving clinical quality.
Serve and optimize open-weight models in production at scale.
Build evaluation frameworks for clinical accuracy, hallucinations, completeness, formatting, and clinician preferences.
Build datasets, benchmarks, and evaluation infrastructure to make model improvements measurable and reproducible.
Design experiments that determine whether approaches improve real-world clinical outcomes.
Move research from idea through dataset, experiment, and evaluation into production; help deploy successful research beyond prototypes.
Balance model quality with latency, inference cost, reliability, and scalability.
Collaborate with clinicians, speech and applied ML engineers, and platform engineers.