AI Prompt Engineer

Pear VC

11d • Remote

About The Position

Paxos Health is looking for a part-time AI Prompt Engineer, likely 10-15 hours per week. This role is remote-friendly, and the hours are flexible. We are open to expanding this to a full-time role in the future pending your preferences and business needs as we grow. We are a fast-growing company (with venture capital funding) that uses LLMs and AI agents to help medical patients fight their health insurance companies to get covered for live-saving and life-changing treatments. Our AI reads all of a patient’s medical documents to come up with the best possible argument about why an upcoming treatment is needed, and then it generates all the paperwork and follows up with the health insurer. We partner with medical device companies so our service is totally free to patients! See our website here.

Requirements

• Hands-on experience writing and testing complex prompts for LLMs.
• Understanding of prompt orchestration, including templating and variable injection; role design (system/user/tool); few-shot exemplar selection; structured JSON output enforcement (schemas, validators, retries); and tool-calling for retrieval and citation (e.g., linking outputs to specific document sections or PDF coordinates).
• Ability to meet deadlines of a couple days when necessary. The hours are flexible, but sometimes we need relatively quick turnaround time for customers.
• Ability to work independently. We will collaborate closely with you, but you should be able to execute on tasks and small projects on your own.
• Relentlessly detail-oriented, with zero tolerance for silent hallucinations
• Experience with prompt/eval tooling (e.g., Langfuse, Langflow, Vellum, PromptLayer, Helicone), with comfort evaluating per-criterion accuracy, traceability, and error tradeoffs.
• Ability to overlap at least 3 hours with U.S. Eastern Time standard business hours.

Nice To Haves

• Experience using RAG technology for processing long documents.
• Light scripting experience: Able to write small Python or TypeScript scripts to wire prompts into pipelines, run evals end-to-end, do basic post-processing, etc.
• Healthcare industry experience.

Responsibilities

• Design high-precision, schema-guided prompts for structured extraction and complex decision logic over long documents, with careful management of versioned JSON schemas and downstream compatibility.
• Build modular, multi-step LLM workflows by decomposing complex problems into clearly defined stages, managing intermediate state, and ensuring reliable, production-ready execution.
• Debug and improve outputs by analyzing failure modes and implementing guardrails such as schema validation and retries, constrained decoding, deterministic tools or retrieval, and rule-based post-processing.
• Maintain a versioned prompt library and contribute prompt patterns and best practices, including standardizing approaches that have proven effective via evaluation sets, A/B testing, and quality metrics.
• Contribute to AI strategy: suggest changes to how we orchestrate and use AI to advance our business goals, particularly as we grow. We’re always open to new frameworks and techniques!

What This Job Offers

• Job Type: Part-time
• Career Level: Mid Level
• Education Level: No Education Listed
• Number of Employees: 11-50 employees