Senior Software Engineer, GenAI Platform

Upstart•San Mateo, CA

15d•Remote

About The Position

Upstart is the leading AI lending marketplace partnering with banks and credit unions to expand access to affordable credit. By leveraging Upstart's AI marketplace, Upstart-powered banks and credit unions can have higher approval rates and lower loss rates across races, ages, and genders, while simultaneously delivering the exceptional digital-first lending experience their customers demand. More than 80% of borrowers are approved instantly, with zero documentation to upload. Upstart is a digital-first company, which means that most Upstarters live and work anywhere in the United States. However, we also have offices in San Mateo, California; Columbus, Ohio; and Austin, Texas. Most Upstarters join us because they connect with our mission of enabling access to effortless credit based on true risk. If you are energized by the impact you can make at Upstart, we’d love to hear from you! The Team Upstart’s new Core GenAI Platform team is building foundational infrastructure that democratizes access to generative AI/LLMs for every product and engineering team across the company. As a Senior Software Engineer on Upstart's Core GenAI Platform team, you will help design and build a unified interface for securely interacting with large language models - abstracting away the complexity of model selection, prompt orchestration, evaluation, and scaling. You'll help build a centralized GenAI layer that enables engineers to use best-in-class LLMs through clean APIs, configurable pipelines, and intuitive tooling. Reporting to our Senior Director of Engineering, the team operates with broad scope and high visibility, partnering across Machine Learning, Product, and Compliance to ensure every model integration is performant, secure, and compliant. Whether it's embedding LLMs in user-facing products or optimizing backend workflows, this team makes it seamless for Upstart Engineering to adopt and scale GenAI company-wide.

Requirements

6+ years of experience in object-oriented software engineering, with a strong systems engineering background
Proven experience independently leading impactful, cross-functional, multi-quarter projects with mid-large teams
Proficiency in backend development with Python (leveraging frameworks like FastAPI, Flask, or similar), microservices architecture, and infrastructure tools like Kubernetes, Docker, and Terraform
Hands-on experience building ML platforms or infrastructure supporting LLMs or inference systems
Familiarity with observability tools (e.g., Prometheus, Grafana, Datadog) and data processing pipelines
Strong written and verbal communication skills and a collaborative approach to cross-functional work
Strong stakeholder management skills

Nice To Haves

Full Stack development skills including hands on experience with React or similar
Proficiency with Kotlin or Java and the Spring framework ecosystem
Experience with LLM toolchains such as LangChain, LlamaIndex, or OpenAI APIs
Deep understanding of model inference optimization (e.g., quantization, ONNX, streaming)
Exposure to retrieval-augmented generation (RAG), vector databases (e.g., FAISS, Pinecone), or prompt engineering
Experience operationalizing LLMs in production, including latency management and prompt versioning
Ability to influence platform direction and drive alignment across multiple engineering teams
Awareness of LLM-specific risk domains, including hallucinations, fairness, and data privacy

Responsibilities

Build the Core GenAI platform powering generative AI use cases across all of Upstart Engineering.
Design and implement reliable, observable infrastructure for model inference, prompt orchestration, and data workflows.
Drive cross-functional adoption of AI tooling by delivering reusable components and intuitive interfaces.
Increase developer productivity through automation, metrics dashboards, and streamlined GenAI integrations.
Set the technical roadmap for platform capabilities, balancing innovation with reliability and compliance.
Partner with ML researchers, product engineers, and design to bridge experimentation and production.
Ensure the platform adheres to emerging standards of security, fairness, and explainability for LLM systems.

Benefits

Competitive Compensation (base + bonus & equity)
Comprehensive medical, dental, and vision coverage with Health Savings Account contributions from Upstart
401(k) with 100% company match up to $4,500 and immediate vesting and after-tax savings
Employee Stock Purchase Plan (ESPP)
Life and disability insurance
Generous holiday, vacation, sick and safety leave
Supportive parental, family care, and military leave programs
Annual wellness, technology & ergonomic reimbursement programs
Social activities including team events and onsites, all-company updates, employee resource groups (ERGs), and other interest groups such as book clubs, fitness, investing, and volunteering
Catered lunches + snacks & drinks when working in offices