Senior Infrastructure Engineer

Upstart

1d•$166,900 - $230,900•Remote

About The Position

The Data Platform Engineering team (Part of Upstart's Decisioning & Data) provides developer tools, frameworks, and scalable data infrastructure as shared services across Upstart. The team's primary objective is to provide Data Analysts, ML scientists, and Software Engineers access to high-quality data and developer tools to create business metrics and training data for respective product verticals. You will join the Data Production Experience (PX) team, which owns Upstart’s Lakehouse (Databricks) infrastructure. This is the platform that Machine Learning, Analytics, and Software Engineering teams use to build the training data and metrics behind our lending, fraud, and pricing models—enabling growth and reducing risk across our business verticals. As the Senior Infrastructure Engineer joining this team, you will focus on: Lakehouse Infrastructure and Security: Own the Databricks platform across multiple environments, including a newly launched regulated environment (Upstart Bank). Responsibilities include expanding infrastructure as code for new Lakehouse features, enforcing a no-click-ops policy, establishing disaster recovery, implementing least-privilege and attribute-based access controls (e.g., based on data classification), ensuring environment isolation, and resolving other known security risks. Compliance: Close control gaps and automate evidence gathering for OCC, SOX and SOC audits across access management, change management, and data quality. Cost and usage efficiency: Reduce Databricks and AWS spend through accurate cost attribution, waste detection, and platform-level optimizations (e.g., compute rightsizing, improving pruning, migrating to Graviton and spot instances, Delta Lake Archival). How you’ll make an impact Evaluate and adopt new technologies, including open-source lakehouse alternatives, to expand platform capabilities. Improve developer efficiency and infrastructure reliability by working with cross-functional stakeholders like Data Engineering, Machine Learning, Analytics, Compliance, and Security. Improve Lakehouse security, usage efficiency, and observability by collaborating closely with the Data Engineering and Security teams. Evangelize good coding and engineering best practices while uplifting fellow engineers and mentoring junior team members. Help break down complex projects and requirements into quarterly commitments and sprints. Own Lakehouse infrastructure components end to end, from technical requirements and design through rollout and production support.

Requirements

Bachelor’s degree in Computer Science or a related field, or equivalent practical experience.
5+ years of experience in infrastructure engineering, DevOps, or data platform engineering.
Hands-on experience operating a lakehouse or big data platform (e.g., Databricks, Snowflake, Apache Spark, Delta Lake, Apache Iceberg).
Hands-on experience with AWS (e.g., IAM, S3, EC2, EKS), Python, infrastructure as code (Terraform or CDK), CI/CD, and Kubernetes.

Nice To Haves

Experience implementing security and compliance controls in audited environments (e.g., SOX, SOC 2, GDPR, GLBA, OCC), including access management, encryption, and change management.
Experience reducing cloud or data platform costs through usage analysis, cost attribution, and infrastructure optimization (e.g., Graviton or spot instances, storage tiering).
Ability to reason from first principles and work effectively through ambiguity on complex, cross-functional problems.
Proficient in DevOps engineering, leveraging Continuous Integration/Continuous Delivery (CI/CD) tools such as Kubernetes, Jenkins, CDK, APM, Terraform, and more. Experienced in automation, alerting, monitoring, security, and declarative infrastructure.
Solid understanding of cloud networking fundamentals.
Skilled at explaining complex concepts in simple, accessible terms and thriving in fast-paced, ambiguous environments.
Experience using AI coding assistants or agents to accelerate infrastructure development and operations.
Familiarity with distributed systems, data architecture design, and big data technologies (e.g., Spark, Kafka, Lakehouse, Databricks)

Responsibilities

Own the Databricks platform across multiple environments, including a newly launched regulated environment (Upstart Bank).
Expand infrastructure as code for new Lakehouse features.
Enforce a no-click-ops policy.
Establish disaster recovery.
Implement least-privilege and attribute-based access controls (e.g., based on data classification).
Ensure environment isolation.
Resolve other known security risks.
Close control gaps and automate evidence gathering for OCC, SOX and SOC audits across access management, change management, and data quality.
Reduce Databricks and AWS spend through accurate cost attribution, waste detection, and platform-level optimizations (e.g., compute rightsizing, improving pruning, migrating to Graviton and spot instances, Delta Lake Archival).
Evaluate and adopt new technologies, including open-source lakehouse alternatives, to expand platform capabilities.
Improve developer efficiency and infrastructure reliability by working with cross-functional stakeholders like Data Engineering, Machine Learning, Analytics, Compliance, and Security.
Improve Lakehouse security, usage efficiency, and observability by collaborating closely with the Data Engineering and Security teams.
Evangelize good coding and engineering best practices while uplifting fellow engineers and mentoring junior team members.
Help break down complex projects and requirements into quarterly commitments and sprints.
Own Lakehouse infrastructure components end to end, from technical requirements and design through rollout and production support.

Benefits

Competitive compensation, including base pay, bonus opportunities, and annual equity grants that vest quarterly
Retirement benefits to help you plan for the future, including a 401(k) or Group Retirement Savings Plan with a company match of $2 for every $1 contributed, up to $15,000 annually (USD in the US, CAD in Canada)
Employee Stock Purchase Plan (ESPP) with discounted stock purchase options for eligible employees (US only)
Comprehensive health coverage designed to support you and your family, including medical, dental, vision, and wellness resources for US and supplemental health coverage for Canada.
Health Savings Account contributions from Upstart for eligible plans (US only)
Income protection benefits, including life insurance and disability coverage for added financial security
Paid time off, sick leave, and company holidays, in line with local requirements
Paid family and parental leave to support caregiving and major life moments (duration varies by country)
Family-centered benefits to support fertility, parenthood, and caregiving needs
Employee Assistance Program (EAP) offering mental health support and life-centered resources
Financial wellness resources, including access to financial planning tools and a financial concierge service (US Only)
Annual wellness allowance to support your physical and emotional well-being and personal development, based on what matters most to you
Annual productivity allowance to invest in relevant tools and resources you need to do your best work, no matter where you work from
Connection and community through team events, all-company updates, and employee resource groups (ERGs)
Onsite perks, including catered lunches and fully stocked micro-kitchens when working from one of our offices in the Bay Area, Austin, Columbus, and New York City