Infrastructure Engineer - Labs

Etched • San Jose, CA

23h • Onsite

About The Position

Etched is hiring an Infrastructure Engineer for its Infra Labs team to own rack and server infrastructure across the company’s labs and data center sites. The role supports systems from physical installation through network integration, provisioning, connection to storage and compute services, observability, bring-up, validation, and testing. Sites include the Santana Row office lab, a hardware facility in Milpitas, and Bay Area colocation locations.

This is a hands-on, full-stack infrastructure role focused on on-premises environments rather than cloud infrastructure or production SRE. The engineer will build and operate compute, networking, storage, provisioning, and core services for existing and next-generation rack-scale systems and silicon. Work spans early-stage and custom hardware, bare-metal operations, lab and data center deployments, and collaboration with Lab Operations, Supercomputing, Deployment, network engineers, and other engineering teams. The position is on-site in San Jose and involves travel to Bay Area lab and data center sites.

Requirements

• At least 4 years of hands-on infrastructure, data center systems, or bare-metal operations experience in on-premises environments involving physical hardware.
• Comfort working across servers, power circuits and PDUs, Linux, networking, and hardware, and troubleshooting problems across these layers.
• Strong Linux systems skills on bare metal, including kernel and driver debugging, PCIe and NIC troubleshooting, storage, and boot issues.
• Experience provisioning bare-metal fleets over a network using PXE/iPXE, DHCP, and DNS, and using BMCs such as Redfish or IPMI for out-of-band management.
• Ability to write infrastructure automation in Ansible (YAML and Jinja2) and Python, and use Git, pull requests, code review, and tests.
• Working knowledge of L2/L3 networking fundamentals, VLANs, LACP, switch configuration, and optics and copper cabling at 100G and above.
• Experience building or operating observability stacks such as Prometheus, Grafana, or VictoriaMetrics.
• Experience handling incident response and deploying software and firmware updates across a fleet.
• Ability and willingness to rack and unrack heavy servers using a lift, run cabling, and work in an active lab.
• Genuine interest in hardware, including early and experimental hardware without an established vendor playbook.
• Ability to work with electrical, mechanical, facilities, and software engineers and document work so it can be repeated.
• Ability to work on-site in San Jose and travel to Bay Area lab and data center sites.

Nice To Haves

• Experience operating GPU, accelerator, or HPC clusters, especially liquid-cooled systems, including in-rack CDUs, facility liquid cooling, leak detection, and automated shutdown.
• Experience with OCP hardware, including Open Rack v3, DC bus bar power, and OpenBMC.
• Experience in hardware bring-up and NPI environments such as EVT, DVT, and PVT where platforms are still changing.
• Experience with high-performance networking, including RoCE/RDMA fabrics, Tomahawk-class switches, Arista EOS or SONiC, and AEC, DAC, and optical cabling at 400G.
• Experience operating Slurm and Kubernetes clusters.
• Storage engineering experience with NVMe, mdadm RAID, and NFS, including performance benchmarking.
• Experience with asset management or DCIM systems.
• Experience with colocation build-outs, including cross-connects, carrier circuits, and remote-hands coordination.
• Experience with infrastructure identity and access systems such as FreeIPA, RADIUS, or TACACS+.
• Experience with Terraform and operating mixed Linux distributions such as RHEL and Ubuntu.

Responsibilities

• Own and support racks and server infrastructure across Etched’s labs and data center sites from installation handoff through bring-up, validation, and testing.
• Deliver infrastructure for new lab and data center sites, including compute, networking, storage, DHCP, DNS, zero-touch provisioning, and NFS.
• Build and maintain bare-metal provisioning for servers, development kits, rack management controllers, and switches using PXE/iPXE, DHCP, BMC/Redfish-driven imaging, and golden images.
• Troubleshoot Linux hosts and early-stage hardware, including kernel, PCIe, NIC driver and firmware, bonding, storage, boot, and power issues; identify root causes and document findings.
• Bring racks onto the network by configuring top-of-rack switches and uplinks and planning VLANs and IP addresses, in partnership with network engineers.
• Diagnose physical-layer and hardware faults, including link flaps, CRC errors, optics failures, power faults, and coolant leaks; coordinate repairs and RMAs with lab staff and vendors.
• Help manage shared lab compute, including Slurm partitions, reservations, and maintenance drains, and expand storage and compute as programs grow.
• Extend Prometheus, VictoriaMetrics, and Grafana observability to lab racks, with health, power, and link-status dashboards and alerts across sites.
• Write infrastructure automation in Ansible and Python through reviewed and tested pull requests, and build tooling for controlled environments and individual lab teams.
• Manage software releases and changes on lab systems under change control, respond to incidents, and maintain runbooks, inventory, and design records.
• Build and operate infrastructure for new facilities, labs, colocation cages, and next-generation silicon and rack validation.
• Develop provisioning, telemetry, hardware-health, asset-inventory, storage, and firmware workflows for lab infrastructure.

Benefits

• Medical, dental, and vision packages with generous premium coverage.
• $500 per month credit for waiving medical benefits.
• Housing subsidy of $2.5k per month for employees living within walking distance of the office.
• Relocation support for those moving to San Jose (Santana Row).
• Wellness benefits covering fitness, mental health, and more.
• Daily lunch and dinner in the office.
• Unlimited compute budget subject to ROI justification.

What This Job Offers

• Job Type: Full-time
• Career Level: Mid Level
• Education Level: No Education Listed