AI Infrastructure Engineer
About The Position
Design, build, and operate on-prem infrastructure that behaves like a cloud environment for internal teams, including AI/ML workloads. Own datacenter and infrastructure operations: compute, storage, networking, and the automation layered on top. Build and maintain Infrastructure-as-Code using Terraform (Terragrunt), Ansible, etc. Deploy, secure, and operate our HashiCorp stack (Vault, Boundary) alongside identity/access via Keycloak. Build and maintain observability with Prometheus, Grafana, and Alertmanager. Deploy and manage containerized workloads via Docker and Kubernetes. Design and maintain networking infrastructure: VLANs, routing, firewall rules, and load balancers. Write scripts/tools (and contribute code where useful) to automate operational toil. Provide general datacenter support for customers colocated in the HostMax datacenter — things like cabling, hardware swaps, rack/stack work, and basic troubleshooting. Participate in business-hours (M-F) on-call rotation for customer and infrastructure support. Participate in an SDLC/Scrum-based workflow, using Jira and Confluence for planning and documentation. Help define standards, runbooks, and best practices as the team grows. Development and deployment of AI infrastructure workloads on-prem (GPU scheduling, model serving infra, self-hosted AI tooling, etc.).
Requirements
- Solid experience with Infrastructure-as-Code and Configuration-as-Code
- Real-world on-prem/datacenter operations experience — this is explicitly not a cloud-native (AWS/GCP/Azure) role, though the environment is designed to feel cloud-like
- Working knowledge of networking fundamentals: VLANs, routing, firewalls, and load balancers
- Hands-on experience with containerization and orchestration (Docker, Kubernetes)
- Proficiency with Git and standard version control workflows
- Development/scripting skills sufficient to build internal tooling and automation (not just run existing playbooks)
- Comfort with hands-on datacenter/hardware work (cabling, rack/stack, hardware troubleshooting) in addition to infra engineering
Responsibilities
- Design, build, and operate on-prem infrastructure that behaves like a cloud environment for internal teams, including AI/ML workloads
- Own datacenter and infrastructure operations: compute, storage, networking, and the automation layered on top
- Build and maintain Infrastructure-as-Code using Terraform (Terragrunt), Ansible, etc.
- Deploy, secure, and operate our HashiCorp stack (Vault, Boundary) alongside identity/access via Keycloak
- Build and maintain observability with Prometheus, Grafana, and Alertmanager
- Deploy and manage containerized workloads via Docker and Kubernetes
- Design and maintain networking infrastructure: VLANs, routing, firewall rules, and load balancers
- Write scripts/tools (and contribute code where useful) to automate operational toil
- Provide general datacenter support for customers colocated in the HostMax datacenter — things like cabling, hardware swaps, rack/stack work, and basic troubleshooting
- Participate in business-hours (M-F) on-call rotation for customer and infrastructure support
- Participate in an SDLC/Scrum-based workflow, using Jira and Confluence for planning and documentation
- Help define standards, runbooks, and best practices as the team grows
- Development and deployment of AI infrastructure workloads on-prem (GPU scheduling, model serving infra, self-hosted AI tooling, etc.)
Benefits
- Medical Insurance
- Dental Insurance
- Vision Insurance
- 401(k)
- Flexible spending account
- Commuter benefits
- Disability insurance
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Senior
Education Level
No Education Listed
