Senior Infrastructure Engineer
About The Position
Quantum Sky is seeking a Senior Infrastructure Engineer to design, build, and operate the cloud and AI infrastructure behind Project 137, the company's applied-research and IP engine. Reporting to the Director of Infrastructure Engineering, the engineer will own Google Cloud Platform (GCP) and Kubernetes environments supporting AI/ML training, inference, and data workloads. The role is approximately 85% hands-on engineering and 15% project coordination. The engineer will build and scale production GCP infrastructure and AI/ML platforms; operate GKE clusters; create reusable platform components and infrastructure automation; and implement reliability, security, backup, and disaster-recovery capabilities. The position also involves mentoring engineers, leading design reviews, documenting standards and runbooks, and planning and tracking infrastructure work across engineering, research, and operations teams. The position is based in Reston, Virginia, with a hybrid/on-site schedule and approximately 10–20% occasional travel to partner sites and data-center locations.
Requirements
- Bachelor's degree in computer science, engineering, or a related technical field; equivalent experience is considered.
- At least 5 years in infrastructure, platform, cloud, or site reliability engineering, including at least 2 years of hands-on production work on Google Cloud Platform.
- Deep expertise across Compute Engine, GKE, VPC, Cloud IAM, Cloud Storage, and Cloud Operations.
- Production experience supporting AI/ML workloads, including GPU or TPU provisioning, training and inference platforms, and MLOps tooling such as Vertex AI or Kubeflow.
- Production experience operating Kubernetes, including cluster operations, Helm, networking and ingress, RBAC, autoscaling, and upgrades.
- Strong Terraform and Git-based workflow skills, with experience using CI/CD tools such as Cloud Build, GitHub Actions, GitLab CI, or Argo CD.
- Proficiency in Python, Go, or Bash for automation and tooling.
- Solid understanding of Linux, TCP/IP, DNS, load balancing, and hybrid cloud connectivity.
- Proven ability to plan, schedule, and track technical work across multiple concurrent workstreams using tools such as Jira or Confluence.
- Clear written and verbal communication skills, including the ability to explain technical trade-offs to engineers, researchers, and leadership.
Nice To Haves
- Google Cloud Professional Cloud Architect, Professional Cloud DevOps Engineer, or Professional Machine Learning Engineer certification.
- Certified Kubernetes Administrator (CKA), Application Developer (CKAD), or Security Specialist (CKS) certification.
- Experience with GPU scheduling and sharing on Kubernetes, such as NVIDIA GPU Operator, Kueue, or Ray, and large-scale distributed training.
- Familiarity with data-center fundamentals for high-density compute, including power, cooling, and capacity planning.
- Multi-cloud experience with AWS or Azure alongside GCP.
Responsibilities
- Design, build, and operate production GCP infrastructure, including landing zones, VPC networking, Cloud IAM, storage, and organization policy.
- Engineer and scale infrastructure for AI/ML workloads, including GPU and TPU capacity, distributed training, model serving and inference, and high-throughput data pipelines.
- Support data-science and research teams using Vertex AI, Kubeflow, and related MLOps tooling, including environment provisioning, model registries, and experiment infrastructure.
- Build and maintain hybrid connectivity between on-premises data-center and cloud environments, including Cloud Interconnect, VPN, DNS, and load balancing.
- Architect and operate GKE clusters, including GPU node pools, autoscaling, workload identity, networking and ingress, and version upgrades.
- Build reusable platform components and Helm charts to provide engineering and research teams self-service access to compute and AI tooling.
- Define and automate infrastructure with Terraform and GitOps practices, using Git as the source of truth for configuration, manifests, and CI/CD pipelines.
- Implement observability, alerting, and SLOs using Cloud Operations, Prometheus, and Grafana; participate in on-call and lead incident root-cause analysis.
- Apply cloud security controls, including least-privilege IAM, secrets management, container image scanning, policy-as-code, and software supply-chain protections.
- Plan, build, and test backup and disaster-recovery capabilities for platform and AI workloads.
- Mentor engineers, lead design reviews, and document architecture standards and runbooks.
- Plan and schedule infrastructure workstreams, maintaining timelines, milestones, and dependency tracking across engineering, research, and operations teams.
- Organize technical work into clear tasks and priorities, and report progress and blockers to the Director of Infrastructure Engineering.
- Travel occasionally, approximately 10–20%, to partner sites and data-center locations.
Benefits
- Health, dental, and vision insurance.
- 401(k) match.
- Paid Time Off.
- Short-term disability, long-term disability, and life insurance.
- Referral bonuses.
- Professional development reimbursement.
- Parental leave.
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Senior
