AI Platform Engineer
About The Position
The AI Platform Engineer is a senior technical individual contributor responsible for building, operating, and continuously improving shared infrastructure and platform services that enable secure, reliable, and scalable use of artificial intelligence across New York University. The role develops and maintains the technical foundation for enterprise AI capabilities, including cloud infrastructure, model gateways and routing, APIs, identity and access management, automated deployment pipelines, observability, usage and cost telemetry, and platform reliability. Working within architectural patterns established by the AI Solutions Architect and adhering to institutional security and technology standards, the engineer translates AI architecture into production-ready infrastructure and reusable platform capabilities. The role is highly hands-on and involves directly building, automating, and operating infrastructure.
Requirements
- Bachelor's degree in Computer Science, Engineering, Information Technology, or a related quantitative field, or an equivalent combination of education and experience.
- At least 3 years of progressive experience in platform engineering, cloud engineering, DevOps/SRE, software engineering, or related infrastructure engineering roles.
- Direct experience operating production cloud services at scale.
- Deep experience with AWS cloud services and cloud-native architecture.
- Demonstrated proficiency with Terraform, AWS CloudFormation, or equivalent infrastructure-as-code tools.
- Hands-on experience with Docker and Kubernetes (EKS) or AWS ECS.
- Hands-on experience building and supporting CI/CD pipelines, such as GitHub Actions, GitLab CI, or Jenkins, and modern source control strategies.
- Strong foundation in API design, gateway management, OAuth2 and OIDC authentication, service-to-service communication, IAM, secrets management, and cloud network security.
- Practical experience setting up metrics, logging, distributed tracing, and alerting tools.
- Proficiency in Python and/or modern programming languages used for automation and platform tooling.
- Practical working knowledge of LLM APIs, foundation models, token-based usage models, model routing and gateways, streaming responses, rate limiting, and the operational constraints and latency characteristics of generative AI services.
Nice To Haves
- Experience with AWS Bedrock.
- At least 3 years of experience configuring and managing Portkey or comparable enterprise AI gateways and LLM proxy systems.
- Familiarity with specialized AI observability and evaluation frameworks such as LangSmith, Phoenix, Arize, or Traceloop.
- Experience with FinOps practices applied specifically to token-based consumption and AI workloads.
- Experience delivering and supporting infrastructure services in a large, federated, higher-education, research institution, or similarly decentralized enterprise environment.
Responsibilities
- Build, operate, and continuously improve shared infrastructure and platform services for secure, reliable, and scalable AI use.
- Develop and maintain cloud infrastructure, model gateways and routing, APIs, identity and access management, and automated deployment pipelines.
- Support observability, usage and cost telemetry, and overall platform reliability.
- Translate AI architecture into production-ready infrastructure and reusable platform capabilities.
- Build, automate, and operate infrastructure directly.
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Senior
