Principal Architect

Amgen•Thousand Oaks, CA

About The Position

Amgen is seeking a Principal Architect to build and scale machine learning and Generative AI solutions from development through production. The role combines machine learning, data engineering, operations, and architecture to deliver reliable, scalable data and AI platforms and measurable business impact. The Principal Architect leads end-to-end design and delivery of ML and GenAI solutions using technologies such as Databricks, Apache Spark, SQL, and Python. The role sets architectural direction for large-scale data engineering and AI initiatives across lakehouse platforms, distributed compute, and cloud ecosystems, and designs batch and streaming pipelines, data models, ML workflows, and agentic AI systems. The role establishes data foundations, MLOps/LLMOps/DataOps/DevOps practices, and evaluation, validation, safety, and governance frameworks. It partners with business stakeholders and product leaders, leads experimentation and cross-functional delivery, mentors L4 and L5 engineers, and keeps current with advances in data engineering and Generative AI.

Requirements

  • Meet one of the following education and experience combinations: doctorate degree and 2 years of experience; master’s degree and 4 years; bachelor’s degree and 6 years; associate’s degree and 10 years; or high school diploma/GED and 12 years.
  • Deep expertise in machine learning, deep learning, and Generative AI, including LLMs, transformers, embeddings, and fine-tuning techniques.
  • Proven track record leading and delivering production-grade ML/GenAI systems end-to-end with measurable business impact.
  • Strong experience designing scalable ML/GenAI system architectures, including distributed systems and high-throughput pipelines.
  • Expertise in MLOps/LLMOps ecosystems, such as MLflow, Kubeflow, Airflow, CI/CD, Docker, and Kubernetes.
  • Strong system design, architecture, and problem-solving skills, with the ability to work independently and lead large initiatives.
  • Demonstrated proficiency using AWS, Azure, or GCP for data engineering solutions, with understanding of cloud architecture and cost optimization.
  • Proven ability to mentor and guide junior and mid-level engineers.

Nice To Haves

  • Cloud certifications in AWS, Azure, or GCP.
  • Experience with big data ecosystems such as Apache Spark and Hadoop and large-scale distributed data processing.
  • Expertise building and optimizing scalable data pipelines and platforms using Databricks, Spark, SQL, and Python.
  • Advanced Python proficiency and experience with modern ML/AI frameworks such as PyTorch, TensorFlow, Hugging Face, or LangChain.
  • Experience designing evaluation and validation frameworks, including automated evaluations, human-in-the-loop systems, safety testing, and monitoring.
  • Experience with Retrieval-Augmented Generation (RAG), vector databases, and knowledge-grounded AI systems.
  • Understanding of agentic AI frameworks, including orchestration, planning, memory management, and tool integration.
  • Foundation in statistical modeling, A/B testing, and causal inference.
  • Experience with NLP, semantic search, embeddings, and vector search systems.
  • Familiarity with Responsible AI practices, including fairness, explainability, governance, and regulatory compliance.
  • Hands-on experience with cloud-native AI/ML services on AWS, Azure, or GCP, including cost and performance optimization.
  • Experience with the Databricks Lakehouse platform for enterprise-scale data engineering, ML, and GenAI workloads.
  • Exposure to red-teaming, adversarial testing, and synthetic data generation.
  • Experience in data modeling and performance tuning for OLAP and OLTP systems.
  • Hands-on experience with workflow orchestration tools such as Apache Airflow and distributed processing frameworks such as Apache Spark.

Responsibilities

  • Lead the end-to-end design, development, and delivery of machine learning and Generative AI solutions, from problem framing through production deployment and business impact realization.
  • Architect large-scale data engineering and ML/GenAI initiatives across Databricks lakehouse platforms, Spark distributed compute, and AWS, GCP, or Azure cloud ecosystems.
  • Design and implement batch and streaming data processing, SQL data modeling, Python-based ML workflows, multi-agent architectures, reasoning workflows, tool integration, and autonomous decision-making systems.
  • Build and optimize scalable ETL/ELT pipelines in Databricks to ensure data availability, consistency, and performance for ML/GenAI use cases.
  • Define and institutionalize evaluation, validation, and governance frameworks for ML/GenAI, including model performance tracking, prompt evaluation, safety guardrails, hallucination mitigation, and compliance.
  • Partner with business stakeholders and product leaders to translate objectives into data-driven solutions with measurable value, defined pipelines, KPIs, and experimentation frameworks.
  • Establish and enforce MLOps, LLMOps, DataOps, and DevOps best practices, including CI/CD, Databricks workflows, monitoring, observability, reproducibility, and cost optimization.
  • Architect and oversee cloud-based data and AI platforms integrating Databricks Lakehouse, Spark processing, and cloud-native services.
  • Drive experimentation strategy through A/B testing, prompt optimization, and data-driven iteration using SQL analytics and Python experimentation frameworks.
  • Mentor L4 and L5 engineers in data engineering and AI/ML development through design reviews, code reviews, and career guidance.
  • Lead collaboration among data engineering, data science, platform engineering, and business teams to deliver production-grade AI solutions.
  • Track advancements in data engineering, lakehouse architectures, and Generative AI/agentic systems and drive adoption of relevant technologies and practices.

Benefits

  • Comprehensive employee benefits package.
  • Retirement and Savings Plan with generous company contributions.
  • Group medical, dental, and vision coverage.
  • Life and disability insurance.
  • Flexible spending accounts.
  • Discretionary annual bonus program, or a sales-based incentive plan for field sales representatives.
  • Stock-based long-term incentives.
  • Award-winning time-off plans.
  • Flexible work models where possible.
  • Career development opportunities.
© 2026 Teal Labs, Inc
Privacy PolicyTerms of Service