AI Researcher (Remote)

Cisco•San Francisco, CA
•$256,900 - $383,500•Hybrid

About The Position

The AI Researcher will define and lead the research agenda for the models and methods behind Splunk Agent Resilience. The role identifies customer problems in AI agent evaluation, behavioral failure discovery, and agent improvement; develops modeling and data approaches; and guides research from early experiments to validated product capabilities. The researcher remains hands-on with experimentation while establishing standards for data quality, benchmarking, robustness, and reproducibility, and provides technical leadership, research reviews, and mentorship across multiple workstreams. In collaboration with Product Management and Engineering, the role uses experimental evidence to inform investments and practical trade-offs among model quality, generalization, latency, and cost. The position focuses on improving the resilience, reliability, quality, and observability of AI agents and LLM-powered systems. Work includes developing and fine-tuning small language models using supervised fine-tuning, reinforcement learning, parameter-efficient techniques such as LoRA, and other post-training approaches; developing and validating novel model adaptation, training, and data-generation methods; leading training and data initiatives; and designing rigorous evaluation methodologies. The researcher translates validated methods into production-ready capabilities and communicates findings through reproducible code, datasets, documentation, reports, publications where appropriate, and presentations to technical and non-technical stakeholders, including customers. The role is hybrid in the Bay Area, New York City, or Seattle, Washington; fully remote candidates within the United States will also be considered. Occasional meetings outside standard local business hours may be needed to accommodate global teams and customers.

Requirements

  • Bachelor's degree in Computer Science, Machine Learning/Artificial Intelligence, Mathematics, or a related field with 10 or more years of related experience; or a Master's degree with 8 or more years of related experience; or a PhD with 5 or more years of related experience.
  • Experience applying machine learning or AI research to deployed products or practical business problems, including defining research problems, developing modeling or data methodologies, and validating results through empirical experimentation.
  • Experience training and fine-tuning encoder or decoder language models using supervised fine-tuning, reinforcement learning, or parameter-efficient methods such as LoRA.
  • Experience developing machine learning solutions using Python and deep learning frameworks such as PyTorch, TensorFlow, or equivalent, including model training, optimization, and evaluation.
  • Experience leading AI/ML research initiatives, defining research direction, guiding research workstreams, evaluating experimental methodologies, and mentoring researchers.

Nice To Haves

  • Master's degree or PhD in Machine Learning, Computer Science, Statistics, or a related field.
  • Experience designing model evaluations, datasets, or metrics to measure model quality, robustness, and system behavior, including ablation studies, contamination checks, and regression testing.
  • Research publications, preprints, or demonstrated technical contributions involving model training, adaptation, evaluation, or data generation, supported by rigorous experimental results.
  • Experience with advanced model training techniques, including preference optimization, contrastive learning, knowledge distillation, or multi-task learning.
  • Experience developing simulation and synthetic data pipelines, optimizing data mixtures and sampling strategies, or improving model performance with noisy or limited labeled data.
  • Experience with distributed model training and evaluating trade-offs involving model quality, training efficiency, inference latency, and computational cost.

Responsibilities

  • Define and execute an applied research agenda to improve the resilience, reliability, quality, and observability of AI agents and LLM-powered systems.
  • Develop and fine-tune small language models using supervised fine-tuning, reinforcement learning, parameter-efficient methods such as LoRA, and other post-training techniques.
  • Research and develop approaches to model adaptation, training objectives, and data generation, validating improvements through controlled experiments and established baselines.
  • Lead large-scale training and data development, including data sourcing, simulation, synthetic data generation, and labeling; establish standards for annotation quality, coverage, provenance, and human validation.
  • Design and implement rigorous model evaluation, including ablation studies, contamination checks, robustness testing, and regression analysis.
  • Provide technical leadership across research initiatives through research reviews, experimental design, problem-solving, and mentorship.
  • Partner with Product Management and Engineering to translate validated research methods into production-ready AI capabilities.
  • Document and communicate findings through reproducible code, datasets, model documentation, technical reports, and research publications where appropriate.
  • Present technical concepts, research findings, and recommendations to technical and non-technical stakeholders, including customers.
  • Collaborate with global teams and customers, including occasional meetings outside standard local business hours to accommodate different time zones.

Benefits

  • Medical, dental, and vision insurance
  • 401(k) plan with Cisco matching contribution
  • Paid parental leave
  • Short- and long-term disability coverage
  • Basic life insurance
  • Paid holidays and a floating holiday for non-exempt employees
  • Birthday day off, paid year-end holiday shutdown, and paid personal wellness days
  • Paid vacation or flexible vacation time off, depending on employee classification
  • Paid sick time
  • Additional paid time away may be requested for critical or emergency family issues
  • Optional paid volunteer days
  • Annual bonuses for eligible non-sales roles
  • Potential Cisco restricted stock unit grants, subject to eligibility and vesting terms
© 2026 Teal Labs, Inc
Privacy PolicyTerms of Service