Senior Artificial Intelligence Engineer

St. Jude Children's Research Hospital • Memphis, TN

1d • $86,320 - $154,960 • Onsite

About The Position

High Performance Research Computing (HPRC) and the Center for Bioimage Informatics (CBI) at St. Jude Children's Research Hospital are seeking a Senior AI Engineer to lead our efforts in advanced AI models, including large language models (LLMs), agentic AI systems, and multi-modal foundation models, and the secure computational infrastructure that powers them. This is a hands-on, high-ownership software systems role, not a prompt-engineering, API-integration, or purely conceptual research position. You will evaluate, fine-tune, deploy, and benchmark AI models; design and enforce safety guardrails and sandboxing for agentic systems; safeguard data security and privacy for sensitive research data; and optimize GPU/HPC resource allocation to balance performance, cost, and efficiency as models and tooling rapidly evolve. This person builds the shared AI architecture, secure environments, and best practices upon which CBI's image data scientists and software engineers rely. Deep bioimaging expertise is not required, though experience with biomedical research or imaging is a plus. This position reports to the Director of HPRC and works closely with CBI as a collaborative team member.

Requirements

• Master’s degree in computer science, computer engineering, data science, information technology or related field required.
• 4 years in designing and developing solutions for large scale AI/Machine Learning (ML) systems and/or building solutions for a product on AI/ML features and capabilities.
• Strong background in industry use cases built on deep learning and machine learning (unsupervised and supervised techniques) is a must.
• Deep learning frameworks such as TensorFlow, Keras, PyTorch, Time series analysis, anomaly detection, forecasting, predictive modeling, graph- based neural networks, Bayesian statistics, and text analytics are a must.

Nice To Haves

• Hands-on experience training, fine-tuning, or adapting LLMs or multi-modal foundation models (e.g., PEFT/LoRA, instruction tuning, preference optimization), including debugging failure modes such as catastrophic forgetting or training instability.
• Experience diagnosing and resolving distributed/multi-GPU training or inference issues (e.g., NCCL communication hangs, CUDA out-of-memory errors, load-balancing across nodes) and scheduling AI workloads on HPC (e.g., Slurm) to maximize GPU utilization.
• Experience designing safety guardrails and sandboxing for agentic systems: tool-access scoping, prompt-injection defense, secrets management, audit logging, and containment of failures.
• Experience optimizing inference cost, latency, and resource usage (e.g., KV-cache management, quantization, batching, speculative decoding, high-throughput serving via vLLM/TensorRT-LLM/Triton) and judgment about when a simpler deterministic pipeline is a better fit than an agentic one.
• Experience safeguarding data security and privacy for AI systems handling sensitive research data, including access controls and institutional data-use/security policies.
• Experience building rigorous, reproducible benchmarking/evaluation frameworks that separate genuine model improvement from prompt overfitting, retrieval effects, or evaluator bias.
• Experience building and scaling AI/ML pipelines and workflows on HPC or cloud environments.
• Demonstrated ownership of a system beyond the prototype stage (observability, versioning, rollback, cost control, incident response).
• Contributions to open-source AI/ML infrastructure projects (e.g., vLLM, PyTorch, Ray, Hugging Face) or a public track record (GitHub, Hugging Face, papers) are a plus.
• Familiarity with biomedical research, imaging, or regulated health data is a plus.
• Demonstrated technical leadership: setting standards, mentoring, and cross-team collaboration.

Responsibilities

• Assists Vice President of Research Informatics in (i) assessing institution’s needs and (ii) implementing AI-enabled services on high-performance AI computing platforms.
• Works with scientists, business stakeholders, analysts, and IS professionals in the design, development, and implementation of AI solutions and services, including technology proof-of-concepts, pilots, and the adoption lifecycle.
• Be responsible for building new and innovative solutions leveraging data science and AI/ML skills and technologies to solve non-trivial problems.
• Develops AI demonstration use-cases and workshop materials and provides workshops and training classes.
• Evaluate commercial and open-source approaches in AI/ML, Data Mining, and Analytics to solve business problems.
• Identifies, develops, and implements standards and operating procedures for solutions and systems consistent with best practices.
• Documents current and future state architecture roadmaps and reference architectures.
• Provides clear written and spoken communications to customers, teams, and vendors.
• Keeps abreast of new and emerging technologies and stays adaptable to their potential applicability.
• Performs other duties as assigned or directed to meet the goals and objectives of the department and institution.

Benefits

• St. Jude is an Equal Opportunity Employer

What This Job Offers

• Job Type: Full-time
• Career Level: Senior