Senior Engineering Manager - Accelerated Compute Memory Systems

Pryon • Boston, MA

77d

About The Position

Pryon is building an industry-leading knowledge management and Retrieval-Augmented Generation (RAG) platform. Our proprietary, cutting-edge natural language processing capabilities transform unstructured data into meaningful experiences that increase productivity with unmatched accuracy and speed. We need an Engineering Manager with deep HPC expertise—someone who can teach, not be taught. You’ll lead the technical team building our ingestion, retrieval, and inference layers, ensuring scalability, reliability, and compliance.

Requirements

• 10+ years in software engineering, 5+ years in management roles with large-scale AI/ML systems and infrastructure.
• Expert-level proficiency in Python and Golang, with 5+ years building production distributed systems.
• Experience with orchestration frameworks (Kubernetes, Ray, Dask).
• Proficiency with vector databases (Pinecone, Weaviate, Qdrant, or similar).
• Experience with message queuing systems (Kafka, Pulsar, RabbitMQ).
• In-depth knowledge and hands on experience building scalable distributed architectures and high-performance compute systems.
• Proven experience in multimodal ingestion pipelines within RAG platforms.
• Direct experience in designing, fine-tuning, and optimizing LLMs for ingestion and retrieval workloads.
• Previous success managing engineering teams delivering production-grade, HPC-scale RAG systems.
• Deep understanding of infra domains: compute, storage, networking, observability, security, disaster recovery, and cost management.
• Familiarity with HPC cluster management softwares such as Slurm.
• Familiarity with cloud platforms (AWS, Azure, GCP) and/or on-prem datacenter operations.

Responsibilities

• Build and lead a team delivering the ingestion, retrieval, and inference layers that will power mission-critical deployments for commercial and federal entities with millions of public users.
• Architect and deliver horizontally scalable, fault-tolerant systems capable of handling billions of documents and burst loads of 30K+ concurrent users.
• Guide implementation of multimodal ingestion pipelines (eg PDF, HTML, DOCX, JSON, XML, PPTX, TIFF).
• Oversee design and optimization of LLM-driven data ingestion and retrieval workflows.
• Own optimization and tuning of high-throughput, low-latency production environments via async orchestration frameworks.
• Establish performance benchmarking, compliance frameworks, and automated testing for scale.
• Balance technical leadership with people leadership, guiding architecture decisions, while also scaling and mentoring a high-performing team.
• Collaborate cross-functionally with Product, Executive Leadership, and Customer Success.

Benefits

• Remote first organization
• 100% Company paid Health/Dental/Vision benefits for you and your dependents
• Life Insurance, Short-term and Long-term Disability
• 401k
• Unlimited PTO

What This Job Offers

• Job Type: Full-time
• Career Level: Manager
• Number of Employees: 101-250 employees