Data & LLM Systems Engineer

AllSpice • Boston, MA

1d

About The Position

We’re looking for a Data & LLM Systems Engineer to help us design, build, and operate the systems that sit at the intersection of hardware design, data, applications, and large language models (LLMs).
In this role, you’ll own how data flows from raw inputs into structured systems, how that data is exposed through our suite of applications, and how LLM interactions are instrumented, analyzed, and improved over time. You’ll work closely with our GenAI, Platform, and Infrastructure teams to ensure DRCY is reliable, observable, and continuously getting better.
This is not a research-only role and not a frontend-only role. It’s a hands-on engineering position focused on building real systems that people and products depend on.

Requirements

• Strong experience building backend systems and APIs
• Solid understanding of data modeling, databases, and data pipelines
• Experience analyzing applications that leverage LLMs (e.g., OpenAI, Anthropic, open-source models) in production systems
• Comfort analyzing data and drawing actionable conclusions
• Ability to reason about system performance, reliability, and cost

Nice To Haves

• Experience with vector databases and semantic search
• Familiarity with analytics warehouses (e.g., BigQuery, Snowflake, Redshift)
• Experience with LLM observability, evaluation frameworks, or experimentation platforms
• Background working on internal developer tools or data platforms
• Exposure to privacy, security, or governance considerations in data systems

Responsibilities

• Design, build, and maintain data pipelines for ingesting, cleaning, transforming, and storing data
• Define and evolve data schemas that support analytics, applications, and LLM workflows
• Work with relational databases, analytical data stores, and vector databases
• Ensure data reliability, performance, and cost efficiency
• Implement best practices around data versioning, lineage, and access control
• Build backend services and APIs that expose data to internal tools and user-facing applications
• Develop applications and internal tooling for managing datasets, experiments, prompts, and configurations
• Collaborate with product and design to ensure tools are usable, safe, and scalable
• Support both real-time and batch processing workflows
• Design systems that track prompt versions, context construction, and model configurations
• Instrument LLM interactions to capture inputs, outputs, metadata, latency, and cost
• Help establish standards for monitoring, debugging, and evaluating LLM behavior in production
• Analyze LLM outputs and user interactions to identify failure modes, drift, and quality issues and help ensure overall reliability and consistency
• Define and track metrics related to response quality, task success, and user outcomes
• Run comparisons between model versions, prompt changes, or system configurations
• Communicate findings clearly to engineering, product, and leadership stakeholders using
• Work closely with other engineers, product, and other stakeholders
• Take end-to-end ownership of systems from design through production and iteration

Benefits

• Supportive and smart colleagues, flexible work, opportunity to make a big impact, competitive salary & equity, health, dental, vision, generous PTO, home office stipend.

What This Job Offers

• Job Type: Full-time
• Career Level: Mid Level
• Education Level: No Education Listed