Generative AI Application Engineer

Purple Squirrel Enterprises•Washington, DC

2d•$135,000 - $150,000•Remote

About The Position

Purple Squirrel Enterprises is exclusively engaged by a well-established SaaS analytics company serving the banking industry to identify and hire a hands-on Generative AI Application Engineer. This role involves designing, building, and shipping production generative AI features, including API-based services, agents, and tools built against large language model (LLM) APIs, along with the necessary application and data layers. It is a hands-on, mid-level engineering role emphasizing software engineering depth and applied generative AI experience, with substantial Python and API development history, direct LLM API development, and strong SQL skills. The development process is AI-First / Specification-Driven, requiring close collaboration with product management, data engineering, and the AI/Modeling team to translate requirements into shipped features and identify opportunities to increase organizational efficiency using generative AI. Candidates must have generative AI applications running in production today, be able to describe their contribution, and provide reachable references. The environment is data-sensitive and regulated, with high standards for production work.

Requirements

At least 5 years of professional software development experience, ideally within a software product organization, including hands-on development of API-based applications and services deployed and used in production
At least 4 years of hands-on Python development experience — deep enough to read, explain, and defend your own design decisions
At least 2 years of substantive generative AI development at the LLM API level — prompt design, structured outputs, tool/function calling, retrieval, and evaluation. Assembling no-code/low-code tooling does not satisfy this requirement
At least two non-trivial generative AI applications you helped build that are in production today, with the ability to describe the problem, architecture, your specific contribution, and how it's evaluated/monitored
Strong SQL skills — joins, aggregation, window functions, CTEs — plus a working understanding of RDBMS performance and role-based access controls
At least 4 years of experience working across multiple resource types in a hyperscaler environment (VMs, containers, object storage, RDBMS services). Microsoft Azure preferred; equivalent AWS or GCP experience considered
Proficiency with version control practices and tools (Git and/or Azure DevOps)
Reachable professional references covering the experience above, including your production generative AI work

Nice To Haves

Hands-on experience with models on Hugging Face — selection, serving format/quantization choices, and evaluation
Direct, personal involvement deploying models for self-hosted LLM inference (LightLLM, vLLM, SGLang, TensorRT-LLM, TGI, Ray Serve, Triton, Ollama, llama.cpp, or similar). Gateway/routing layers like LiteLLM also relevant
Self-hosted inference deployments supporting production applications (not just experiments), including capacity planning, batching/concurrency, versioning, and monitoring
Experience building applications/UIs that interface with self-hosted, open-source LLMs via APIs
Experience with Amazon Bedrock or Microsoft Foundry (formerly Azure AI Foundry)
Machine learning experience — feature engineering, model training/evaluation using Pandas, scikit-learn, or similar
Python charting libraries (Matplotlib, Seaborn, Plotly)
Azure SQL and PySpark experience for large datasets
Additional experience in C#, Java, or Scala
React/TypeScript UI development (AI Elements or shadcn/ui a plus)
Exposure to banking, financial services, or another regulated data environment

Responsibilities

Design, build, and ship production generative AI features developed directly against LLM APIs — prompt construction, structured output handling, tool/function calling, retrieval, context management, and failure handling
Build and maintain the evaluation and regression harnesses that determine whether a generative AI feature is behaving correctly as models, prompts, and data change
Integrate generative AI components with existing product services, APIs, and data models
Work with models hosted on Hugging Face and with internally hosted inference engines, including configuration and deployment of the serving layer
Design, build, and maintain API-based applications and services following the team's software and product life-cycle processes
Write and tune SQL against relational data models for application state, feature data, evaluation datasets, and product analytics
Deploy and operate application components across multiple resource types in a hyperscaler environment (VMs, containers, object storage, RDBMS services) — primarily Microsoft Azure
Contribute to analytics product work that doesn't involve AI/models, when needed
Contribute to model-based analytics such as customer segmentation, churn forecasting, and lifetime value forecasting, working alongside senior members of the AI/Modeling team
Support internal initiatives that use generative AI, models, and heuristics to increase organizational efficiency
Work with product management, data engineering, and AI/Modeling teammates to translate requirements into shipped features
Participate in code review — both giving and receiving
Communicate progress, technical trade-offs, and blockers clearly to your team and to stakeholders outside it
Write automated tests appropriate to the work, including evaluation tests for generative AI components and model-evaluation tests for ML components
Monitor deployed AI applications in production for accuracy, latency, token consumption, and cost; participate in production support as needed
Follow security, data handling, and change-management practices required when working with sensitive financial data

Benefits

Base salary: $135,000 – $150,000
$1,100/month company benefit contribution toward eligible pre-tax benefits (medical, dental, vision, HSA) — flexible to allocate based on your plan selection
Medical, dental, and vision coverage, including HSA-compatible plan options
UHC Rewards program — earn up to $300 (non-HSA) or $1,000 (HSA) annually through wellness activities
Employer-paid Life/AD&D insurance (1x salary up to $200,000)
Employer-paid Short-Term and Long-Term Disability coverage
Voluntary benefits available: additional Life/AD&D, Critical Illness, Accident, and Hospital Indemnity coverage
Employee Assistance Program (EAP)
401(k) plan with employer match — up to 4.5% total match for employees contributing 6% of salary, immediately vested
Company holidays plus a Flexible Time Off (FTO) policy — no fixed PTO cap, built on mutual trust and manager coordination