Generative AI Application Engineer

Purple Squirrel Enterprises • Washington, DC

5h • Remote

About The Position

Purple Squirrel Enterprises is partnering with a well-established SaaS analytics company serving the banking industry to find a hands-on Generative AI Application Engineer. This role is focused on designing, building, and shipping production generative AI features, including API-based services, agents, and tools that interact directly with large language model (LLM) APIs. The position requires a strong software engineering background with applied generative AI experience, including Python, API development, and SQL. The work environment is data-sensitive and regulated, with a high standard for production-ready features. The engineer will collaborate with product management, data engineering, and the AI/Modeling team to translate requirements into shipped features and identify opportunities to improve organizational efficiency using generative AI. This is a mid-level engineering role, emphasizing practical application and production deployment over research or primary machine learning modeling.

Requirements

• At least 5 years of professional software development experience, ideally within a software product organization, including hands-on development of API-based applications and services deployed and used in production.
• At least 4 years of hands-on Python development experience.
• At least 2 years of substantive generative AI development at the LLM API level — prompt design, structured outputs, tool/function calling, retrieval, and evaluation.
• At least two non-trivial generative AI applications you helped build that are in production today, with the ability to describe the problem, architecture, your specific contribution, and how it's evaluated/monitored.
• Strong SQL skills — joins, aggregation, window functions, CTEs — plus a working understanding of RDBMS performance and role-based access controls.
• At least 4 years of experience working across multiple resource types in a hyperscaler environment (VMs, containers, object storage, RDBMS services). Microsoft Azure preferred; equivalent AWS or GCP experience considered.
• Proficiency with version control practices and tools (Git and/or Azure DevOps).
• Reachable professional references covering the experience above, including your production generative AI work.

Nice To Haves

• Hands-on experience with models on Hugging Face — selection, serving format/quantization choices, and evaluation.
• Direct, personal involvement deploying models for self-hosted LLM inference (LightLLM, vLLM, SGLang, TensorRT-LLM, TGI, Ray Serve, Triton, Ollama, llama.cpp, or similar). Gateway/routing layers like LiteLLM also relevant.
• Self-hosted inference deployments supporting production applications (not just experiments), including capacity planning, batching/concurrency, versioning, and monitoring.
• Experience building applications/UIs that interface with self-hosted, open-source LLMs via APIs.
• Experience with Amazon Bedrock or Microsoft Foundry (formerly Azure AI Foundry).
• Machine learning experience — feature engineering, model training/evaluation using Pandas, scikit-learn, or similar Python charting libraries (Matplotlib, Seaborn, Plotly).
• Azure SQL and PySpark experience for large datasets.
• Additional experience in C#, Java, or Scala.
• React/TypeScript UI development (AI Elements or shadcn/ui a plus).
• Exposure to banking, financial services, or another regulated data environment.

Responsibilities

• Design, build, and ship production generative AI features developed directly against LLM APIs, including prompt construction, structured output handling, tool/function calling, retrieval, context management, and failure handling.
• Build and maintain evaluation and regression harnesses to ensure generative AI features behave correctly as models, prompts, and data change.
• Integrate generative AI components with existing product services, APIs, and data models.
• Work with models hosted on Hugging Face and internally hosted inference engines, including configuration and deployment of the serving layer.
• Design, build, and maintain API-based applications and services following team processes.
• Write and tune SQL against relational data models for application state, feature data, evaluation datasets, and product analytics.
• Deploy and operate application components across multiple resource types in a hyperscaler environment (e.g., VMs, containers, object storage, RDBMS services), primarily Microsoft Azure.
• Contribute to analytics product work that doesn't involve AI/models when needed.
• Contribute to model-based analytics such as customer segmentation, churn forecasting, and lifetime value forecasting.
• Support internal initiatives that use generative AI, models, and heuristics to increase organizational efficiency.
• Collaborate with product management, data engineering, and AI/Modeling teammates to translate requirements into shipped features.
• Participate in code reviews.
• Communicate progress, technical trade-offs, and blockers clearly to the team and stakeholders.
• Write automated tests, including evaluation tests for generative AI components and model-evaluation tests for ML components.
• Monitor deployed AI applications in production for accuracy, latency, token consumption, and cost; participate in production support as needed.
• Follow security, data handling, and change-management practices for sensitive financial data.

Benefits

• $1,100/month company benefit contribution toward eligible pre-tax benefits (medical, dental, vision, HSA) — flexible to allocate based on your plan selection
• Medical, dental, and vision coverage, including HSA-compatible plan options
• UHC Rewards program — earn up to $300 (non-HSA) or $1,000 (HSA) annually through wellness activities
• Employer-paid Life/AD&D insurance (1x salary up to $200,000)
• Employer-paid Short-Term and Long-Term Disability coverage
• Voluntary benefits available: additional Life/AD&D, Critical Illness, Accident, and Hospital Indemnity coverage
• Employee Assistance Program (EAP)
• 401(k) plan with employer match — up to 4.5% total match for employees contributing 6% of salary, immediately vested
• Company holidays plus a Flexible Time Off (FTO) policy — no fixed PTO cap, built on mutual trust and manager coordination

What This Job Offers

• Job Type: Full-time
• Career Level: Mid Level
• Education Level: No Education Listed