Software Engineer III - AIML

JPMorgan Chase & Co. • Mclean, VA

2d • $137,750 - $185,000

About The Position

As a Software Engineer III at JPMorganChase within Corporate AIML Data Platforms-Chief Data and Analytics Office, you will contribute to an agile team building and delivering secure, stable, and scalable technology products. The role focuses on Go- and Python-based backend services that support LLM-enabled experiences and critical technology solutions across multiple technical areas and business functions.

You will design, develop, troubleshoot, and operate backend services and streaming gRPC applications. The position also involves integrating LLM services with supporting systems, collaborating with platform and SRE teams, improving service reliability and observability, and applying enterprise-authorized AI development tools responsibly within the software development lifecycle.

Requirements

• Formal training or certification in software engineering concepts and at least 3 years of applied experience.
• Proficiency in Go for backend and service development, including goroutines and channels, context usage, and performance troubleshooting.
• Working knowledge of Python for supporting services, internal tooling, automation, evaluation harnesses, or data and ETL tasks.
• Production experience with gRPC and Protobuf, including API design and backward- and forward-compatibility considerations.
• Experience building and operating microservices or APIs with a production mindset, including monitoring, debugging, incident support, and continuous improvement.
• Proficiency in automation and continuous delivery methods, with strong unit and integration testing discipline.
• Proficiency in secure coding practices and core SDLC activities, including design, build, test, deploy, and operate.
• Hands-on experience using enterprise-authorized AI-assisted software development tools at work, with the ability to critically evaluate, validate, and refine generated outputs for correctness, performance, and security.
• Understanding of responsible AI use in engineering workflows, including data sensitivity, secure handling of inputs and outputs, and resiliency and security expectations; ability to guide peers on safe and effective usage.

Nice To Haves

• Experience building end-to-end LLM streaming experiences, including partial-response streaming, tool or function-calling orchestration, session state management, and safety or guardrail hooks.
• Familiarity with OpenTelemetry and tracing and metrics patterns for streaming RPCs, including correlation IDs, span links, and per-message metrics.
• Experience with Kubernetes and Docker and operating services in containerized environments.
• Experience with messaging and streaming systems such as Kafka, RabbitMQ, or SQS; caching such as Redis; and asynchronous orchestration alongside streaming.
• Knowledge of RPC security best practices, including mTLS, authentication and authorization, secrets management, secure-by-default API design, and abuse prevention such as rate limiting and payload limits.
• Understanding of cost management and spend alerting for AI/LLM workloads.

Responsibilities

• Execute software design, development, and technical troubleshooting for LLM-enabled applications and platforms.
• Design, build, and maintain backend services in Go, delivering secure, high-quality production code with strong testing discipline.
• Build gRPC services defined with Protobuf, including API design, schema evolution, and compatibility and versioning practices.
• Implement bidirectional streaming gRPC endpoints for LLM workloads, including interactive sessions, partial-response streaming, incremental results, cancellations, and flow control.
• Integrate LLM services with authentication and authorization, rate limits and quotas, caching, queues and streams, and persistence; partner with platform and SRE teams as needed.
• Review and debug SDK and service integrations, participate in code reviews, and contribute to shared engineering standards.
• Instrument services with metrics, logging, and tracing, and contribute to runbooks and operational readiness, including alerts, dashboards, and on-call support as applicable.
• Identify recurring issues and contribute fixes and automation to improve reliability, latency, and operational stability.
• Use enterprise-authorized AI coding assistance to improve code quality, delivery speed, and productivity; validate outputs through peer review, automated testing, and secure coding standards, and share reusable patterns and learnings.
• Apply knowledge of software development lifecycle tools, including enterprise-authorized AI-assisted development and automation capabilities, to improve the value realized through automation.

Benefits

• Competitive total rewards package, with base salary determined by role, experience, skill set, and location; eligible roles may also receive commission-based pay and/or discretionary incentive compensation in cash and/or forfeitable equity.
• Comprehensive health care coverage.
• On-site health and wellness centers.
• Retirement savings plan.
• Backup childcare.
• Tuition reimbursement.
• Mental health support.
• Financial coaching.
• Other benefits and programs based on eligibility.

What This Job Offers

• Job Type: Full-time
• Career Level: Mid Level
• Education Level: No Education Listed