Computational Linguist, Generative AI - Sr. Associate

JPMorgan Chase & Co.•Wilmington, DE

6d•$104,500 - $165,000

About The Position

Join a team building and evaluating next-generation conversational experiences powered by generative and agentic AI. You will help define how we measure quality, safety, and customer impact at scale. This role blends deep language expertise with rigorous data-driven evaluation. You will partner closely with engineers, analysts, and annotators to turn insights into measurable improvements. If you enjoy experimentation and clear standards, this is a chance to push what is possible. As a Computational Linguist, Generative AI within the Chase Digital Assistant team, you will help us evaluate and strengthen large language model-powered customer experiences. You will move between qualitative language analysis and quantitative measurement to ensure our evaluations are transparent, repeatable, and ready to support release decisions. You will design and maintain taxonomies, rubrics, and documentation that make model behavior understandable and improvable. You will work with partners across product, engineering, analytics, and operations to translate requirements into durable evaluation systems. You will help us continuously improve model correctness, customer experience, and business outcomes. In this role, you will support the evolution from traditional conversational systems to large language model-driven workflows while maintaining strong traceability and governance. You will contribute to knowledge modeling, including ontology design and knowledge graph approaches, to improve semantic reasoning and data integration. You will help create practical guardrails for generative use cases and ensure measurement stays aligned with real customer needs. You will also help solve “cold start” challenges by supporting synthetic data approaches where appropriate. Your work will directly influence how reliably and responsibly we ship new capabilities.

Requirements

Master’s degree in Computational Linguistics, Natural Language Processing, Linguistics, or a related field
Demonstrated experience applying computational linguistics or natural language processing to chatbots or conversational AI
Hands-on experience evaluating generative and agentic AI systems and workflows (for example, AutoGen, LangGraph, CrewAI, Sierra, or similar)
Strong linguistic foundation in discourse and pragmatics
Advanced knowledge of conversational AI development, including training, design, and conversation analysis
Hands-on experience with large language model integration, prompt engineering, evaluation, and performance monitoring
Proficiency in Python, Git, Linux, and Bash scripting
Working knowledge of common natural language processing and data science libraries (for example, pandas, NumPy, scikit-learn, NLTK)
Experience with transformer-based models (for example, BERT- and GPT-style models), including fine-tuning and applied use
Comfort working with JSON Lines, CSV, and notebook-based workflows
Experience with taxonomies, ontologies, and knowledge graphs, plus familiarity with evaluation methods such as human comparison and red teaming

Nice To Haves

Experience designing hybrid conversational architectures that combine structured flows with large language model-driven experiences
Familiarity with large language model safety, bias, and compliance considerations
Demonstrated success working effectively in a highly matrixed organization
Knowledge of current generative AI evaluation challenges, including subjective and preference-based tasks
Experience with retrieval augmented generation pipelines and agent orchestration patterns
Experience generating or validating synthetic data to address cold start and low-resource scenarios
Domain experience supporting customer-facing financial products or customer service journeys

Responsibilities

Serve as a subject matter expert across the product lifecycle, maintaining a strong understanding of the Chase Digital Assistant from both customer and technical perspectives
Manage and evolve the intent and entity taxonomy, including taxonomy governance, versioning, traceability, and rollback practices
Evaluate model behavior through qualitative linguistic analysis and quantitative measurement, surfacing actionable insights
Design scalable evaluation processes for large language model workflows, partnering with engineers, analysts, and annotators
Implement and maintain metrics across model correctness, customer experience, AI assurance, and business outcomes
Introduce new guardrail metrics for generative and agentic AI use cases, and monitor them over time
Own the evaluation artifact suite, including model descriptions, prompts, rubrics, judge prompts, guidelines, calibration data, and reliability measures
Support integration of large language models with existing conversational architectures and event tracking systems
Apply ontology design principles to improve semantic consistency and data integration aligned to business standards
Design frameworks to incorporate knowledge graphs into classification and extraction approaches
Identify optimization opportunities that improve model performance, data quality, and feature coverage