Application Architect - DARPA SciFy Project - Penn Engineering

University of Pennsylvania

20h•$100,000 - $150,000

About The Position

We are seeking an Application Architect with expertise in NLP and large language models to lead development on the DARPA SciFy project. The University of Pennsylvania leads a multi-university team building AI systems for automated scientific claim assessment and technology forecasting, in partnership with Two Six Technologies. The architect will design and build the core infrastructure for ClaimSpy, our flagship claim feasibility assessment system, extending it from verifiable materials science claims to speculative AI technology forecasting — supporting DARPA's "create and avoid surprise" mission. This is a 12-month grant funded position with the possibility of an extension dependent on continued funding.

Requirements

Bachelor's in Computer Science with four to six years of substantial software development experience or an equivalent combination of education and experience.
Strong Python skills and hands-on experience with LLM APIs (OpenAI, Anthropic, Google) and agent frameworks
Experience building agentic systems with tool use, multi-step reasoning, and orchestration
Experience developing APIs that expose LLM capabilities as services (REST/GraphQL endpoints, streaming, authentication, rate limiting)
Experience with API development, system integration, and full-stack web development
Ability to rapidly prototype and iterate in an agile research environment
Strong communication skills for coordinating across a distributed team and with government stakeholders

Nice To Haves

Experience with RAG systems, vector databases, and retrieval pipelines
Experience building interactive data exploration or sensemaking interfaces
Familiarity with scientific literature processing and structured knowledge extraction
Containerization and deployment (Docker, Kubernetes)
Open source contributions
Experience with evaluation frameworks, benchmarking, or LLM-as-judge pipelines
Prior experience on government-funded research projects

Responsibilities

Design and build ClaimSpy's agent architecture: multi-LLM pipelines with tool-calling chains for literature retrieval, computational analysis, and claim assessment
Develop an interactive Claim Card interface for analysts to construct, refine, and explore structured speculative claims, inspect reasoning traces, and review evidence bundles
Extend the Panel of Virtual Experts (PoVE) system to support Tech Council in a Box (TCBX), enabling multi-perspective deliberation over speculative AI forecasts
Integrate Two Six Technologies' speculative claims component into the ClaimSpy pipeline, including ablation studies to assess integration quality
Instrument systems with logging, observability, and error tracing across tool-calling chains to support operational pilot deployments
Collaborate with researchers on AutoRubric integration for automated LLM-as-judge evaluation of claim assessments
Conform to DARPA's testing and evaluation framework
Collaborate with postdocs and graduate students to translate research prototypes into production-ready systems
Assist with technical documentation, system testing, and monthly DARPA reporting
Participate in program meetings, technical demonstrations, and pilot exercises with transition partners

Benefits

Penn offers comprehensive medical, prescription, behavioral health, dental, vision, and life insurance benefits to protect you and your family’s health and welfare.
You can also use flexible spending accounts to pay for eligible health care and dependent care expenses with pre-tax dollars.
Take advantage of Penn's exceptional tuition benefits. You, your spouse, and your dependent children can get tuition assistance here at Penn. Your dependent children are also eligible for tuition assistance at other institutions.
Penn offers generous retirement plans to help you save for your future. Penn’s Basic, Matching, and Supplemental retirement plans allow you to save for retirement on a pre-tax or Roth basis. Choose from a wide variety of investment options through TIAA and Vanguard.
Penn provides you with a substantial amount of time away from work during the course of the year. This allows you to relax, take vacations, attend to personal affairs, recover from illness or injury, spend time with family—whatever your personal needs may be.
In partnership with Genworth Financial, Penn offers faculty and staff (and your eligible family members) long-term care insurance to help you cover some of the costs of long-term care services received at home, in the community or in a nursing facility. If you apply when you’re newly hired, you won’t have to provide proof of good health or be subject to underwriting requirements. Eligible family members must always provide proof of good health and are subject to underwriting.
Penn is committed to supporting our faculty and staff as they balance the competing demands of work and personal life. That’s why we offer a wide variety of programs and resources to help you care for your health, your family, and your work-life balance.
Penn provides an array of resources to help you advance yourself personally and professionally.
As a member of the Penn community, you have access to a wide range of University resources as well as cultural and recreational activities. Take advantage of the University’s libraries and athletic facilities, or visit our arboretum and art galleries. There’s always something going on at Penn, whether it’s a new exhibit at the Penn Museum, the latest music or theater presentation at the Annenberg Center, or the Penn Relays at Franklin Field to name just a few examples. As a member of the Penn community, you’re right in the middle of the excitement—and you and your family can enjoy many of these activities for free.
From arts and entertainment to transportation and mortgages, you'll find great deals for University faculty and staff. Not only do Penn arts and cultural centers and museums offer free and discounted admission and memberships to faculty and staff. You can also enjoy substantial savings on other goods and services such as new cars from Ford and General Motors, cellular phone service plans, movie tickets, and admission to theme parks.
Flexible work options offer creative approaches for completing work while promoting balance between work and personal commitments. These approaches involve use of non-traditional work hours, locations, and/or job structures.
Penn offers a forgivable loan for eligible employees interested in buying a home or currently residing in West Philadelphia, which can be used for closing costs or home improvements.
Penn will reimburse eligible employees on qualified expenses in connection with the legal adoption of an eligible child, such as travel or court fees, for up to two adoptions in your household.