Performance Test Engineer

Systems Engineering Solutions Corporation•Chantilly, VA

About The Position

This program provides IT services focused on building, securing, and operating the Department of Veteran Affairs LGY’s home loan product-line technology. The contract’s purpose is to modernize and sustain critical home loan technology systems that support LGY’s delivery of mortgage-related services to program stakeholders, to provide continuous delivery and security integration. The Performance Developer will support the Department of Veterans Affairs (VA) Loan Guaranty Service (LGY). The Performance developer will be responsible for building the observability & monitoring artifacts, telemetry and logging that supports serverless system performance, diagnosing performance bottlenecks, and supporting performance optimization efforts across the LGY platform. The Performance Developer will not just write Lambda code, they will own the engineering pattern for how LGY Lambda services are built, instrumented with AWS ADOT, deployed through CI/CD, tested, and monitored in Dynatrace & Splunk. This role works closely with development first and then assists operations teams to ensure system performance scalability, and reliability across Salesforce and AWS-based integration services. The ideal candidate must have development experience working with API-driven systems, cloud infrastructure, and performance monitoring tools to develop & analyze telemetry, logs, and system metrics to identify and resolve performance issues.

Requirements

Bachelor’s degree in Computer Science, Information Technology, Engineering, or related field or equivalent professional experience and 3-7 years of experience.
Development background experience with Nodejs and Python on serverless architecture. THIS IS A MUST
Years of experience in performance engineering, application performance monitoring, or system performance analysis.
Experience with Dynatrace and Splunk
Experience analyzing application performance metrics and system telemetry.
Must be able to decipher code for troubleshooting performance issues in API-driven or distributed systems.
Familiarity with cloud-based environments such as AWS.
Strong analytical and problem-solving skills.
Experience working with monitoring tools and log analysis platforms.

Nice To Haves

Experience supporting VA or federal government IT systems.
Experience with AWS services such as Lambda, API Gateway, and cloud monitoring tools.
Familiarity with Salesforce-based applications and enterprise API integrations.
Experience supporting systems that process sensitive or regulated data.

Responsibilities

Design, build, & maintain AWS Lambda services in Node.js & Python based serverless portfolio.
Lead AWS ADOT-based instrumentation for Lambda workloads, including OpenTelemetry layer adoption, configuration, OTLP settings, & runtime compatibility validation.
Using GitHub actions develop pipelines to standardize CDK Lambda constructs for Dynatrace & AWS ADOT libraries, reusable layers, environment variables, active tracing, Lambda Insights, and centralized log configuration.
Must have dev background experience implementing & supporting end-to-end tracing, metrics development and structured logging across API Gateway, Step Functions, SQS, S3, RDS/Postgres, Salesforce integrations, Oracle/SOAP services, document APIs.
Validate Dynatrace instrumentation compatibility for Nodejs Function & implement wrapper or alternate instrumentation strategies where required.
Develop and maintain correlation ID and trace propagation across Lambda, Step Functions, and downstream service calls so transactions are observable end to end.
Enforce observability practices for AWS cloud workloads, including PII-safe structured logging
Build & maintain GitHub Actions pipelines for linting, unit tests, coverage, deploy, and post-deployment telemetry validations.
Create automated tests that verify instrumentation is working, including trace/log/metric arrival in Dynatrace & Splunk, cold-start visibility, and alert validation in each environment.
Develop python-based solutions to capture Lambda errors duration, errors, throttles, retries, cold starts, workflow failures, and downstream latency.
Tune Lambda memory, timeout, concurrency, packaging, and network configuration to balance AWS ADOT overhead, Dynatrace visibility, reliability, and cost.
Modernize older Lambda implementations and pipelines to a common AWS ADOT plus Dynatrace standard across multiple env.
Investigate performance incidents and collaborate with development teams to diagnose root causes.
Review system logs, API metrics, and AWS service telemetry to identify infrastructure or application performance constraints.
Provide performance optimization recommendations to development teams when performance bottlenecks are identified.