Lead Associate Principal, Software Engineering: Software Development Test (SDET)

OCC • Chicago, IL

1d • $122,500 - $204,600 • Hybrid

About The Position

OCC's Enterprise Quality Engineering organization seeks an Associate Principal in Performance Testing to design and execute comprehensive performance and failover testing strategies across mission-critical clearing, risk, and settlement systems. This role combines hands-on performance engineering expertise to ensure systems meet regulatory RTO/RPO requirements, handling multi-region failover scenarios, cloud-native architectures, and distributed event-driven systems.

Requirements

• 7+ years hands-on experience in performance testing and engineering, including design and execution of complex performance test scenarios
• 5+ years designing and executing failover/DR testing across multi-region architectures (cloud and hybrid environments)
• Strong expertise in performance testing tools: LoadRunner (enterprise scaling), JMeter, Gatling, with ability to evaluate tool trade-offs
• Proficiency in scripting/programming: Python, Java, JavaScript, with ability to develop performance testing frameworks
• Solid hands-on experience with AWS cloud environments, including EC2, RDS, Kafka, and performance monitoring
• Working knowledge of Kubernetes and containerized application performance testing (Docker, Kafka, event-driven systems)
• Experience with UC4 (Automic) job scheduling in test execution and orchestration
• Hands-on experience testing Kafka producer/consumer performance, lag validation, and message throughput
• Strong understanding of relational databases (SQL Server, Oracle, DB2) and ability to write queries for performance analysis
• Experience with CI/CD pipelines (Jenkins, GitHub, GitLab CI) and integration of performance tests into automation
• Proficiency with APM/observability platforms (Splunk, Dynatrace, AppDynamics) and correlation of test results with production telemetry
• Working knowledge of Linux operating systems and command-line tools for performance analysis
• Experience with test management tools (Jira, Zephyr, SpiraTest) and defect tracking integration
• Background in finance domain OR highly regulated, low-latency, high-availability systems (clearing, settlement, risk)
• Experience with multi-region testing (primary/secondary/tertiary architectures) with performance validation across regions
• B.S. Degree in Computer Science, an analytical field, or equivalent practical experience required.
• 7+ years of experience in the IT industry with 5+ years in Quality Engineering, tool architecture and orchestration including opensource and commercial toolset.
• 5+ years of experience in Agile methodology.
• 5+ years of experience in integration of Jira and Test management tool.
• Strong communication skills, written and oral, analytical abilities, problem solving skills, sound judgment and time management skills.
• Experience in creating automation scripts.
• Experience in developing automation frameworks.
• Experience in developing Jenkins pipelines.
• Experience in the Options and Futures industry is preferred

Nice To Haves

• Familiarity with AI/ML-augmented testing tools for automated test case generation, self-healing automation, and predictive defect analysis.
• Experience with API and contract testing (Postman, RestAssured, Pact) and service virtualization for testing complex distributed and microservices architectures.
• Understanding of Site Reliability Engineering (SRE) principles, SLIs/SLOs/SLAs, and how performance engineering feeds operational resilience metrics.
• Experience with observability/APM platforms (Splunk, Dynatrace, AppDynamics, AWS Vantage, Logic Monitor) and correlating test results with production telemetry.
• Testing Experience in multi region (primary/secondary/tertiary) hybrid architecture across cloud and on-premise
• Familiarity with FinOps or cost-aware performance engineering practices in cloud environments.
• Experience testing critical financial market infrastructure (clearing, settlement, risk management, or exchange systems) or other highly regulated, low-latency, high-availability environments.
• AWS certification and/or performance engineering certification preferred

Responsibilities

• Define test plans, load profiles, and ramp scenarios for high-volume transaction systems
• Execute tests using LoadRunner, JMeter, and Gatling; analyze results and recommend optimizations
• Lead DR testing across primary, secondary, and tertiary regions
• Build and maintain performance test automation using Python, Java, and open-source tools
• Develop CI/CD-integrated pipelines (Jenkins, GitHub Actions) for continuous performance validation
• Implement regression testing to catch performance degradation from code changes
• Validate application behavior across multi-region deployments
• Define and validate RTO/RPO targets for critical applications
• Design failover scenarios (region switching, data replication, automation, post-failover validation)
• Partner with infrastructure/application teams to verify state consistency and data integrity during failover
• Test replication lag, failover triggers, and cross-region data consistency
• Validate Kafka producer/consumer behavior under failover conditions
• Confirm performance SLAs hold during region-switching scenarios
• Integrate test results with observability platforms
• Maintain and evolve LoadRunner scripts and environments
• Document methodologies, baselines, and trend analysis
• Mentor QE engineers on performance and failover testing techniques
• Collaborate with development, DevOps, and infrastructure teams on requirements and capacity planning
• Ensure testing aligns with PGP compliance and financial market infrastructure regulations
• Document test evidence to support regulatory audits
• Validate performance SLAs meet regulatory RTO/RPO requirements

Benefits

• A hybrid work environment, up to 2 days per week of remote work
• Tuition Reimbursement to support your continued education
• Student Loan Repayment Assistance
• Technology Stipend allowing you to use the device of your choice to connect to our network while working remotely
• Generous PTO and Parental leave
• 401k Employer Match
• Competitive health benefits including medical, dental and vision

What This Job Offers

• Job Type: Full-time
• Career Level: Principal