Principal Software Engineer - Workload Communications Domain Lead

SciTec•Princeton, NJ

10d•$164,000 - $218,000

About The Position

SciTec, a Firefly Aerospace company, is seeking a Principal Software Engineer to serve as their Workload Communications Domain Lead. This role involves guiding engineering teams in designing and optimizing data and control flow between distributed workloads for mission-critical data processing and visualization. The position is hands-on, requiring technical expertise in distributed messaging, event streaming, and RPC systems, as well as the ability to provide practical guidance and mentorship to other engineers. The ideal candidate will assess current communication methods, develop evidence-based recommendations, build reference implementations, and work directly with engineers to enhance organizational capabilities.

Requirements

Bachelor’s degree in Computer Science, Engineering, Mathematics, Physical Sciences, or a related technical field.
10+ years of professional experience designing, implementing, and troubleshooting production software systems, including significant experience with distributed communication systems.
Deep production experience with one or more message brokers or event streaming platforms (e.g., Kafka, RabbitMQ, ActiveMQ, MQTT).
Experience with one or more RPC or low-level messaging frameworks (e.g., gRPC, ZeroMQ).
Strong understanding of delivery semantics, ordering, idempotency, backpressure, and fault handling in distributed systems.
Experience with serialization formats and schema evolution (e.g., Protocol Buffers, Avro, FlatBuffers, JSON Schema).
Demonstrated ability to load test and profile communication paths, analyze results, and turn findings into actionable recommendations.
Experience evaluating technical alternatives and clearly articulating their tradeoffs.
Proficiency in at least one backend language such as C++, Python, Go, Rust, Java, or TypeScript/Node.js.
Comfortable working in Linux development environments.
Strong technical writing, mentoring, and cross-team communication skills, with the ability to influence engineering practices through technical credibility.
Ability to obtain and maintain a DoD security clearance.
U.S. citizenship is required due to U.S. government contracts.

Nice To Haves

Experience leading technology standardization, technical governance, or internal platform/guidance efforts across multiple teams.
Experience with high-throughput or low-latency data pipelines, including network and memory performance tuning.
Experience with distributed tracing and observability tooling (e.g., OpenTelemetry).
Experience running messaging systems in containerized or Kubernetes environments.
Familiarity with mission data workflows or sensor processing architectures.
Active DoD Secret clearance or higher.

Responsibilities

Assess current communication technologies, patterns, recurring problems, and capability gaps across engineering teams.
Define evaluation criteria and use load tests and benchmarks under representative traffic and failure conditions to select and maintain a Core Collection of recommended communication technologies, patterns, and practices.
Document recommended technologies, valid alternatives, and exceptional cases through decision guides, reference architectures, architecture decision records (ADRs), and tuning and troubleshooting guides.
Build and maintain validated reference implementations, starter templates, and shared libraries for common communication use cases.
Guide the choice between synchronous request/response, asynchronous messaging, event streaming, and other communication patterns.
Establish practices for delivery guarantees, ordering, retries, deduplication, and idempotency.
Guide backpressure, buffering, batching, flow control, and consumer concurrency design.
Optimize throughput, latency, serialization overhead, network utilization, and memory pressure.
Establish practices for message schemas, data contracts, and backward- and forward-compatible schema evolution.
Guide partitioning, fan-out, replay, retention, and consumer-group design.
Establish practices for fault handling, dead-letter processing, observability, and recovery.
Partner with engineers through pair programming, design and code reviews, troubleshooting, and focused workshops to support consistent adoption of recommended practices.
Collaborate with Data Storage and Deployment Environments domain leads on cross-domain problems such as consumer lag, memory pressure, distributed tracing, and end-to-end throughput and latency.
Periodically reassess and update recommendations and documentation as technologies, workloads, and engineering needs change.
Track adoption and engineering outcomes against established baselines and report results to engineering leadership.
Perform other duties as assigned.