Senior Software Engineer - Pipelines

Maxinsights Corporation • Santa Clara, CA

9d

About The Position

We are looking for a senior software engineer to own and evolve the pipeline infrastructure that powers our production workflows. This role is focused on pipeline scheduling, maintenance, support, scaling, reliability, and operational quality. The work goes beyond writing one-off scripts or adding tasks to an existing queue: the goal is to build pipeline systems that can keep running as workload, product needs, and team complexity grow. You will work across services, data movement, workflow orchestration, failure recovery, monitoring, developer experience, and production support. We care about strong engineering judgment, practical technical sense, clear communication, and the ability to turn recurring operational pain into maintainable systems.

Requirements

• Strong backend engineering fundamentals and experience owning production systems with real operational requirements.
• Practical experience with pipeline orchestration, schedulers, queues, async workers, data movement, or workflow engines.
• Good judgment around reliability, retries, idempotency, concurrency, state management, observability, and failure recovery.
• Ability to debug production issues across application logic, infrastructure, data dependencies, and external services.
• Clear technical and product sense: you can prioritize the right reliability and scaling problems without over-engineering.
• High standards for engineering quality, including tests, runbooks, refactoring, and long-term maintainability.

Nice To Haves

• Experience building or operating production data pipelines, workflow platforms, job schedulers, or internal automation systems.
• Experience with large-scale backfills, dependency management, rate limits, leases, distributed locks, or multi-tenant job execution.
• Experience improving operational excellence through dashboards, alerts, runbooks, self-healing automation, or incident review processes.
• Experience migrating fragile scripts or cron-based workflows into a more reliable platform without disrupting users.

Responsibilities

• Design, build, and evolve pipeline scheduling and orchestration systems for production workflows.
• Maintain and support critical pipelines, including monitoring, incident response, failure recovery, backfills, and operational tooling.
• Improve pipeline scalability, throughput, reliability, and cost efficiency as workload and product requirements grow.
• Define clear interfaces, ownership boundaries, retry semantics, idempotency, data contracts, and observability standards for pipeline execution.
• Collaborate with product, operations, data, and engineering partners to turn ambiguous workflow needs into robust delivery plans.
• Identify recurring manual work or fragile processes and replace them with simple, well-designed automation.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: No Education Listed