Senior Sustaining and Forward Deployed Engineer (AWS & Databricks)

Abacus Insights•Boston, MA

1d•Remote

About The Position

Abacus Insights is hiring a senior individual contributor responsible for production operations, incident response, and post-launch reliability across its platform. The role combines site reliability engineering, production operations, deep technical problem-solving, customer-facing forward-deployed work, and hands-on software development and automation. The engineer will own complex, ambiguous, high-impact production issues, particularly those involving AWS infrastructure, Databricks workloads, and large-scale data pipelines. They will lead technical incident response, debug live systems, engage directly with customers on escalations and deployments, and turn operational and customer learnings into durable product and platform improvements. The role requires production-quality code where it improves reliability and operability, primarily in Python, with JVM-based systems as needed. The position includes technical leadership without formal authority, mentoring, and close collaboration with Product, Engineering, Data, Customer Success, and customer teams. Compensation is based on experience, skills, and location and includes base salary, eligibility for performance bonuses, and equity grants. The role offers work-from-anywhere flexibility.

Requirements

10+ years of experience in software engineering, SRE, sustaining engineering, or production operations.
Deep hands-on experience operating production systems in AWS.
Strong experience troubleshooting Databricks and large-scale data platforms.
Proficiency in Python and experience building production services or tooling.
Strong understanding of distributed systems, incident management and root cause analysis, monitoring, alerting, and observability.
Experience with CI/CD pipelines that leverage Infrastructure as Code.
Proven ability to own problems end-to-end, from detection through permanent resolution.
Excellent communication skills, especially during incidents and customer escalations.
Ability to work backward from customer impact to root cause across systems and codebases and deliver fixes in environments with minimal documentation.
Strong instinct for operational risk and ability to proactively identify failure modes and harden systems before customer impact.

Nice To Haves

Experience in healthcare, health insurance, or regulated data environments.
Familiarity with Kubernetes (EKS), EMR, or Lambda.
Familiarity with Spark internals.
Experience with Snowflake or similar data warehouses.
Experience with FHIR, MDM systems, or entity resolution.
Prior experience in SWAT, escalation engineering, or tiger-team roles.
Experience contributing to or operating within SRE/on-call programs.

Responsibilities

Act as a senior technical escalation point during production incidents and lead real-time triage, mitigation, and recovery.
Drive root cause analysis focused on systemic, long-term fixes; identify recurring failure patterns and advocate for architectural or operational improvements.
Partner with Customer Success and Engineering to manage customer impact during incidents.
Own post-launch reliability, stability, and operational quality of core systems; investigate and resolve complex field issues and production defects.
Ensure fixes developed during incidents or customer escalations are incorporated into the core product.
Improve service operational readiness through runbooks, monitoring, and alerting, and reduce toil by automating repeated manual work.
Work directly with strategic customers on production-grade technical challenges, deployments, integrations, and escalations.
Translate customer learnings into product, platform, and operational improvements, and contribute reusable tools, playbooks, and best practices.
Troubleshoot AWS compute, storage, networking, IAM, and security, as well as Databricks jobs, clusters, and Spark-based data pipelines.
Debug performance degradation, scalability issues, job failures, and data correctness problems; partner with platform and data teams to improve reliability, scale, and operability.
Write production-quality code to automate operations, improve reliability and observability, reduce manual intervention, and decrease incident frequency.
Review code with an emphasis on operability, resiliency, and maintainability, and advocate for engineering standards that make systems operable.
Provide technical leadership, mentor engineers through pairing, reviews, and incident leadership, and collaborate across Product, Engineering, Data, and Customer teams.
Operate effectively in high-pressure, ambiguous environments, especially during customer-impacting incidents.