Senior Lead Software Engineer - SRE/Databricks

JPMorgan Chase & Co.•Jersey City, NJ

58d•$171,000 - $260,000

About The Position

We have an opportunity to impact your career and provide an adventure where you can push the limits of what's possible. The Chief Data & Analytics Office (CDAO) at JPMorgan Chase is responsible for accelerating the firm's data and analytics journey. This includes ensuring the quality, integrity, and security of the company's data, as well as leveraging this data to generate insights and drive decision-making. The CDAO is also responsible for developing and implementing solutions that support the firm's commercial goals by harnessing artificial intelligence and machine learning technologies to develop new products, improve productivity, and enhance risk management effectively and responsibly. As a Senior Lead Software Engineer at JPMorgan Chase within the AIML Data Platforms and Chief Data and Analytics Team, you will develop and deliver advanced technology products focused on data and analytics. Tackle complex cloud data platform challenges, especially around Datalakes Tools. In this role you will work in an agile environment, collaborating with cross-functional teams.

Requirements

Formal training or certification on software engineering concepts and 5+ years applied experience.
Extensive experience with AWS Databricks platform administration and engineering support is a MUST.
Strong understanding of SRE principles, including SLIs, SLOs, error budgets, and incident management.
Experience with monitoring tools, automation frameworks, and CI/CD pipelines.
Proficient in Python and/or Java application program development with use of automated unit testing.
Experience with terraform development and understanding of terraform enterprise.
Experience in delivering system design, application development, testing, and operational stability.
Knowledge of Big Data distributed compute frameworks like Spark, Glue, MapReduce etc.
Excellent troubleshooting, analytical, and communication skills.

Nice To Haves

Experience in Data pipelines using Spark.
Exposure to AWS & Databricks Platform administration.
Knowledge of containerization (Docker, Kubernetes) and orchestration.
Familiarity with distributed systems and large-scale data processing.

Responsibilities

Designs, implements, and maintains a managed AWS Databricks platform, and provides engineering and operational support for the platform to SRE and app teams.
Performs platform design, set-up and configuration, workspace administration, resource monitoring, providing engineering support to data engineering teams, Data Science/ML, and Application/integration teams.
Leads evaluation sessions with external vendors, startups, and internal teams to drive outcomes-oriented probing of architectural designs, technical credentials, and applicability for use within existing systems and information architecture.
Drives continuous improvement in system observability, alerting, and capacity planning.
Collaborates with engineering and data teams to optimize infrastructure and deployment processes, focusing on automation and operational excellence.
Executes creative software solutions, design, development, and technical troubleshooting with ability to think beyond routine or conventional approaches to build solutions or break down technical problems.
Develops secure high-quality production code, and reviews and debugs code written by others.
Identifies opportunities to eliminate or automate remediation of recurring issues to improve overall operational stability of software applications and systems.
Adds to team culture of diversity, opportunity, and respect.
Implements Site Reliability Engineering (SRE) best practices to ensure reliability, scalability, and performance of data platforms.
Develops and maintains incident response procedures, including root cause analysis and postmortem documentation.