Software Development Engineer, Lead

Matrix Medical Network

11h•$122,000 - $183,000•Remote

About The Position

We are seeking a Lead Software Development Engineer to architect and deliver our next-generation cloud data platform while leading several high-impact strategic initiatives, including Control Tower, Next-Gen Data Ingestion, and an AI Rule-Based Call List Generator. This is a hands-on technical leadership role focused on modernizing legacy, monolithic data processes into scalable, cloud-native architectures. The ideal candidate brings deep expertise in the Azure data ecosystem, real-time streaming, and AI-enabled data workflows, and thrives in building modern platforms from the ground up.

Requirements

8+ years of hands-on experience in software and data engineering, with strong cloud and distributed computing experience.
Deep expertise in the Azure ecosystem, including Databricks, Azure Data Factory, Synapse, Event Hub, Azure Functions, and ADLS Gen2.
Strong programming skills in Python, SQL, and PySpark, including Spark SQL and performance tuning.
Advanced Databricks experience, including Auto Loader, Unity Catalog, Delta Lake internals, and Databricks Workflows.
Strong understanding of Lakehouse architecture and Medallion (Bronze/Silver/Gold) design patterns.
Experience designing secure cloud solutions using Azure Key Vault, RBAC, and Managed Identities.
Solid data modeling background (Star Schema, Snowflake, Data Vault 2.0).
Experience with real-time and event-driven architectures (Kafka, Event Hub, Spark Streaming, Delta Live Tables).
Familiarity with modern data quality frameworks (Great Expectations, Unity Catalog Expectations).
Experience building REST API–based ingestion pipelines using Databricks, Azure Functions, or Logic Apps.
Experience with iPaaS platforms (SnapLogic, MuleSoft, Boomi, Azure Integration Services).
Hands-on experience with Azure DevOps CI/CD pipelines.
Proficiency with Infrastructure as Code (IaC) tools such as Terraform.
Proven experience modernizing legacy ETL/ELT systems (SSIS, Informatica, stored procedures, on-prem batch jobs).
Experience supporting high-volume, real-time, or operational data environments (e.g., call center, healthcare, transactional systems).
Strong communication, leadership, and problem-solving skills.
Ability to collaborate effectively across engineering, architecture, cloud, and business teams.
Comfortable owning initiatives end-to-end and translating business needs into scalable technical solutions.

Nice To Haves

Exposure to AI/ML platforms such as Azure Machine Learning, Databricks MLflow, feature stores, and model pipelines.
Experience supporting AI-driven systems (rule engines, decisioning workflows, recommendation engines).
Experience in healthcare or other regulated industries.
Azure certifications such as DP-203 or Azure Solutions Architect.

Responsibilities

Architect and build a modern, scalable Azure-based cloud data platform.
Lead the migration and modernization of legacy on-prem and monolithic data pipelines.
Define and establish data engineering frameworks, standards, and best practices.
Design and build scalable ETL/ELT pipelines using Azure Databricks, Azure Data Factory, Synapse Analytics, Event Hub, Azure Functions, and ADLS Gen2.
Develop real-time and event-driven data pipelines using Spark Streaming, Delta Live Tables, Kafka, and Event Hub.
Optimize Spark workloads with strong focus on performance, reliability, and scalability.
Control Tower: Lead technical design and hands-on engineering implementation.
Next-Gen Data Ingestion Platform: Architect automated, scalable, and reusable ingestion pipelines.
AI Rule-Based Call List Generator: Partner with Analytics and AI teams to build robust data pipelines that power AI-driven decisioning systems.
Provide technical leadership, mentorship, and guidance to engineers.
Collaborate closely with Data Architecture, Cloud Engineering, Product, Analytics, and Business teams.
Champion DevOps, CI/CD, automation, and modern engineering practices across the platform.