Senior Data Engineer

Critical Propulsion•Dallas, TX

About The Position

Databricks pipelines that run in production, not notebooks that run on your laptop. You build, test, and maintain the ingestion infrastructure that moves data from source to consumption. You write resilient, fault-tolerant code that handles real-world data at scale inside 5-day pulse cycles. Critical Propulsion amplifies human delivery. Our swarm model pairs 3-4 senior operators with AI agents to cut cycle times 30-50% and multiply effective capacity 2-5x versus traditional teams. Zero handoffs. Single timezone. No offshore blended teams. No sprint theater. Week 1 is productive delivery, not onboarding. We define outcomes and trust operators to get there.

Requirements

Experience building and maintaining data pipelines on Azure using ADF, Databricks, PySpark, SQL in a production environment.
Professional-level Python coding skills, including debugging, optimization, and modular design.
Ability to write code that is readable, maintainable, and extendable by other operators.
Experience with agentic development, using AI agents for tasks beyond simple autocomplete.
Ability to design for stability, traceability, and fault tolerance.
Proven ability to operate as an independent contributor without constant supervision.
Direct and proactive communication skills.
Autonomous operation, minimizing unnecessary meetings and communication overhead.

Nice To Haves

Deep hands-on experience with Azure Data Factory and Azure Databricks in production environments.
Strong PySpark skills for large-scale data processing and transformation.
Experience building or consuming RESTful APIs, especially Python-based API architecture.
Familiarity with near-real-time data processing patterns and streaming ingestion.
Hands-on experience with agentic AI tooling: Claude Code, Cursor, Copilot Workspace, or similar.
Comfort with spec-driven development, building pipelines from detailed specs within a single pulse cycle.
Background in consulting or client-facing delivery.
Prior experience on small, autonomous teams.

Responsibilities

Build, test, and maintain the ingestion infrastructure that moves data from source to consumption.
Write resilient, fault-tolerant code that handles real-world data at scale inside 5-day pulse cycles.
Build and maintain data pipelines on Azure.
Write Python at a professional level, including debugging, optimization, and modular design.
Embrace agentic development, delegating real work to AI agents and reviewing their output critically.
Design for stability, traceability, and fault tolerance.
Operate as an independent contributor, seeing the outcome, picking up the work, and closing it.
Communicate directly and proactively about blockers or better approaches.
Operate autonomously, avoiding unnecessary meetings and status updates.