Senior Data Engineer

Quartermaster•New York, NY

4d•$210,000 - $250,000•Hybrid

About The Position

Quartermaster is building the world's most comprehensive maritime intelligence platform. Our SmartMast™ system transforms commercial and civilian vessels into a persistent, distributed sensing network—combining HD video, AI, radar, RF sensing, and AIS to deliver real-time maritime domain awareness at global scale. With 600+ sensors deployed across 25+ countries and more than 400,000 vessels identified outside of AIS, we are setting a new standard for what ocean surveillance and safety can look like. We are a mission-driven, high-velocity team building dual-use technology for defense agencies, coast guards, and commercial maritime operators. Quartermaster collects extraordinary data: AIS signals, electro-optical imagery, radar tracks, RF observations, GPS trails, and ADS-B feeds—all arriving continuously from a distributed fleet of sensors crossing every major body of water on earth. We need a Senior Data Engineer who can turn that raw stream into a reliable, queryable, and analytically powerful foundation. This is a hyper-specific, mission-critical role. Beyond standard pipeline construction, you will implement best data engineering practices to directly drive our strategic growth. You will build the data infrastructure and analytical tools required to utilize Covered Area Intelligence (CAI) to direct our global fleet expansion and sensor emplacement. Furthermore, you will build the backend tools and data models that allow our application to intelligently select the best vessels to fill critical gaps in our coverage. We want the kind of engineer who has operated at scale: someone with the rigor and systems thinking of the best data engineering teams in the world.

Requirements

7+ years of dedicated data engineering experience (or equivalent mastery) architecting and operating production environments handling high-volume, real-time streaming data.
Proven track record of designing data infrastructure to solve spatial optimization or network growth problems (e.g., routing, network expansion, or geographic coverage mapping).
Deep architectural expertise in Python and a strong command of advanced NoSQL/SQL query patterns; hands-on experience optimizing high-scale, production MongoDB instances is highly preferred.
Production-proven experience evaluating, selecting, and scaling streaming and batch frameworks (e.g., Kafka, Kinesis, Spark, Flink) to handle multi-million event-per-day workloads.
Demonstrated ability to build developer-centric data platforms, creating clean abstractions and reliable environments that application developers, ML engineers, and data analysts can seamlessly build upon.
Expert-level mastery of data modeling, specifically tailored for complex time-series, large-scale geospatial, and event-driven workloads.
Strong systems integration background, with a deep understanding of API design, microservices architecture, and managing strict data contracts between data systems and product application layers.
Advanced infrastructure fluency: comprehensive experience architecting AWS ecosystems (S3, Glue, Kinesis, DocumentDB, Redshift), infrastructure-as-code, container orchestration, and advanced CI/CD for data workflows.
Uncompromising engineering standards: a history of championing rigorous code reviews, automated testing, comprehensive documentation, and designing systems with operational self-healing and reliability at their core.

Nice To Haves

Experience with AIS data, maritime vessel tracking, or geospatial data processing (H3, GeoJSON, ArcGIS, PostGIS).
Background in defense, intelligence, or government data environments with exposure to data classification and access control requirements.
Experience building feature stores or serving layers that bridge offline data processing with real-time application needs.
Prior work at a high-scale consumer or enterprise tech company with rigorous data engineering practices.

Responsibilities

Lead the architecture and implementation of scalable, fault-tolerant data pipelines that ingest, normalize, and enrich AIS signals, vessel detections, radar tracks, and imagery metadata from our global SmartMast fleet.
Architect the technical strategy for Covered Area Intelligence (CAI), designing the foundational data models and infrastructure that directly guide global fleet expansion and optimal sensor emplacement.
Design and own the backend systems, tools, and analysis layers that empower our core application to programmatically identify coverage gaps and dynamically select the best target vessels to fill them.
Define the vision and standards for our core data platform, establishing long-term data models, schema evolution standards, partitioning strategies, and retention policies across MongoDB and cloud storage.
Design enterprise-grade APIs, data services, and integration layers that enable application engineers and data scientists to securely consume processed data for real-time product features like vessel matching and coverage analytics.
Partner strategically with AI/ML and Data Science leadership to operationalize complex model outputs—championing the transition of experimental pipelines into highly monitored, version-controlled, production-grade data products.
Establish and enforce strict data observability standards, implementing robust frameworks for latency tracking, SLA/SLO monitoring, data quality validation, and end-to-end lineage documentation.
Diagnose and ruthlessly optimize systemic bottlenecks in data access patterns, ensuring high performance for geospatial visualizations, bulk queries, and customer-facing portal features.
Drive the technical roadmap for our data stack, evaluating and introducing modern tooling to mature our data engineering practices as our global footprint scales.
Mentor junior/mid-level engineers and serve as the technical authority on data contracts, architectural reviews, and data governance best practices.