Senior Data Engineer, Unified Platform

DRW•Chicago, IL

7d•$160,000 - $250,000

About The Position

The Senior Data Engineer will design, build, and operate the data platform and managed services used by Traders, Quantitative Researchers, and Back-Office personnel within DRW’s Unified Platform organization. The role combines technical design ownership, software development, production support, and mentorship. The engineer will own projects and service components, work directly with platform users to solve technical problems, and contribute to metadata management and governance capabilities. The Data Ecosystem team builds platform capabilities, managed infrastructure, and metadata management systems that underpin DRW’s common query layer and support thousands of batch and streaming data pipelines. The position covers managed services for data storage, distributed query execution, batch and streaming processing, and workload orchestration, as well as automation, metadata tools, governance, reliability, upgrades, and data migrations. The annual base salary range is $160,000 to $250,000, depending on experience, qualifications, and relevant skill set. The position is also eligible for an annual discretionary bonus.

Requirements

At least 7 years of experience working with modern data technologies and/or building data-intensive distributed systems.
Expertise in Java and/or Python, with a track record of delivering high-quality, maintainable production software.
Strong familiarity with SQL, Bash, and Linux environments.
Experience building and operating managed data or infrastructure services for other teams, including operational automation and production support.
Demonstrated ownership of engineering projects or service components from technical design through deployment and ongoing operation.
Strong understanding of distributed systems trade-offs, including fault tolerance, concurrency, partitioning, and recovery from partial failures.
Production experience with batch and streaming systems, including practical knowledge of state management, retries, replay, and data consistency.
Experience with data processing or query technologies such as Flink, Spark, or ClickHouse, and ability to evaluate performance and operational trade-offs.
Experience with multiple storage technologies, including object storage, relational or NoSQL databases, and event streaming systems such as Kafka.
Familiarity with open table formats such as Iceberg or Delta Lake.
Practical experience with data modeling, schema governance and evolution, and data quality controls.
Experience integrating metadata management, data cataloging, or lineage capabilities into data systems.
Experience with data formats and serialization systems such as Arrow, Parquet, Protobuf, or Avro.
Experience with cloud-native data infrastructure, including Kubernetes, workflow orchestration, automated deployment, and production monitoring.
Strong communication and problem-solving skills, including troubleshooting with platform users, cross-team collaboration, prioritization, and delivery in a fast-paced trading environment.

Nice To Haves

Experience extending or scaling data catalogs or lineage tracking tools across multiple storage and compute technologies.
Experience implementing access controls, usage tracking, or other governance capabilities in shared data environments.
Familiarity with financial datasets and the needs of trading, quantitative research, or back-office users.

Responsibilities

Design, build, and operate managed platform services for data storage, distributed query execution, batch and streaming processing, and workload orchestration.
Own technical designs and implementation decisions for services and components, and lead delivery from requirements through deployment and support.
Automate provisioning, configuration, upgrades, and monitoring for managed services to reduce manual operations and improve deployment consistency.
Build and integrate metadata management tools, including data catalogs and lineage tracking, to help users discover datasets and understand their structure, ownership, and dependencies.
Implement schema governance, data quality checks, access controls, and usage tracking in line with established policies, working with security and data stakeholders on technical requirements.
Work with users to diagnose integration and performance issues, clarify technical requirements, and improve the managed services their workloads depend on.
Improve service and shared infrastructure reliability and performance through observability, capacity management, and testing.
Participate in production support and incident response, resolve root causes, and maintain operational runbooks.
Deliver platform upgrades and data migrations with validation and compatibility checks to protect downstream workloads.
Lead design and code reviews, document technical trade-offs, and contribute to shared architectural patterns.
Mentor less experienced engineers through collaboration and feedback, promoting maintainable code, automated testing, and sound operational practices.