Pricing & Derived Data - Senior Software Engineer

DRW•Chicago, IL

6d•$175,000 - $250,000

About The Position

The Pricing and Derived Data team within DRW’s Unified Platform is building a data platform that provides intra-day and end-of-day prices across the firm. The team prices millions of instruments across asset classes and exchanges in near-real-time, serving clients ranging from low-latency trading systems to researchers working with large historical datasets. The platform will extend the existing system to meet rapidly growing demand and varied access patterns. The successful candidate will help shape the platform’s future direction, including ensuring consistent prices across warehouse queries, streams, and gRPC APIs; enabling clients to load a snapshot and transition to live updates without gaps or duplicates; and making technology choices about where to use Elixir or other technologies. The current architecture is mostly Elixir, with gRPC for inter-service streaming and PostgreSQL, Kafka, and ClickHouse in the data layer. The role is suited to a senior generalist with substantial experience building data products and leading significant projects from design through production operation, including architectural decisions. Relevant areas include event streaming, change data capture, open table formats, and stream and batch processing. Prior Elixir or financial-domain experience is not required, but candidates should be interested in those areas. DRW values humility, respect, learning, adaptability, independent project ownership, collaboration, and strong listening and communication skills.

Requirements

Considerable experience building data products.
Experience leading significant projects from design through production operation, including responsibility for architectural decisions.
Ability to explain technology choices and the reasons for making or reconsidering them.
Deep understanding of the architectural requirements suited to technologies used in event streaming, change data capture, open table formats, and stream and batch processing.
Examples of leading projects such as high-volume or low-latency data processing systems, scalable ingestion pipelines, or operationally supporting time- and correctness-sensitive data products.
Humility and respect for others, and trust in teammates.
Ability to learn, adapt, and grow.
Ability to drive projects independently, with a preference for collaboration.
Excellent listening and communication skills.

Nice To Haves

Experience with Kafka, Delta Lake, Iceberg, ClickHouse, Spark, or Flink.
Prior experience with Elixir or the financial domain is not required, but interest in these areas is desired.

Responsibilities

Help build and shape a data platform for intra-day and end-of-day pricing across the firm.
Make architectural decisions about data platform technologies and system design.
Address consistency of prices across warehouse queries, streaming subscriptions, and gRPC APIs.
Enable clients to load a snapshot and transition to live updates without gaps or duplicates.
Design and build high-volume and/or low-latency data processing systems, including ingestion, storage, and distribution.
Scale ingestion pipelines handling thousands or tens of thousands of events per second on a single node and/or across multiple nodes.
Operate and support data products with time-sensitive and correctness-sensitive delivery requirements.
Drive projects from design through production operation and collaborate with teammates.