Senior Software Engineer — Distributed Compute / Spark Systems

Granica•San Francisco, CA

6d•$160,000 - $240,000•Onsite

About The Position

Granica is hiring a Senior Software Engineer to build distributed compute systems for enterprise-scale data and AI workloads. You will work on the core infrastructure behind Crunch, Granica’s continuous optimization product for enterprise lakehouse data. This includes systems for distributed execution, workload optimization, query performance, scheduling, resource management, and compute cost reduction across petabyte- and exabyte-scale environments. You will own core systems that directly affect customer compute spend, query latency, workload reliability, cluster efficiency, and the performance of large-scale analytical data processing. This is a hands-on engineering role for someone who has deep systems experience and wants to build at the intersection of Spark, distributed query execution, lakehouse compute, workload scheduling, storage-aware optimization, and AI infrastructure. You will work on distributed compute systems involving Apache Spark, Spark SQL, Trino, Presto, Flink, Databricks, Snowflake-adjacent environments, cloud object stores, and lakehouse formats such as Apache Iceberg, Delta Lake, and Apache Hudi.

Requirements

Strong engineering depth in distributed systems, data processing systems, query engines, databases, or cloud infrastructure
Production experience with distributed compute or query systems such as Apache Spark, Spark SQL, Trino, Presto, Flink, Databricks, EMR, Glue, Hive, or similar systems
Hands-on experience improving performance, reliability, or cost efficiency for large-scale data-processing workloads
Understanding of distributed execution, query planning, scheduling, resource management, fault tolerance, and workload isolation
Experience with Spark internals, Spark SQL, Catalyst, Adaptive Query Execution, shuffle, joins, aggregation, spill, memory management, or task scheduling
Familiarity with lakehouse formats and columnar data such as Iceberg, Delta Lake, Hudi, Parquet, or ORC
Familiarity with cloud object storage systems such as S3, GCS, or ADLS and the performance tradeoffs of running distributed compute on top of them
Strong programming skills in Scala, Java, Go, Rust, C++, or similar systems-oriented languages
Curiosity about workload optimization, cost modeling, adaptive execution, and how compute efficiency affects AI and analytics at scale
A pragmatic builder’s mindset: rigorous, hands-on, and comfortable owning complex systems end to end

Nice To Haves

Experience contributing to Apache Spark, Spark SQL, Trino, Presto, Flink, Velox, DuckDB, DataFusion, Iceberg, Delta Lake, Hudi, Parquet, ORC, or related systems
Experience with Catalyst, Adaptive Query Execution, cost-based optimization, query planning, vectorized execution, or distributed runtime systems
Experience optimizing joins, aggregations, shuffles, scans, spills, caching, partitioning, skew handling, or task scheduling
Experience building workload schedulers, execution control planes, resource managers, or multi-engine compute platforms
Experience reducing compute cost or improving workload efficiency in large-scale production data environments
Background in query engines, distributed runtimes, storage-aware execution, indexing, caching, encoding, compression, or adaptive query optimization
Research or open-source contributions in distributed systems, databases, query processing, data processing, or cloud infrastructure

Responsibilities

Build distributed compute systems for large-scale analytical and AI workloads
Improve performance and cost efficiency across Spark, Trino, Presto, Flink, Databricks, and Snowflake-adjacent environments
Design workload-aware systems for query execution, resource allocation, scheduling, and compute optimization
Optimize execution performance across joins, aggregations, scans, shuffles, spills, caching, partitioning, and task scheduling
Build systems that learn from workload patterns and automatically improve execution plans, cluster usage, and compute efficiency
Develop infrastructure for adaptive workload routing, execution planning, and data-processing reliability across large customer environments
Debug performance bottlenecks across query execution, metadata, storage, network, memory, CPU, and distributed compute layers
Work with lakehouse tables and columnar formats such as Iceberg, Delta Lake, Hudi, Parquet, and ORC to improve end-to-end workload performance
Build systems that reduce compute waste caused by inefficient scans, poor partitioning, small files, skew, unnecessary shuffles, and suboptimal workload placement
Improve reliability and failure recovery for large distributed data-processing jobs
Implement algorithms in workload optimization, execution efficiency, cost modeling, and data-processing performance
Contribute to open-source or publish research when appropriate