Senior C++ Storage Engineer
Voltron Data
·
Posted:
July 31, 2023
·
Remote
About the position
We are seeking a highly motivated Senior C++ Storage Engineer to join our team at Voltron Data. In this role, you will have the opportunity to support and contribute to the growth of the Voltron Data and Apache Arrow ecosystems. Working closely with our development teams, you will be responsible for implementing performant storage and I/O functions for a wide range of networked, cloud, and local storage solutions. As a Series A, venture-backed startup, we are committed to making a significant impact in the data analytics field with Apache Arrow. We value diversity and inclusivity, and prioritize employee growth and success in shaping our organization's future.
Responsibilities
- Implement performant storage and I/O functions targeting networked, cloud, and local storage solutions
- Learn about the Apache Arrow memory layout, compute primitives, and APIs
- Familiarize oneself with different partners for compute kernels and the query execution engine on Apache Arrow
- Embrace the Apache development process
- Implement new high-performance storage and I/O primitives
- Benchmark existing I/O library functions to identify bottlenecks
- Discover and implement optimizations in data reads and writes
- Participate in peer code review of all PRs related to file storage and interacting with different filesystems
- Contribute to technical discussions and technical design documents
- Develop a comprehensive set of low-level benchmarks for I/O functions targeting various storage technologies
- Ensure compatibility and performance of filesystem interactions across platforms
- Identify and build reusable software components for a high-quality and maintainable codebase
- Analyze I/O throughput in a massively parallel and distributed query engine to identify inefficiencies and propose solutions
- Build high-quality storage solutions across the Voltron Data and Apache Arrow ecosystems
- Strong experience in C++ development, especially using Modern C++
- Experience with various data lake storage technologies such as S3
Requirements
- Highly motivated Senior C++ Storage Engineer
- Experience in implementing performant storage and I/O functions targeting networked, cloud, and local storage solutions
- Familiarity with Apache Arrow memory layout, compute primitives, and APIs
- Knowledge of different partners for compute kernels and the query execution engine on Apache Arrow
- Understanding and embracing the Apache development process
- Ability to implement new high-performance storage and I/O primitives
- Proficiency in benchmarking existing I/O library functions to identify bottlenecks
- Skill in optimizing data reads and writes
- Participation in peer code review related to file storage and interaction with different filesystems
- Contribution to technical discussions and technical design documents
- Development of comprehensive benchmarks for I/O functions targeting various storage technologies
- Ensuring compatibility and performance across different platforms (Linux, MacOS, and Windows)
- Building reusable software components for a high-quality and maintainable codebase
- Analysis of I/O throughput in a massively parallel and distributed query engine
- Identification and resolution of inefficiencies in storage
- Strong experience in C++ development, particularly using Modern C++
- Familiarity with various data lake storage technologies, such as S3
Benefits
- Work from Anywhere - Payroll and Benefits in 150+ Countries
- Unlimited PTO
- Medical, Dental, and Vision
- Retirement [USA Only]
- Home Office Budget
- Continuing Education Budget