Senior Platform Engineer - ClickHouse

Qualys•Foster City, CA

About The Position

Qualys is seeking a highly motivated and talented Senior Software Engineer to work on its next-generation Data Platform and middleware, with a particular focus on ClickHouse. Working with a team of engineers and architects, this role prototypes, designs, develops, and supports a highly scalable SaaS cloud-security data platform. The platform is microservices-based, processes more than 100 million transactions and terabytes of data per day, and uses open-source technologies. The engineer owns data technologies and services that provide a data platform as a service to Qualys applications. The role includes scaling and operating the platform across on-premises and cloud environments, supporting high transaction volumes and hundreds of terabytes of data, and delivering resilient client SDKs. The engineer also advocates for the technology stack, contributes to open source where needed, instruments day-one and day-two operations, and evaluates solutions to improve scalability, availability, and cost effectiveness.

Requirements

BS/MS degree in Computer Science, Applied Math, or a related field.
Expertise in ClickHouse, including the areas outlined in the job description.
5+ years of hands-on experience developing SaaS software products and solutions for service-oriented deployments in public or private clouds.
Data-driven critical-thinking and problem-solving expertise.
Ability to debug systems involving software and people.
Ability to write high-quality, production-ready code that is readable, well-tested, and built with well-designed APIs.
Passion for innovation and ability to work within agile software development methodologies.
Experience with infrastructure orchestration and automation tools such as Terraform.
Hands-on experience using CI/CD pipelines.
Strong analytical and debugging skills.
Java programming skills, including object-oriented design, prototyping, development, testing, and profiling.
Knowledge of JVM concepts such as garbage collection, heap, stack, profiling, and class loading.
Strong knowledge of data structures and algorithms.
Ability to clearly articulate and communicate technical concepts within and across teams.

Nice To Haves

Experience with distributed systems such as Cassandra, Redis, Elasticsearch, Kafka, Ceph, Flink, Spark, or similar technologies.
Experience applying data encryption and data security standards.
Experience with container and orchestration technologies such as Docker and Kubernetes.
Experience with monitoring tools such as Graphite, Grafana, and Prometheus.
Experience with HashiCorp technologies such as Consul, Vault, Terraform, and Vagrant.
Experience with configuration management tools such as Chef, Puppet, or Ansible.
In-depth experience with continuous integration and continuous deployment pipelines.
Exposure to Maven, Ant, or Gradle for builds.

Responsibilities

Apply deep expertise in ClickHouse internals and architecture, including engines, partitioning, ordering keys, indexing, and query execution.
Design and review schemas based on real-world read and write patterns, optimizing ingestion, query performance, and storage efficiency.
Operate ClickHouse at scale on premises or in the cloud, including replication, sharding, workload management, performance tuning, and production troubleshooting.
Drive capacity planning for CPU, memory, storage I/O, and network resources, accounting for workload growth, concurrency, background merges, and reliability requirements.
Engineer services that run data platforms handling millions of transactions per second and scaling to hundreds of terabytes of data.
Drive the data source and provide the data platform as a service to Qualys applications.
Advocate for the owned technology stack and contribute to open-source projects when enhancements are needed for the Qualys ecosystem.
Instrument day-one and day-two operations for data technologies supporting hundreds of thousands or millions of operations per second.
Migrate workloads, burst into cloud environments, and scale the stack with zero or minimal downtime.
Deliver resilient client SDKs that enable applications to communicate with the data source.
Continuously evaluate alternative solutions to address scalability, availability, and cost-effectiveness.