Staff Agentic Data Engineer – Data Platform

Nucleus Security•Remote, FL

3d•Remote

About The Position

Nucleus is a cybersecurity company focused on making vulnerability management more efficient through a fast, scalable platform that combines large volumes of security data and turns them into actionable insights. The Staff Agentic Data Engineer is a senior technical leadership role responsible for defining and evolving the architecture behind the data platform. The role suits an engineer who can solve complex data problems at scale, design systems for large and growing datasets, and establish patterns for other engineers to follow. The position owns large-scale data processing and analytics architecture, from ingestion and normalization through transformation, aggregation, and customer-facing analytics. It requires deep hands-on work across data modeling, analytical SQL, distributed processing, and database performance, as well as technical leadership through architecture guidance, standards, reusable patterns, and mentorship. The engineer partners with engineering and product teams to turn ambiguous requirements into scalable architectures and proactively addresses limitations affecting platform reliability, performance, and scalability. Nucleus describes its team as collaborative and hardworking, and says this role can directly influence data architecture, analytical processing, and platform scalability. The position is fully remote. Nucleus is committed to diversity, collaboration, teamwork, and equal consideration for qualified applicants.

Requirements

10+ years of professional software engineering, data engineering, or data platform experience, including significant experience designing and operating production data systems.
Deep expertise in relational and analytical data modeling, including schemas for large-scale analytical and reporting workloads.
Expert-level SQL and demonstrated experience designing, analyzing, and optimizing complex analytical queries over large datasets.
Strong experience with SingleStore/MemSQL or comparable distributed SQL/analytical database technologies.
Significant hands-on experience with Spark, Flink, or similar technologies, including designing and optimizing distributed data-processing workloads.
Strong understanding of distributed data processing, including partitioning, parallelism, data skew, resource utilization, and performance optimization.
Experience designing high-throughput job processing systems, including scheduling, concurrency management, retries, failure recovery, idempotency, and observability.
Experience designing data pipelines for batch ingestion, transformation, enrichment, aggregation, and downstream analytics.
Strong understanding of database performance characteristics, including indexing, partitioning, query execution plans, data distribution, and memory/compute trade-offs.
Experience identifying and resolving performance bottlenecks across application, database, and data-processing layers.
Proficiency in Python, Java, or another production programming language used to build data-processing and distributed systems.
Strong understanding of testing strategies for data-intensive systems, including unit, integration, data-quality, and end-to-end testing.
Proven ability to independently own technically ambiguous problems and drive them from investigation through production implementation.
Demonstrated ability to influence architecture and technical direction across multiple teams without relying solely on organizational authority.
Ability to clearly communicate complex data architecture and performance trade-offs to engineers, product stakeholders, and technical leadership.
Ability to collaborate effectively with distributed teams across multiple time zones.

Nice To Haves

Deep production experience with SingleStore/MemSQL, particularly for large-scale analytical workloads.
Advanced experience with pipeline performance tuning, partitioning strategies, memory management, and optimization of large distributed workloads.
Experience designing platforms supporting both high-volume ingestion and low-latency analytical queries.
Experience building or operating distributed job execution and orchestration systems.
Experience with large-scale cybersecurity, vulnerability, asset, telemetry, or event datasets.
Experience with vulnerability scanning technologies across the security stack, including SCA, SAST, DAST, IAST, vulnerability scanning, and container security.
Experience designing systems for incremental processing, backfills, replay/reprocessing, deduplication, and late-arriving data.
Experience with data observability and operational monitoring for large-scale processing workloads.
Experience integrating AI/LLM-driven capabilities with data platforms, including preparing and processing large datasets for AI-assisted analytical workflows.
Experience operating data-intensive applications on Linux and cloud infrastructure.
Familiarity with Agile/Scrum methodologies in a professional engineering environment.

Responsibilities

Own the architecture and evolution of large-scale data processing and analytics systems, from ingestion and normalization through transformation, aggregation, and customer-facing analytics.
Design and evolve data models for high-volume analytical workloads, balancing query performance, storage efficiency, maintainability, and scalability.
Design and optimize complex analytical SQL queries across large datasets, identifying bottlenecks and developing scalable solutions.
Architect distributed and asynchronous job processing systems that reliably process large volumes of data across concurrent workloads.
Design and evolve data-processing workflows using Spark, Flink, or similar technologies for transformation, enrichment, aggregation, and computation.
Drive effective use of SingleStore/MemSQL, including schema design, indexing and partitioning strategies, query optimization, and workload design.
Establish architectural patterns for batch processing, data transformation, job orchestration, failure recovery, idempotency, and reprocessing.
Diagnose complex performance and scalability issues across databases, distributed processing systems, application services, and infrastructure.
Partner with engineering and product teams to translate ambiguous analytical and data-processing requirements into scalable technical architectures.
Define technical standards and reusable patterns for data modeling, data processing, and analytical query development.
Provide technical leadership across engineering initiatives, influence architecture, and mentor engineers while remaining hands-on with implementation.
Identify architectural limitations proactively and lead improvements to platform scalability, reliability, and performance.