Senior Software Engineer (Big Data)
About The Position
Bitsight is seeking an experienced Big Data Engineer to join its engineering team and help develop and operate the data pipeline at the core of its Security Ratings product. The pipeline ingests security intelligence from multiple sources, processes terabytes of data through a proprietary rating algorithm, and delivers daily ratings for more than 1,000,000 organizations. It is built on AWS and designed for reliability and scalability. The role involves architecting and evolving the pipeline to handle larger data volumes, including through cloud data warehouses. The engineer will integrate new data sources, uncover features in existing data, refine the rating algorithm, and collaborate with data science and research teams to ensure the quality and integrity of Bitsight’s data. The team values adaptable generalists, strong critical thinking, and collaboration. The position is remote within the United States, with Boston or Raleigh preferred.
Requirements
- Prior experience building and operating large-scale data pipelines.
- Strong proficiency in modern Java programming.
- Strong computer science fundamentals in design, data structures, and algorithms.
- Experience working with a team of engineers, designers, and product owners to deliver a product to end users.
- Excellent written and verbal communication skills in English.
- Strong collaboration skills and ability to work with a variety of technologies and learn new concepts.
Nice To Haves
- Experience with Python.
- Experience with Hadoop or Spark ecosystems.
- Experience with Data Lakehouse technologies, such as Apache Iceberg.
- Experience with Elasticsearch or OpenSearch.
- Experience with CI/CD and Kubernetes.
- Experience with modern AI development tools, such as GitHub Copilot, Cursor, or LLMs.
- Experience working in the IT security domain.
Responsibilities
- Build and operate large-scale data pipelines.
- Help architect and evolve the pipeline to support greater data volumes using cloud data warehouses.
- Integrate new data sources, surface new features from existing data, and refine the rating algorithm.
- Collaborate with data science and research teams to maintain trust in the quality and integrity of data.
- Work with engineers, designers, and product owners to deliver products to end users.
- Use continuous, incremental development methodologies.
Benefits
- Bitsight states that it offers best-in-class benefits, but does not specify individual benefits in the posting.
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Senior
Education Level
No Education Listed
