Full Stack Data Discovery Engineer

Apryse•Boston, MA

1d•$90,000 - $138,000

About The Position

We are hiring a Full-Stack Data Discovery Engineer to design and ship end to end systems that uncover technology usage across public and private ecosystems. You will build innovative backend pipelines (APIs, crawlers, document finger printers, package-registry miners, etc.) and frontend dashboards or analysis that transform raw signals into actionable insights. You will combine engineering skill with investigative creativity to discover patterns across ecosystems and turn them into actionable intelligence.

Requirements

Education: Bachelor's degree in Computer Science, Engineering, Library Science, Information Systems, Data Management, or a related field (Master's degree preferred).
Experience: 1-5 years of proven experience as a full-stack developer and data engineer. Creating the initial inception and idea of the project.
Technical Skills: Back-end: Python, SQL, Java and Node.js
Front-end: Modern JS/TS + React, component libraries, auth patterns, state mgmt.
Data & search: schema design, dedup/near‑dup logic, Elasticsearch/OpenSearch; building usable search/triage UIs.
Acquisition: Scrapy/Playwright/Puppeteer; API design with rate‑limit/backoff; ethical crawling.
Experience with cloud-native architecture and containerization. Familiarity with metadata standards (e.g., Dublin Core, XML) and data management tools.

Nice To Haves

Knowledge of data visualization tools (e.g. Power BI, Tableau) to present findings.
Experience building internal platforms/tools used by end users or GTM teams.

Responsibilities

Own the full stack: Design, build and optimize scalable data pipelines to discover OSINT and software usage across a wide public ecosystem.
Pipeline development: Develop APIs, microservices, crawlers, document fingerprinting to gather data securely and efficiently. Implement backoff/caching, data normalization, and persist to SQL/NoSQL indexes
Data Discovery: Conduct systematic searches across the web, public databases, developer ecosystems and other platforms to identify potential external data repositories relevant to organizational objectives.
Metadata and Attribution Analysis: Programmatically uncover and analyze metadata associated with identified data sources to understand data structure, content, quality, and potential use cases.
Signals & scoring: develop heuristics/ML‑lite ranking to identify relevant artifacts , deduplicate, and assign confidence scores.
Data Governance: Ensure data quality, security, compliance and governance.
Productize discovery: build internal tools that let non‑engineers run searches, review candidates, and export leads—fast and safely
Documentation and Reporting: Document data structures, origins (data lineage), and quality issues. Create clear, concise reports and presentations to communicate findings and recommendations to technical and non-technical stakeholders.
Collaboration: Work closely with data stewards, data architects, and internal business units to define data requirements and facilitate the integration of new data sources.
Innovation and Scale: Continuously explore new data sources, improve attribution logic and propose ML-based enhancements to finding and classifying data.

Benefits

Competitive salary commensurate with experience & qualifications.
A comprehensive extended benefits package including health, dental and vision for you and your family.
A great team environment and resources, supporting you to do the best work of your life and providing unlimited career growth potential.
Annual recurring WFH allowance for you to purchase items you need for your home office.
On going support for learning development so you can master your craft.
Work with the hardware you're most comfortable with (Windows or Mac).
Diverse and inclusive workplace where we all learn from each other.
Excellent work-life balance with a flexible remote work environment.
Competitive salary commensurate with experience and qualifications.
A comprehensive extended benefits package including health, dental and vision for you and your family, with company paid offerings.
401K savings program with company match.
Generous paid time off (PTO) is offered to support the ability to rest and recharge.
A great team environment and resources, supporting you to do the best work of your life and providing unlimited career growth potential.
Highly autonomous and entrepreneurial environment.
Annual recurring WFH allowance for you to purchase items you need for your home office.
Ongoing support for learning development so you can master your craft.
Work with the hardware you're most comfortable with (Windows or Mac).
Diverse and inclusive workplace where we all learn from each other.