Sr. Software Engineer - Big Data, Data Lake

ON & UP HOLDINGS LLC • Remote, US,

3h • $120,000 - $150,000 • Remote

About The Position

We are seeking a senior engineer to help design and implement the core data platform powering our next-generation product architecture. This role will focus on building scalable data ingestion, transformation, and integration across multiple internal and external data sources, using AWS-native services. You will work closely with engineering leadership to establish a robust data lake and transformation layer that supports our APIs, microservices, and downstream analytics. The ideal candidate is comfortable working at the intersection of data engineering and application integration.

Requirements

• Strong leadership skills - comfortable both leading software engineers and implementing/deploying solutions.
• Strong cross-functional collaboration with product, engineering, and analytics teams.
• Strong communication skills.
• Strong problem-solving skills and knowledge of applied algorithms.
• Hands-on experience implementing and using AWS data services: S3, Glue (ETL/Jobs, Data Catalog, DataBrew), Athena, Lake Formation, Step Functions, Lambda, Kinesis Data Stream and API Gateway.
• Expertise in designing and optimizing data pipelines for high-volume, multi-source ingestion of structured and semi-structured data in a multi-tenant data architecture.
• Experience with entity resolution, data cleansing, data quality, and anomaly detection.
• Expert level skills developing back-end distributed systems and data pipelines using Python, SQL, Step Functions, Lambdas.
• Strong proficiency with TypeScript and Node.js for building backend services (REST/JSON APIs, async workflows, integration with AWS services).
• Comfort working with cloud data warehouses (Athena, Snowflake, Redshift, BigQuery, or similar).
• Experience building event-driven and notification systems (SNS/SQS, EventBridge, Pub/Sub, webhooks) and orchestration frameworks (StepFunctions or equivalent).
• Experience integrating AWS data pipelines with external platforms (e.g., Snowflake, Metabase, reporting tools, etc.).
• Hands-on experience applying security best practices for data storage, transfer, and API access.
• 7-10 years of professional experience working in a software development environment, ideally with exposure to big data and data-rich applications.
• Experience working across disciplines, partnering with BI, ML, and product teams to translate ideas into customer-facing features.
• 5+ years working in an AWS cloud-native environment.
• Authorization to work in the U.S. is required.

Nice To Haves

• Prior experience in data-rich SaaS products.
• Experience with auth & access control for data-driven applications in a multitenant environment.
• Familiarity with data governance concepts (lineage, permissions, quality checks).
• Deployment experience (AWS CDK, CI/CD, containerized services, serverless functions).
• Exposure to generative AI or LLM-based data exploration tools.
• Comfortable operating in an AI-first engineering organization where planning, design, implementation, testing, and documentation are frequently augmented by agentic engineering tools such as Spec Kit, Claude Code, Cursor, GitHub Copilot, and similar platforms.

Responsibilities

• Design and implement the core data platform powering our next-generation product architecture.
• Build scalable data ingestion, transformation, and integration across multiple internal and external data sources, using AWS-native services.
• Establish a robust data lake and transformation layer that supports APIs, microservices, and downstream analytics.
• Collaborate with engineering leadership.
• Operate in a team's on-call rotation to address complex problems in real-time and keep services operational and highly available.
• Translate business requirements into system design, architecture diagrams, and technical documentation.
• Present complex concepts or solutions to diverse audiences through clear and concise written communication and effective verbal communication.

Benefits

• Competitive salary
• Health benefits for eligible full-time employees
• 401k matching
• Subsidy for internet or cell phone
• Generous PTO days
• Paid holidays that incorporate two days to honor and celebrate your heritage, culture, or traditions
• Half Day Summer Fridays

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: No Education Listed