AWS/Databricks Cloud Data Engineer

Moser Consulting • Indianapolis, IN

4d • $85,000 - $125,000 • Hybrid

About The Position

Moser Consulting is seeking a Senior Cloud Data Engineer for its consulting team and genomics organization. The role provides senior-level strategic technical leadership alongside two existing engineers to build AWS-based data pipelines supporting genomic data science models. Hands-on production AWS pipeline-building experience is the top priority.

The engineer will design, build, and maintain scalable data pipelines and acquisition workflows across cloud and on-premises environments, covering ingestion, transformation, and delivery. The role collaborates with cross-functional teams to translate business requirements into robust data solutions and follows best practices in architecture, cloud infrastructure, documentation, and continuous improvement. It includes building AWS-native pipelines to ingest and store genomic data retrieved via API calls, and generating and managing sequencing and post-sequencing data-processing workflows.

Moser describes its consulting environment as collaborative and fast-paced, and the work schedule can be hybrid or onsite at either of the company's two Indianapolis offices. The stated salary range is $85,000–$125,000.

Requirements

• Proven, hands-on experience designing and building AWS-based data pipelines end to end, from API ingestion through storage and delivery.
• Hands-on experience working with High Performance Computing (HPC) clusters.
• Experience with dimensional modeling, slowly changing dimensions (SCDs), and medallion architecture.
• Previous cloud experience; AWS is preferred, with hands-on exposure to Lambda, S3, and Docker.
• Experience implementing and managing role-based access control (RBAC), including group and permissions management.
• Strong Python experience for pipeline development, ETL, and data engineering, including complex datasets from disparate batch and streaming sources.
• Experience with SQL and Python data tooling to build scalable, optimized ETL/ELT pipelines, with attention to low latency, high throughput, and cost-efficiency.
• Experience developing APIs in both consumer and producer capacities using REST APIs.
• CI/CD experience with modern DevOps tooling and deployment pipelines.
• Experience with API-based access patterns for retrieving and integrating data from external systems.
• Experience with Amazon ECS and/or EKS for container orchestration.
• Ability to learn new technologies quickly and apply them to solve problems.
• Strong attention to detail and commitment to high-quality work.
• Excellent oral and written communication skills.
• Professionalism and ethics, including trust, a positive attitude, commitment, honesty, collaboration, and approachability.
• Ability to adapt in a fast-paced consulting environment, manage time effectively, and handle multiple tasks simultaneously.
• Lambda functions must be developed and deployed via code or infrastructure-as-code, such as CloudFormation, Terraform, CDK, or SAM; GUI- or console-based Lambda creation is not acceptable.

Nice To Haves

• Pandas experience.
• Experience with version control systems such as Git and agile development frameworks.
• Ability to design and document end-to-end testing plans for Python-based applications.
• PySpark experience for large-scale distributed data processing.
• Experience with Datadog or similar observability and monitoring platforms.
• Familiarity with Dagster or other modern data orchestration frameworks.
• Terraform or other infrastructure-as-code experience for cloud resource provisioning.
• Database experience with AWS RDS, MySQL, or PostgreSQL.
• Familiarity with D3.js for custom data visualizations.

Responsibilities

• Participate in team stand-ups and other agile ceremonies to support project delivery.
• Build and maintain data acquisition pipelines from databases, flat files, REST APIs, and other sources.
• Apply development design patterns and best practices across workloads.
• Support dimensional modeling and semantic-layer efforts for analytics under senior guidance.
• Troubleshoot and resolve pipeline failures, query performance issues, data quality concerns, notebook timeouts, data-model inconsistencies, semantic-model refresh errors, and capacity throttling using monitoring and query optimization techniques.
• Participate in requirements-gathering meetings and translate business needs into technical specifications.
• Pursue ongoing learning through online training, certifications, and mentorship opportunities.
• Create and maintain technical and operational documentation, data-flow diagrams, and data-mapping documents.
• Collaborate with cross-functional teams and communicate progress to technical and non-technical stakeholders.
• Build and maintain AWS-native pipelines using Lambda, ECS, EKS, and S3 to ingest and store genomic data retrieved via API calls.
• Generate and manage sequencing and post-sequencing data-processing workflows supporting genomics data science models.
• Develop and deploy Lambda functions through code or infrastructure-as-code rather than GUI or console-based creation.
• Develop scalable, optimized ETL/ELT pipelines to ingest, transform, and load large, complex datasets from batch and streaming sources, with attention to latency, throughput, and cost-efficiency.

Benefits

• Training opportunities and avenues for skill enhancement.
• Fully invested 401(k) plan.
• PPO and HDHP medical plans.
• Employer-paid dental and vision plans.
• Onsite fitness center.
• Wellness program.
• Regular catered lunches.
• Hybrid or onsite schedule with flexibility to work from either of the two Indianapolis offices.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: No Education Listed