AWS / Databricks Cloud Data Engineer

Moser Consulting • Indianapolis, IN

4d • $115,000 - $149,000 • Hybrid

About The Position

We are seeking a Senior Cloud Data Engineer to join our consulting team. In this role, you will design, build, and maintain scalable data pipelines and acquisition workflows across cloud and on-premises environments, contributing to the full spectrum of data engineering – from ingestion through transformation and delivery. You will collaborate with cross-functional teams to translate business requirements into robust data solutions, while upholding best practices in architecture, cloud infrastructure, documentation, and continuous improvement.

Requirements

• Experience with dimensional modeling, SCDs, and medallion architecture
• Previous cloud experience required; AWS preferred with hands-on exposure to Lambda, S3, and Docker
• Knowledge of Azure Active Directory group management and role-based access control (RBAC)
• Strong Python experience across pipeline development, ETL, and data engineering tasks, with exposure to complex datasets from disparate batch and streaming sources and an emphasis on low latency, high throughput, and cost-efficiency; pandas experience a plu
• Leveraging SQL and python data tooling develop scalable, optimized ETL/ELT pipelines to ingest, transform, and load large, complex datasets from disparate sources (batch and streaming). Streamlining for low latency, high throughput and cost-efficiency.
• Experience with API development in both consumer and producer capacities (REST APIs)
• CI/CD experience with modern DevOps tooling and deployment pipelines
• Demonstrated ability to learn new technologies quickly and apply them to solve problems
• Strong attention to detail and commitment to producing high-quality work
• Excellent oral and written communication skills
• Professionalism and ethics including trust, positive attitude, commitment, honesty, collaboration, and approachability
• Proven ability to adapt in a fast-paced consulting environment, including managing time effectively and handling multiple tasks simultaneously

Nice To Haves

• Experience with version control systems (Git) and agile development frameworks
• Design and document end-to-end testing plans for Python-based applications
• PySpark experience for large-scale distributed data processing
• Experience with Datadog or similar observability and monitoring platforms
• Familiarity with Dagster or other modern data orchestration frameworks
• Terraform or infrastructure-as-code (IaC) experience for cloud resource provisioning
• Database experience with databases such as AWS RDS, MySQL, or PostgreSQL

Responsibilities

• Participate in team stand-ups and other agile ceremonies to support project delivery
• Assist in building and maintaining data acquisition pipelines from diverse sources including databases, flat-files, and REST APIs
• Leverage development design patterns and best practices across workloads.
• Support data modeling efforts including dimensional models and semantic layers for analytics under senior guidance
• Proactively troubleshoot and resolve pipeline failures, query performance issues, data quality concerns, notebook timeouts, data model consistencies, semantic model refresh errors, and capacity throttling using monitoring tools and query optimization techniques
• Participate in requirement gathering meetings and translate business needs into technical specifications
• Engage in ongoing learning through online trainings, certifications, and mentorship opportunities
• Create and maintain technical documentation, operational documentation, data flow diagrams, and data mapping documents
• Collaborate effectively with cross-functional teams and communicate progress to technical and non-technical stakeholders

Benefits

• Training Opportunities: We believe in lifelong learning and provide numerous avenues for skill enhancement.
• Fully Invested 401K Plan: We help secure your future with a fully invested 401K plan.
• PPO and HDHP Medical Plans: Choose the health insurance program that best fits your needs.
• Employer-Paid Dental and Vision Plans: We cover dental and vision plans, ensuring our employees have access to comprehensive health care.
• Onsite Fitness Center: Stay fit and healthy with our state-of-the-art fitness center.
• Wellness Program: We promote a healthy lifestyle with our wellness program.
• Catered Lunches: Enjoy delicious catered lunches regularly.

What This Job Offers

• Job Type: Full-time
• Career Level: Mid Level
• Education Level: No Education Listed