Senior Data Engineer

Intapp • Palo Alto, CA

13d

About The Position

We are seeking a Senior Data Engineer to design, build, and optimize scalable data infrastructure that powers business intelligence, analytics, and AI-driven applications. The ideal candidate brings great communication skills, deep expertise in Python, containerization, cloud-native architectures, and modern data warehousing, with a growing interest in AI agents and emerging integration patterns.

Requirements

• 7+ years of professional experience in data engineering or related roles
• Proven track record building and operating production data pipelines at scale
• Python: Advanced proficiency including data libraries (pandas, PySpark, SQLAlchemy), testing frameworks, and packaging
• Containerization & Orchestration: Strong hands-on experience with Docker and Kubernetes (EKS, GKE, or self-managed clusters)
• Data Warehousing: Deep expertise with Redshift or Snowflake, including performance tuning, data modeling, and administration including DBT
• AWS: Extensive experience across the AWS ecosystem, particularly data-related services (S3, Glue, Lambda, IAM, VPC, CloudWatch)
• SQL: Expert-level SQL skills for complex analytical queries and data transformations
• Experience with workflow orchestration tools (Airflow, Dagster, etc.)
• Familiarity with version control (Git), CI/CD pipelines, and agile development practices
• Strong understanding of data governance, security, and compliance principles

Nice To Haves

• Experience with or exposure to AI agents, LLM integrations, or Model Context Protocol (MCP) implementations
• Background in real-time streaming architectures (Kafka, Kinesis, Flink)
• Experience with dbt for transformation layer management
• Familiarity with data observability and quality tools (Monte Carlo, Great Expectations, dbt tests)
• Knowledge of additional programming languages (Scala, Java, Go)
• Experience in professional services, legal tech, or enterprise SaaS environments
• Relevant certifications a plus (AWS Data Analytics, Snowflake SnowPro, Kubernetes CKA/CKAD)

Responsibilities

• Data Infrastructure & Pipeline Development Design and implement robust, scalable ETL/ELT pipelines to ingest, transform, and deliver data across the organization
• Build and maintain data models optimized for analytical workloads and downstream consumption
• Develop reusable frameworks and libraries to accelerate data engineering workflows
• Cloud & Platform Engineering Architect and manage cloud-native data solutions on AWS, leveraging services such as S3, EC2, ECS, ECR and EventBridge
• Deploy and orchestrate containerized workloads using Docker and Kubernetes in production environments
• Implement infrastructure-as-code practices using Terraform, CloudFormation, or similar tools
• Data Warehousing Design and optimize data warehouse architectures using Redshift or Snowflake
• Develop efficient data models, partitioning strategies, and query optimization techniques
• Manage data lifecycle, governance, and cost optimization within warehouse environments
• Emerging Technologies Explore and integrate AI agents and Model Context Protocol (MCP) patterns into data workflows where applicable
• Collaborate with AI/ML teams to ensure data infrastructure supports model training, inference, and feature engineering needs
• Stay current with emerging data and AI technologies, evaluating their potential business impact
• Collaboration & Leadership Partner with analytics, product, and engineering teams to understand data requirements and deliver solutions
• Mentor junior engineers and contribute to team best practices, code reviews, and technical documentation
• Participate in architectural decisions and contribute to the data platform roadmap

Benefits

• Expand Your Skills: Unlock your potential with professional development opportunities supported by a community of experienced professionals. We offer reimbursement for training and continuing education to help you stay ahead in your career.
• Enjoy Where You Work: Thrive in our modern, open offices designed to inspire creativity and collaboration. Our complimentary lunches and fully stocked kitchens ensure you have everything you need to stay energized throughout the day.
• Support What Matters Most: Our comprehensive wellness and flexible time off programs and our benefits are designed to care for you and your family. Our family-formation benefits and support during your family-building journey ensure you have the resources you need when it matters most. We believe in giving back and supporting our communities with paid volunteer time off and a donation matching program for the causes you care about.

What This Job Offers

• Job Type: Full-time
• Career Level: Mid Level
• Number of Employees: 1,001-5,000 employees