Senior AI & Data Engineer

Intapp • Palo Alto, CA

1d

About The Position

We are seeking a Senior Data Engineer to design, build, and optimize scalable data infrastructure that powers business intelligence, analytics, and AI-driven applications. The ideal candidate brings great communication skills, deep expertise in Python, containerization, cloud-native architectures, and modern data warehousing, with a growing interest in AI agents and emerging integration patterns. What you will do: Data Infrastructure & Pipeline Development Design and implement robust, scalable ETL/ELT pipelines to ingest, transform, and deliver data across the organization Build and maintain data models optimized for analytical workloads and downstream consumption Develop reusable frameworks and libraries to accelerate data engineering workflows Cloud & Platform Engineering Architect and manage cloud-native data solutions on AWS, leveraging services such as S3, EC2, ECS, ECR and EventBridge Deploy and orchestrate containerized workloads using Docker and Kubernetes in production environments Implement infrastructure-as-code practices using Terraform, CloudFormation, or similar tools Data Warehousing Design and optimize data warehouse architectures using Redshift or Snowflake Develop efficient data models, partitioning strategies, and query optimization techniques Manage data lifecycle, governance, and cost optimization within warehouse environments Emerging Technologies Explore and integrate AI agents and Model Context Protocol (MCP) patterns into data workflows where applicable Collaborate with AI/ML teams to ensure data infrastructure supports model training, inference, and feature engineering needs Stay current with emerging data and AI technologies, evaluating their potential business impact Collaboration & Leadership Partner with analytics, product, and engineering teams to understand data requirements and deliver solutions Mentor junior engineers and contribute to team best practices, code reviews, and technical documentation Participate in architectural decisions and contribute to the data platform roadmap What you will need: Experience 7+ years of professional experience in data engineering or related roles Proven track record building and operating production data pipelines at scale Technical Skills Python: Advanced proficiency including data libraries (pandas, PySpark, SQLAlchemy), testing frameworks, and packaging Containerization & Orchestration: Strong hands-on experience with Docker and Kubernetes (EKS, GKE, or self-managed clusters) Data Warehousing: Deep expertise with Redshift or Snowflake, including performance tuning, data modeling, and administration including DBT AWS: Extensive experience across the AWS ecosystem, particularly data-related services (S3, Glue, Lambda, IAM, VPC, CloudWatch) SQL: Expert-level SQL skills for complex analytical queries and data transformations Foundational Skills Experience with workflow orchestration tools (Airflow, Dagster, etc.) Familiarity with version control (Git), CI/CD pipelines, and agile development practices Strong understanding of data governance, security, and compliance principles Preferred Qualifications: Experience with or exposure to AI agents, LLM integrations, or Model Context Protocol (MCP) implementations Background in real-time streaming architectures (Kafka, Kinesis, Flink) Experience with dbt for transformation layer management Familiarity with data observability and quality tools (Monte Carlo, Great Expectations, dbt tests) Knowledge of additional programming languages (Scala, Java, Go) Experience in professional services, legal tech, or enterprise SaaS environments Education Bachelor's degree in Computer Science, Engineering, Mathematics, or related field; or equivalent practical experience Relevant certifications a plus (AWS Data Analytics, Snowflake SnowPro, Kubernetes CKA/CKAD) What you will gain: At Intapp, you’ll get the opportunity to bring your talents and intellectual curiosity to create better outcomes for our people and our clients. You’ll be part of a growing public company, with a modern work environment that’s connected yet flexible and where your professional growth and well-being are top priorities. We’ll collaborate and grow together, supporting each other in a positive, open atmosphere that fosters creativity, innovation, and teamwork. Here, you will have the opportunity to: Expand Your Skills: Unlock your potential with professional development opportunities supported by a community of experienced professionals. We offer reimbursement for training and continuing education to help you stay ahead in your career. Enjoy Where You Work: Thrive in our modern, open offices designed to inspire creativity and collaboration. Our complimentary lunches and fully stocked kitchens ensure you have everything you need to stay energized throughout the day. Support What Matters Most: Our comprehensive wellness and flexible time off programs and our benefits are designed to care for you and your family. Our family-formation benefits and support during your family-building journey ensure you have the resources you need when it matters most. We believe in giving back and supporting our communities with paid volunteer time off and a donation matching program for the causes you care about. Join us and be a part of a collaborative and welcoming culture where your contributions are valued, and your professional growth is a priority. Together, we are building a company of long-term value that we can all be proud of.

Requirements

• 7+ years of professional experience in data engineering or related roles
• Proven track record building and operating production data pipelines at scale
• Python: Advanced proficiency including data libraries (pandas, PySpark, SQLAlchemy), testing frameworks, and packaging
• Containerization & Orchestration: Strong hands-on experience with Docker and Kubernetes (EKS, GKE, or self-managed clusters)
• Data Warehousing: Deep expertise with Redshift or Snowflake, including performance tuning, data modeling, and administration including DBT
• AWS: Extensive experience across the AWS ecosystem, particularly data-related services (S3, Glue, Lambda, IAM, VPC, CloudWatch)
• SQL: Expert-level SQL skills for complex analytical queries and data transformations
• Experience with workflow orchestration tools (Airflow, Dagster, etc.)
• Familiarity with version control (Git), CI/CD pipelines, and agile development practices
• Strong understanding of data governance, security, and compliance principles

Nice To Haves

• Experience with or exposure to AI agents, LLM integrations, or Model Context Protocol (MCP) implementations
• Background in real-time streaming architectures (Kafka, Kinesis, Flink)
• Experience with dbt for transformation layer management
• Familiarity with data observability and quality tools (Monte Carlo, Great Expectations, dbt tests)
• Knowledge of additional programming languages (Scala, Java, Go)
• Experience in professional services, legal tech, or enterprise SaaS environments
• Relevant certifications a plus (AWS Data Analytics, Snowflake SnowPro, Kubernetes CKA/CKAD)

Responsibilities

• Design and implement robust, scalable ETL/ELT pipelines to ingest, transform, and deliver data across the organization
• Build and maintain data models optimized for analytical workloads and downstream consumption
• Develop reusable frameworks and libraries to accelerate data engineering workflows
• Architect and manage cloud-native data solutions on AWS, leveraging services such as S3, EC2, ECS, ECR and EventBridge
• Deploy and orchestrate containerized workloads using Docker and Kubernetes in production environments
• Implement infrastructure-as-code practices using Terraform, CloudFormation, or similar tools
• Design and optimize data warehouse architectures using Redshift or Snowflake
• Develop efficient data models, partitioning strategies, and query optimization techniques
• Manage data lifecycle, governance, and cost optimization within warehouse environments
• Explore and integrate AI agents and Model Context Protocol (MCP) patterns into data workflows where applicable
• Collaborate with AI/ML teams to ensure data infrastructure supports model training, inference, and feature engineering needs
• Stay current with emerging data and AI technologies, evaluating their potential business impact
• Partner with analytics, product, and engineering teams to understand data requirements and deliver solutions
• Mentor junior engineers and contribute to team best practices, code reviews, and technical documentation
• Participate in architectural decisions and contribute to the data platform roadmap

Benefits

• Expand Your Skills: Unlock your potential with professional development opportunities supported by a community of experienced professionals. We offer reimbursement for training and continuing education to help you stay ahead in your career.
• Enjoy Where You Work: Thrive in our modern, open offices designed to inspire creativity and collaboration. Our complimentary lunches and fully stocked kitchens ensure you have everything you need to stay energized throughout the day.
• Support What Matters Most: Our comprehensive wellness and flexible time off programs and our benefits are designed to care for you and your family. Our family-formation benefits and support during your family-building journey ensure you have the resources you need when it matters most.
• We believe in giving back and supporting our communities with paid volunteer time off and a donation matching program for the causes you care about.

What This Job Offers

• Job Type: Full-time
• Career Level: Mid Level
• Number of Employees: 1,001-5,000 employees