Senior Data Engineer
About The Position
Being a Senior Data Engineer at iManage Means… You get excited about data, and you believe in the democratization of data to support data-driven decision making and AI. iManage runs on data. iMData, our Enterprise Data Platform, is the engine that powers business decisions across the company. We're looking for a Senior Data Engineer to join this team and own the platform and pipelines that keep that engine running. As a Senior Data Engineer on the Enterprise Data team, you own the technical side of the iMData platform. You are excited about the big picture of how our data can work together to deliver incredible insights, not just about point-in-time data ingestion. You are passionate about mentoring junior team members and helping them grow. You're comfortable engaging directly with business stakeholders, gathering requirements firsthand, and crafting solutions that deliver critical data to best meet their requirements. You're a self-starter who can independently take an ambiguous problem, break it down, and drive it to completion. You'll partner with your manager and Machine Learning Engineers to architect, design and build our Databricks data platform and our Databricks MCP connector.
Requirements
- A Bachelor’s degree or higher in Computer Science or equivalent field.
- 5–7 years of experience in data engineering, with a strong track record building and operating production-grade pipelines.
- Hands-on production experience with Databricks, Delta Lake, and medallion architecture.
- Familiarity with Terraform for infrastructure as code.
- An understanding of CI/CD practices, specifically using GitHub Actions and Databricks Asset Bundles (DABs) for pipeline deployment and automation.
- A deep expertise in PySpark. You know how to tune, debug, and scale distributed workloads at scale.
- Strong data modeling skills, including building star schemas in gold layers to support AI, analytical, and reporting workloads.
- A strong engineering mindset. You enjoy independently working through hard problems and are equally comfortable walking the team through how you got there.
- Comfort working across organizational boundaries with data analysts, data scientists, and business stakeholders.
- A passion to understand data requirements and deliver scalable, robust solutions that meet those requirements.
- A passion to be a thought leader and work collaboratively within a team.
- A creative mindset with a desire to explore new technologies and create innovative solutions.
Nice To Haves
- Experience with one or more tools for real-time data streaming, such as Microsoft Event Hub or Apache Kafka.
- Experience with platform migrations, particularly Synapse to Databricks or similar Lakehouse transitions.
- Exposure to data governance, lineage, or observability tooling such as Informatica and Collibra.
Responsibilities
- Using PySpark to extract, transform, and load data into our Enterprise Data Platform using a medallion architecture.
- Designing gold layer star schemas to support AI, analytical, and reporting workloads.
- Serving as a technical lead and mentor for junior and mid-level data engineers on the team.
- Defining and advocating for engineering best practices: code standards, testing, documentation, and pipeline design patterns.
- Learning and using system-specific APIs to extract data from a wide-variety enterprise SaaS applications, including Salesforce and Zendesk.
- One time migration of pipelines, Spark notebooks, triggers, data models, and Power BI dashboards from Microsoft Synapse to Azure Databricks.
- Gathering data requirements from various business stakeholders and translating these requirements into deliverables which provide optimum value to the business.
- Working in an agile fashion aligned with our team Agile Methodologies.
- Applying best practices to ensure the security and privacy of the data repositories.
- Developing CI/CD workflows to enhance our SDLC.
- Maintaining and improving our Infrastructure as Code using Terraform.
Benefits
- Flexible working policy
- Annual performance-based bonus
- Comprehensive Health/Vision/Dental/Life Insurance
- 401k Retirement Savings Plan with a company match up to 4%
- Enhanced leave for expecting parents (20 weeks 100% paid for primary leave, and 10 weeks 100% paid for secondary leave)
- Flexible time off policy
- Multiple company wellness days each year
- Access to RethinkCare, a global behavioral health platform
- Unlimited access to LinkedIn Learning courses
- Interactive Microsoft courses & training
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Senior
