Data Engineer III

Robert Half • San Ramon, CA

4d • $104,000 - $153,000

About The Position

Robert Half, one of FORTUNE’s World’s Most Admired Companies and a Fortune 100 Best Companies to Work For is hiring for a Data Engineer III to join the ATI Data Science Innovation department.

Requirements

• Bachelor's in Computer Science, Engineering, or related field (Master's preferred)
• 5+ years with Python and SQL in data engineering for big data ML/analytics workloads
• 5+ years designing, building, and troubleshooting scalable ETL/ELT pipelines for business-critical production systems
• 3+ years with cloud data services (AWS), container orchestration (Docker, Kubernetes), and IaC (Terraform, CloudFormation)
• 3+ years architecting ML workflows and data platforms with CI/CD, automated testing, and distributed processing (Spark)
• 3+ years collaborating cross-functionally with Data Science, MLOps, Platform Engineering, and DevOps teams
• 3+ years implementing data quality testing and optimizing SQL/Python for cost/performance in the cloud
• Understanding of the full Data Science SDLC, and experience mentoring engineers

Nice To Haves

• 2+ years hands-on with Databricks (Delta Lake, Unity Catalog, Databricks SQL)
• Experience with MLflow experiment tracking and model registry workflows
• Experience designing pipelines that serve AI/ML inference — real-time feature engineering, embedding generation, and context retrieval for LLM-based systems
• Understanding of how data engineering supports Agentic AI: agent-accessible data services, low-latency retrieval, and pipelines enabling autonomous multi-step workflows
• Familiarity with Databricks Mosaic AI, Vector Search, and/or Feature Store
• FinOps awareness — compute cluster optimization, cost attribution by workload
• Familiarity with Salesforce/Heroku data infrastructures
• Experience with data virtualization (e.g., Dremio)
• Understanding of Platform Engineering concepts and internal developer platforms
• Experience migrating from legacy data warehouse/lake to unified lakehouse architecture
• Familiarity with Odaseva data security and management

Responsibilities

• Lead architecture and design of complex data pipelines on Databricks lakehouse architecture (Unity Catalog, Delta Lake, Structured Streaming)
• Define technical approach for data engineering initiatives, mentor less-senior engineers, and set standards for code quality through leadership and code reviews
• Design and build data foundations that enable AI/ML capabilities — feature stores, embedding pipelines, vector search indexes, and model training datasets
• Align data engineering solutions with business strategy, including support for Agentic AI workloads
• Own health, scalability, and modernization of data infrastructure with Databricks as the strategic platform — including workload migration, compute optimization, and Unity Catalog adoption
• Optimize pipeline performance (Delta Lake table layouts, clustering, Z-ordering) and establish monitoring/alerting best practices with clear SLAs
• Build data infrastructure supporting Agentic AI systems — real-time data access layers, context retrieval pipelines, and agent-accessible data services
• Collaborate cross-functionally with DevOps, Platform Engineering, and MLOps roles to integrate data solutions into the broader technology environment and shared AI infratstructure – Mlflow registries, feature stores, and agent orchestration layers
• Provide consultation to Senior Leadership on complex projects and drive continuous improvement initiatives
• Champion data governance at all layers for data, models, and AI assets
• Implement data quality strategies (master data management, validation rules, Delta Live Tables expectations) to ensure trust in enterprise data
• Serve as liaison across data engineering, AI engineering, and business teams; promote data literacy and stewardship

Benefits

• group health insurance benefits (medical, vision, dental)
• FSA and HSA healthcare accounts
• life and accident insurance
• adoption and fertility assistance
• paid parental leave of up to 6 weeks
• short/long term disability
• paid time off for vacation, personal needs, and sick time
• up to 11 paid holidays per calendar year
• 401(k) savings and investment plan or deferred compensation plan (if eligible), with an employer match of 100% on the first 3% of your contributions for eligible employees

What This Job Offers

• Job Type: Full-time
• Career Level: Senior