Principal Data Engineer

Autodesk•San Francisco, CA

20h

About The Position

We are seeking a Principal Data Engineer to provide technical leadership in designing, building, and scaling data infrastructure that powers machine learning, personalization, and search experiences. You will architect scalable data pipelines in production, drive technical strategy, and mentor engineering teams while partnering with Machine Learning Engineering, Platform Engineering, and Data Science. Your work will be critical to strategic initiatives including optimization of digital conversion metrics, development of Autodesk Assistant (an LLM-driven chatbot), RAG (Retrieval-Augmented Generation) systems, eCommerce personalization engines, and intelligent search capabilities. As a Principal Engineer, you will set technical direction, establish best practices, and drive innovation across the data engineering organization. Our team culture is built on collaboration, mutual support, and continuous learning. We emphasize an agile, hands-on, and technical approach at all levels of the team.

Requirements

8+ years of data engineering experience with at least 3+ years in a senior or lead capacity.
Expert-level proficiency in Python, Java, or Scala
Deep knowledge of SQL and extensive experience with relational and NoSQL databases
Strong expertise in big data technologies (Kafka, Flink, Spark, Parquet, Iceberg, Delta Lake)
Advanced experience with ETL orchestration tools like Apache Airflow
Extensive experience with cloud platforms (AWS, Azure, or GCP) and their data services
Deep understanding of data warehousing solutions like Snowflake, Redshift, or BigQuery
Proven expertise in data modeling, data architecture design, and ETL/ELT processes at scale
Demonstrated ability to lead technical initiatives and mentor engineering talent
Strong communication skills with ability to explain complex technical concepts to diverse audiences.
Bachelor’s degree in computer science, or related field (Master's/PhD strongly preferred)

Nice To Haves

Experience building data pipelines for Retrieval-Augmented Generation, vector databases (Pinecone, Weaviate, Milvus), and semantic search
Experience with real-time personalization engines, recommendation systems, feature stores, and A/B testing infrastructure
Experience building ML data pipelines, feature engineering systems, and model serving infrastructure
Knowledge of search technologies (Elasticsearch, Solr, OpenSearch), relevance tuning, and search analytics
Deep experience with event streaming, behavioral analytics, and customer data platforms
Experience with A/B testing infrastructure and metrics computation at scale
Knowledge of MLOps, model monitoring, and ML pipeline orchestration
Knowledge of eCommerce metrics, conversion optimization, and digital analytics

Responsibilities

Define and drive technical architecture for data platforms supporting ML and personalization at scale.
Design, build, and maintain highly scalable, low-latency data pipelines supporting real-time ML inference and personalization.
Build data pipelines for RAG systems, including vector embeddings and semantic search.
Design real-time feature engineering and event processing systems for eCommerce personalization and recommendation engines.
Develop sophisticated data models optimized for ML training, real-time inference, and analytics workloads.
Implement complex stream processing architectures using technologies like Kafka and Flink.
Oversee and optimize database systems (SQL, NoSQL, vector databases) ensuring high performance and scalability.
Establish data engineering standards, patterns, and best practices across the organization.
Provide technical leadership, guidance, and mentorship to senior and mid-level data engineers.
Lead cross-functional teams to deliver complex data engineering solutions in production.

Benefits

From health and financial benefits to time away and everyday wellness, we give Autodeskers the best, so they can do their best work.
Learn more about our benefits in the U.S. by visiting https://benefits.autodesk.com/
Salary is one part of Autodesk’s competitive compensation package.
For U.S.-based roles, we expect a starting base salary between $130,600 and $211,200.
Offers are based on the candidate’s experience and geographic location, and may exceed this range.
In addition to base salaries, our compensation package may include annual cash bonuses, commissions for sales roles, stock grants, and a comprehensive benefits package.
At Autodesk, we're building a diverse workplace and an inclusive culture to give more people the chance to imagine, design, and make a better world.
Autodesk is proud to be an equal opportunity employer and considers all qualified applicants for employment without regard to race, color, religion, age, sex, sexual orientation, gender, gender identity, national origin, disability, veteran status or any other legally protected characteristic.
We also consider for employment all qualified applicants regardless of criminal histories, consistent with applicable law.
We take pride in cultivating a culture of belonging where everyone can thrive.
Learn more here: https://www.autodesk.com/company/diversity-and-belonging