Engineer

TATA Consulting Services • Dallas, TX

34d

About The Position

We are looking for a highly skilled and passionate Databricks Machine Learning Engineer to build scalable ML pipelines, automating training and deployment, managing model lifecycle with MLflow. Responsibilities: Collaborate closely with data scientists, data engineers, and business stakeholders to gather requirements and understand the business objectives driving model development. Develop various methods for implementing feature engineering techniques Preprocess and transform large datasets, selecting appropriate algorithm Leveraging Databricks MLflow for experiment tracking and model management Build robust pipelines that automate the training, validation, and deployment of models. Monitor Model performance and collaborating closely with data scientists and stakeholders to finetune models Migrate on Prem Pyspark, SAS data pipeline and ML Models to Databricks Define and implement ML best practices in Databricks Evaluate new Databricks ML features and tools, helping the organization stay at the forefront of innovation in AI and Big Data Explore AI and Gen AI solutions for business requirements and implement various solutions.

Requirements

• Proven expertise in implementing various ML Models using Databricks.
• Strong PySpark and Python experience
• Familiarity with ML Ops/LLM Ops and distributed systems.
• Experience with Big Data platform like Cloudera Hadoop and Could platforms like AWS, GCP.
• Solid understanding of system design patterns, scalability, observability, and performance tuning.
• Strong analytical and problem-solving skills.
• Passion for exploring and building with emerging technologies.

Responsibilities

• Collaborate closely with data scientists, data engineers, and business stakeholders to gather requirements and understand the business objectives driving model development.
• Develop various methods for implementing feature engineering techniques Preprocess and transform large datasets, selecting appropriate algorithm
• Leveraging Databricks MLflow for experiment tracking and model management
• Build robust pipelines that automate the training, validation, and deployment of models.
• Monitor Model performance and collaborating closely with data scientists and stakeholders to finetune models
• Migrate on Prem Pyspark, SAS data pipeline and ML Models to Databricks
• Define and implement ML best practices in Databricks
• Evaluate new Databricks ML features and tools, helping the organization stay at the forefront of innovation in AI and Big Data
• Explore AI and Gen AI solutions for business requirements and implement various solutions.

What This Job Offers

• Job Type: Full-time
• Career Level: Mid Level
• Industry: Professional, Scientific, and Technical Services
• Education Level: No Education Listed
• Number of Employees: 5,001-10,000 employees