About The Position

As a Machine Learning Engineer, you will leverage your expertise in PySpark and MLlib to design and optimize scalable machine learning workflows on Databricks. You’ll work closely with data scientists and product teams to build robust ML pipelines that support advanced analytics and generative AI applications.

Requirements

  • 5+ years' of relevant experience
  • Strong proficiency in PySpark and distributed data processing
  • Experience with Databricks and MLlib
  • Familiarity with deep learning frameworks like PyTorch
  • Solid understanding of cloud-based ML deployment (AWS preferred)
  • API development experience with FastAPI

Responsibilities

  • Apply advanced ML techniques including supervised, unsupervised, and deep learning using PySpark, MLlib, and PyTorch
  • Design and monitor end-to-end ML workflows in Databricks, including feature engineering, model training, and inference
  • Collaborate with cross-functional teams to translate business requirements into scalable ML solutions
  • Integrate Generative AI models (e.g., GPT, Claude) for tasks such as summarization, text generation, and conversational AI
  • Develop APIs for model serving using FastAPI, supporting synchronous, asynchronous, and streaming inference
  • Utilize AWS services (S3, Lambda, SageMaker, EC2) for model deployment and orchestration
  • Implement CI/CD pipelines for ML models using GitHub Actions or Jenkins
© 2024 Teal Labs, Inc
Privacy PolicyTerms of Service