Data Engineer with Python Spark Databricks and AIML

TekCommands • Dallas, TX

1d • Onsite

About The Position

We are seeking an experienced Data Engineer with expertise in Python, Spark, Databricks, and AI/ML technologies to build scalable data platforms and Generative AI solutions. The ideal candidate should have experience developing data pipelines, LLM applications, AI agent workflows, and cloud-based data engineering solutions.

Requirements

• 7–12 years of Data Engineering experience.
• Strong expertise in Python, PySpark, and Apache Spark.
• Hands-on experience with Databricks and Delta Lake.
• Experience with Snowflake and PostgreSQL.
• Experience building LLM/GenAI applications and pipelines.
• Strong knowledge of LangChain, LangGraph, AI Agents, and RAG (Retrieval-Augmented Generation).
• Experience with OpenAI, Azure OpenAI, or Hugging Face APIs.
• Good understanding of Azure Cloud architecture and scalable data solutions.
• Strong analytical, problem-solving, and communication skills.

Nice To Haves

• Telecom domain experience.
• Experience with Model Context Protocol (MCP).
• AI Agent orchestration.
• Docker.
• CI/CD pipelines.
• Azure or GCP cloud platforms.

Responsibilities

• Design and build scalable data pipelines using Python, Spark, Databricks, and Delta Lake.
• Develop and manage data solutions on Azure, Snowflake, and PostgreSQL.
• Build and optimize LLM/GenAI pipelines, including data ingestion, prompt engineering, fine-tuning, and deployment.
• Develop AI workflows using LangChain, LangGraph, AI Agents, and RAG architectures.
• Integrate with OpenAI, Azure OpenAI, Hugging Face, and other LLM APIs.
• Collaborate with data scientists, AI engineers, and business teams.
• Ensure data quality, governance, security, and performance across enterprise data platforms.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: No Education Listed