Databricks Engineer

TATA Consulting Services•Seattle, WA

32d

About The Position

5+ years of experience in data engineering or similar roles. Strong expertise in Databricks, Apache Spark, and PySpark. Proficiency in SQL, Python, and data modeling concepts. Experience with cloud platforms (Azure preferred; AWS/GCP is a plus). Knowledge of Delta Lake, Lakehouse architecture, and partitioning strategies. Familiarity with data governance, security best practices, and performance tuning. Hands-on experience with version control (Git) and CI/CD pipelines.

Requirements

5+ years of experience in data engineering or similar roles.
Strong expertise in Databricks, Apache Spark, and PySpark.
Proficiency in SQL, Python, and data modeling concepts.
Experience with cloud platforms (Azure preferred; AWS/GCP is a plus).
Knowledge of Delta Lake, Lakehouse architecture, and partitioning strategies.
Familiarity with data governance, security best practices, and performance tuning.
Hands-on experience with version control (Git) and CI/CD pipelines.

Nice To Haves

Experience with cloud platforms (Azure preferred; AWS/GCP is a plus).

Responsibilities

Design and develop ETL/ELT pipelines using Azure Databricks and Apache Spark.
Integrate data from multiple sources into the data lake and data warehouse environments.
Optimize data workflows for performance and cost efficiency in cloud environments (Azure/AWS/GCP).
Implement data quality checks, monitoring, and alerting for pipelines.
Collaborate with data scientists and analysts to provide clean, curated datasets.
Ensure compliance with data governance, security, and privacy standards.
Automate workflows using CI/CD pipelines and orchestration tools (e.g., Airflow, Azure Data Factory).
Troubleshoot and resolve issues in data pipelines and platform components.

Benefits

Discretionary Annual Incentive.
Comprehensive Medical Coverage: Medical & Health, Dental & Vision, Disability Planning & Insurance, Pet Insurance Plans.
Family Support: Maternal & Parental Leaves.
Insurance Options: Auto & Home Insurance, Identity Theft Protection.
Convenience & Professional Growth: Commuter Benefits & Certification & Training Reimbursement.
Time Off: Vacation, Time Off, Sick Leave & Holidays.
Legal & Financial Assistance: Legal Assistance , 401K Plan, Performance Bonus, College Fund, Student Loan Refinancing.

Stand Out From the Crowd

Upload your resume and get instant feedback on how well it matches this job.

Upload and Match Resume