Databricks Platform Engineer

Bright Vision Technologies•Bridgewater Township, NJ

1d•Remote

About The Position

Bright Vision Technologies is a forward-thinking software development company dedicated to building innovative solutions that help businesses automate and optimize their operations. We leverage cutting-edge cloud data platform technologies to design scalable, secure, and high-performance analytics environments. As we continue to grow, we’re looking for a skilled Databricks Platform Engineer to join our dynamic team and contribute to our mission of transforming business processes through technology. This is a fantastic opportunity to join an established and well-respected organization offering tremendous career growth potential.

Responsibilities

Architect and implement scalable data solutions on the Databricks Lakehouse Platform, utilizing Apache Spark and PySpark for distributed processing at scale.
Design and execute ETL/ELT pipelines with SQL and Delta Lake, ensuring ACID transactions, schema evolution, and reliable data ingestion from sources like AWS S3, Azure ADLS, and Google Cloud Storage (GCS).
Build and orchestrate Databricks Workflows for automated job scheduling, dependency management, and end-to-end data pipeline execution.
Develop advanced Data Modeling strategies, including dimensional modeling and unified analytics layers, optimized for Databricks' lakehouse architecture.
Drive Performance Optimization through Spark tuning, caching strategies, and Delta Lake optimizations to handle petabyte-scale workloads efficiently.
Integrate with Cloud Platforms (AWS, Azure, GCP) for seamless hybrid deployments, leveraging native storage and compute resources.
Implement CI/CD pipelines with Git for version-controlled Databricks notebooks, clusters, and workflows, enabling rapid iteration and deployment.
Enforce Security & Access Controls using Unity Catalog, row/column-level security, and fine-grained permissions to protect sensitive data assets.
Collaborate in Agile methodologies, contributing to sprint planning, code reviews, and iterative delivery of data engineering features.
Monitor, troubleshoot, and scale Databricks environments, delivering cost-effective, high-performance data platforms that power business intelligence and ML use cases.