Azure Databricks Engineer
About The Position
Highly skilled Data Engineer with experience in designing, developing and managing end-to-end data pipelines on Azure Cloud Platform. Proficient in Azure Databricks & Unity Catalogs, Data Factory and ADLS, with a strong background in SQL, ETL process and Python/PySpark for large-scale data processing. Demonstrates solid understanding of Medallion Architecture for building scalable and maintainable data solutions. Experienced in Databricks workspace management and azure administration, including cluster configuration, access control. Strong collaborator who effectively translates business and customer requirements into technical solutions with minimal supervision.
Requirements
- Azure Services: Azure Data Factory (ADF), Azure Databricks, Azure Data Lake Storage
- Programming: Pyspark, SQL(Expert), Python
- Data Engineering: ELT/ETL pipeline design, ingestion framework, orchestration
- Architecture: Medallion Architecture, Delta Lake design
- Monitoring & Troubleshooting: Performance tuning, data quality validation, pipeline/jobs failure analysis
Nice To Haves
- Azure Administration:
- CI/CD & Devops: Integraton with Azure devOps for ADF pipeline development
- Data Governance & Secutiry: Knowledge of data lineage, RBAC
- Others: Good understanding of API, FTP/SFTP, Automation & Scripting
Benefits
- Discretionary Annual Incentive.
- Comprehensive Medical Coverage: Medical & Health, Dental & Vision, Disability Planning & Insurance, Pet Insurance Plans.
- Family Support: Maternal & Parental Leaves.
- Insurance Options: Auto & Home Insurance, Identity Theft Protection.
- Convenience & Professional Growth: Commuter Benefits & Certification & Training Reimbursement.
- Time Off: Vacation, Time Off, Sick Leave & Holidays.
- Legal & Financial Assistance: Legal Assistance, 401K Plan, Performance Bonus, College Fund, Student Loan Refinancing.
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Mid Level
Industry
Professional, Scientific, and Technical Services
Education Level
No Education Listed
Number of Employees
5,001-10,000 employees
