Data Engineer
About The Position
Design, build, and operate Azure Lakehouse architectures using Azure Databricks, Azure Data Lake Storage (ADLS Gen2), Azure Synapse Analytics, and Azure Data Factory to support analytical and operational workloads. Process large-scale structured and unstructured datasets using optimized batch and streaming pipelines leveraging Apache Spark, Delta Lake, Python, SQL, and Scala. Design, develop, and maintain scalable ETL/ELT pipelines using Databricks Workflows, Spark jobs, and Delta Lake, ensuring reliability, performance, and data quality at enterprise scale. Implement real-time and batch data processing solutions and optimize pipelines for production use.
Requirements
- US Citizenship required.
- Must be able to OBTAIN and MAINTAIN a Federal or DoD “PUBLIC TRUST” security clearance.
- Bachelor’s degree obtained in Computer Science, Engineering, Data Science, or equivalent experience.
- 5+ years of experience as a Data Engineer.
- 3+ years delivering enterprise solutions on Azure, specifically using Azure Databricks and Azure-native data services.
- Hands-on experience designing, building, and operating Azure Lakehouse architectures using Azure Databricks, ADLS Gen2, Azure Synapse Analytics, and Azure Data Factory.
- Deep expertise in Apache Spark, Delta Lake, Python, SQL, and Scala, with demonstrated ability to process large-scale structured and unstructured datasets using optimized batch and streaming pipelines.
- Proven experience designing, developing, and maintaining scalable ETL/ELT pipelines using Databricks Workflows, Spark jobs, and Delta Lake, ensuring reliability, performance, and data quality.
Nice To Haves
- Experience implementing real-time and batch streaming solutions.
- Master’s degree in computer science, Engineering, Data Science, or equivalent experience
- Experience with Snowflake-specific capabilities such as Snowpipe, Snowpark, Streams & Tasks, Snowflake performance tuning, or Snowflake certification.
Responsibilities
- Design, build, and operate Azure Lakehouse architectures using Azure Databricks, Azure Data Lake Storage (ADLS Gen2), Azure Synapse Analytics, and Azure Data Factory to support analytical and operational workloads.
- Process large-scale structured and unstructured datasets using optimized batch and streaming pipelines leveraging Apache Spark, Delta Lake, Python, SQL, and Scala.
- Design, develop, and maintain scalable ETL/ELT pipelines using Databricks Workflows, Spark jobs, and Delta Lake, ensuring reliability, performance, and data quality at enterprise scale.
- Implement real-time and batch data processing solutions and optimize pipelines for production use.
Benefits
- Medical, Rx, Dental & Vision Insurance
- Personal and Family Sick Time & Company Paid Holidays
- Position may be eligible for a discretionary variable incentive bonus
- Parental Leave and Adoption Assistance
- 401(k) Retirement Plan
- Basic Life & Supplemental Life
- Health Savings Account, Dental/Vision & Dependent Care Flexible Spending Accounts
- Short-Term & Long-Term Disability
- Student Loan PayDown
- Tuition Reimbursement, Personal Development & Learning Opportunities
- Skills Development & Certifications
- Employee Referral Program
- Corporate Sponsored Events & Community Outreach
- Emergency Back-Up Childcare Program
- Mobility Stipend
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Mid Level
Number of Employees
5,001-10,000 employees
