Senior Engineer, Big Data
About The Position
We are seeking a highly skilled and experienced Sr Big Data Engineer to drive enterprise architecture strategy, platform engineering, modernization initiatives, and end-to-end solution delivery. This role will lead data and analytics transformation efforts leveraging Databricks, cloud-native technologies, and modern data platform architectures. The ideal candidate will partner closely with Enterprise Information Management (EIM), business stakeholders, and technology leaders to design and deliver scalable, secure, and high-performing data solutions that enable advanced analytics, AI/ML, and enterprise reporting capabilities. This is a hands-on technical leadership role, with approximately 40% architecture and strategy, 30% engineering and platform implementation, and 30% governance, mentoring, and stakeholder collaboration.
Requirements
- Bachelor's degree in Computer Science, Information Systems, Engineering, Data Science, or a related field.
- 5-7 years of experience in Solution Architecture, Data Architecture, or Enterprise Architecture roles.
- 5+ years of experience designing and delivering enterprise-scale data management and analytics solutions.
- 3+ years of experience in Platform Engineering and cloud-native data platforms.
- Experience implementing medallion architectures and enterprise data models across ingestion, curation, and consumption layers.
- Strong experience architecting and implementing Data Governance, Data Quality, Metadata Management, and Data Security solutions.
- Hands-on experience with Databricks and Lakehouse architecture.
- Experience delivering cloud-based solutions using Azure, AWS, or Google Cloud Platform.
- Proven experience supporting AI/ML initiatives and modern data analytics ecosystems.
- Advanced hands-on experience with Python, PySpark, SQL, Apache Spark, Delta Lake, and distributed data-processing frameworks.
- Experience with CI/CD, Infrastructure as Code, automated testing, and production deployment of Databricks workloads.
Nice To Haves
- Experience with healthcare data standards such as FHIR, HL7, X12, HEDIS, or claims and clinical data is preferred.
Responsibilities
- Define and establish enterprise architecture patterns, standards, frameworks, and best practices for scalable data and analytics solutions.
- Design, architect, and implement modern data platforms with a focus on performance, scalability, security, reliability, and maintainability.
- Lead the adoption and enablement of Databricks Lakehouse capabilities, including Data Engineering, Data Governance, Analytics, and AI/ML workloads.
- Evaluate emerging technologies, tools, and frameworks to support enterprise business and technology objectives.
- Collaborate with business leaders, executives, data teams, and engineering organizations to translate strategic goals into technology solutions.
- Provide hands-on architectural leadership throughout the solution lifecycle, from strategy and design through implementation and operationalization.
- Conduct architecture and design reviews to ensure alignment with enterprise standards and best practices.
- Develop automation frameworks and engineering solutions to improve Databricks platform reliability, operational efficiency, and developer productivity.
- Partner with data ingestion, curation, and consumption teams to optimize data processing, platform performance, and cloud costs.
- Drive data governance, security, compliance, and data quality initiatives across the enterprise data ecosystem.
- Mentor engineering teams and promote cloud-native, platform engineering, and modern data architecture best practices.
Benefits
- competitive benefits and compensation package
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Senior
