Senior Data Engineer - Encounter Processing & Data Platform
About The Position
The Senior Data Engineer will serve as the primary technical owner for the maintenance, enhancement, operational support, and reliability of an enterprise data processing framework supporting Medicaid Encounter Processing and enterprise data ingestion. This platform processes provider, member, reference, and encounter data for downstream Medicaid systems and reporting using Scala, Apache Spark, Hive, and Drools on the Cloudera Data Platform (CDP). The role involves designing, building, and continuously improving the client's online platform, researching and implementing new technology solutions, and taking responsibility for the resiliency and availability of different products. The engineer will be a productive member of the team, ensuring data processing reliability, performance, scalability, and operational stability, while supporting continuous improvement of data processing frameworks and production support processes.
Requirements
- 7+ years of experience developing enterprise-scale distributed data processing applications.
- Strong hands-on experience with Scala.
- Strong experience with Apache Spark 2.x and/or 3.x.
- Experience with Apache Hive 3.x, including data processing and Hive table management.
- Hands-on experience implementing business rules using the Drools Rules Engine.
- Strong SQL development and query optimization skills.
- Experience supporting and troubleshooting enterprise data processing applications in production environments.
- Experience with Medicaid or healthcare data, particularly Medicaid Encounter Processing.
- Experience with Cloudera Manager, HDFS, and YARN.
- Experience integrating with IBM DataStage.
- Familiarity with AWS infrastructure supporting Cloudera.
- Experience supporting enterprise-scale data platforms in production environments.
Responsibilities
- Own, maintain, and enhance the Scala/Spark application framework supporting Medicaid Encounter Processing.
- Develop, maintain, and enhance Drools business rules based on evolving business requirements.
- Manage enterprise data ingestion processes for master data and source-system files.
- Develop, maintain, and support Spark and Hive batch processing workflows.
- Investigate production issues, troubleshoot application defects, and implement effective resolutions.
- Perform operational administration, monitoring, and troubleshooting of the Cloudera Data Platform.
- Ensure data processing reliability, performance, scalability, and operational stability.
- Support continuous improvement of data processing frameworks and production support processes.
- Help design, build and continuously improve the client's online platform.
- Research, suggest and implement new technology solutions following best practices/standards.
- Take responsibility for the resiliency and availability of different products.
- Be a productive member of the team.
Benefits
- A challenging, innovating environment.
- Opportunities for learning where needed.
Stand Out From the Crowd
Upload your resume and get instant feedback on how well it matches this job.
What This Job Offers
Job Type
Full-time
Career Level
Senior
Education Level
No Education Listed
