Arine-posted 4 months ago
$165,000 - $180,000/Yr
Full-time • Senior
Remote • San Francisco, CA
101-250 employees
Hospitals

As a key technical leader and team architect working in a fast-paced environment, you will drive the design, development, and optimization of scalable data ingestion pipelines within the Arine platform. Leveraging expert-level proficiency in Python and AWS, you will architect solutions that handle diverse file types and large-scale healthcare datasets. You will have a direct impact on building reusable, configurable tools set for handling data needs for the entire company.

  • Act as the team architect by leading system design reviews, offering recommendations, conducting comprehensive peer reviews, and demonstrating expert-level proficiency in Python and AWS services
  • Architecting and implementing scalable data ingestion pipelines handling different file types into Arine platform
  • Develop reusable components that can be integrated into data pipelines to enhance efficiency and minimize future implementation time
  • Creating configuration-driven, containerized toolsets that can be easily used and maintained by diverse engineering profiles
  • Work collaboratively with cross-functional teams to ensure their data requirements are met through ETL components
  • Implementing incremental data ingestion strategies for large-scale healthcare datasets
  • Building monitoring and alerting systems for data ingestion processes and pipeline health
  • Applying software engineering best practices including test-driven development and modular design to data infrastructure
  • Refactoring and rebuilding existing data ingestion processes to improve scalability and operational efficiency
  • Working with containerization technologies (Docker, Kubernetes) to create portable and maintainable data solutions
  • Identify and escalate inefficiencies within and across teams
  • Provide technical guidance, mentorship to junior engineers, and promote best practices and coding standards
  • Author and support high-quality technical documentation, assisting junior engineers in doing the same
  • 10+ years of professional experience in data engineering with focus on large-scale data ingestion and infrastructure
  • Deep expertise in Python programming and modern data engineering tools
  • Experience creating an automated production grade ETL process using Python and SQL
  • Strong understanding of ETL/ELT frameworks and distributed data processing
  • Experience with data processing, validation, cleaning and debugging data sets
  • Experience with API integration for seamless data exchange between systems
  • Proven experience handling and processing various file types and formats, including specialized healthcare standards such as HL7, 834, 837, and NCPDP
  • Experience integrating and consolidating data from diverse source systems into a unified repository, including data from EHR and claim systems, as well as from file-based and API integrations
  • Experience with processing large data sets (over 10GB)
  • Experience with incremental data processing and change data capture (CDC) methodologies
  • Strong experience designing scalable data architectures in AWS environment
  • Deep understanding of software engineering principles including test-driven development, loose coupling, single responsibility, and modular design
  • Experience with containerization technologies (Docker, Kubernetes) and building configuration-driven, maintainable systems
  • Proven ability to build tools and systems that can be operated by diverse engineering profiles through configuration rather than code changes
  • Passion for building new and improving existing data infrastructure with robust, maintainable, and operationally excellent data systems
  • Familiarity with healthcare data and regulatory environments (HIPAA compliance) is a plus
  • Strong collaboration and communication skills; comfortable working with diverse technical and non-technical stakeholders
  • Excellent verbal and written communication skills with ability to explain technical infrastructure concepts to diverse audiences
  • Familiarity with healthcare data and regulatory environments (HIPAA compliance)
  • Dynamic role with opportunities for learning and growth
  • Collaboration with experienced Clinicians, Engineers, Software Architects, Data Scientists, and Digital Health Entrepreneurs
© 2024 Teal Labs, Inc
Privacy PolicyTerms of Service