Senior Staff Data Engineer - Kafka

HCA Healthcare•Nashville, TN

About The Position

The Senior Staff Data Engineer / Engineering Leader leads a team responsible for developing, implementing, and maintaining enterprise data pipelines on HCA Healthcare’s Google Cloud Platform (GCP). The role combines technical and people leadership, overseeing scalable data integration solutions and a high-performing engineering team. It guides the integration of data from HCA source systems into GCP and the development and support of data exchanges with external vendors, partners, and affiliates. The position serves as a primary development resource for designing, coding, testing, implementing, documenting, and maintaining solutions for GCP cloud enterprise data initiatives. It works closely with data teams and broader project teams, often in a matrixed environment, and sets technical direction for groups of applications and related technologies. The role partners with architecture, product, data, and business stakeholders to build and monitor scalable platforms and pipelines, automate workflows, uphold data-delivery SLAs, enable self-service data discovery, and support enterprise-wide analysis of structured, semi-structured, and unstructured data using medallion architecture. The role also assigns and monitors engineers’ day-to-day work, sets expectations, coaches and develops team members, removes obstacles, and promotes collaboration and engineering best practices. It supports data security, testing, performance, CI/CD, deployment and operations, and may design and integrate AI agent-based solutions. The position requires independent problem-solving, adapting to changing priorities, and staying current with emerging technologies. The working conditions include prolonged sitting or standing at computer workstations and the ability to provide after-hours support.

Requirements

  • Bachelor’s degree in computer science, a related technical field, or equivalent experience.
  • At least 5 years of experience as a Data Engineer.
  • At least 10 years of experience in Information Technology.
  • At least 3 years of hands-on experience with GCP or an equivalent cloud platform.
  • Strong understanding of best practices and standards for GCP, AWS, or Azure data process design and implementation.
  • Experience with many of the following: Cloud Run, GKE, Cloud Functions, Spark Streaming, Kafka, Pub/Sub, Bigtable, Firestore, Cloud SQL, Cloud Spanner, HL7, FHIR, JSON, Avro, Parquet, Python, Go, Java, BigQuery, Dataflow, Data Fusion, Cloud Composer, DataProc, CI/CD, Cloud Logging, Vertex AI, NLP, and GitHub.
  • Ability to multitask and balance competing priorities.
  • Ability to define and use best-practice techniques and impose order in a fast-changing environment.
  • Strong problem-solving skills.
  • Strong verbal, written, and interpersonal skills and ability to work in a highly matrixed, team-oriented environment.
  • Ability to work under tight timelines and exercise sound judgment and independent problem-solving.
  • Ability to provide after-hours support.
  • Ability to work with Linux or UNIX, cloud platforms, and distributed, highly scalable processing environments.
  • Prolonged sitting or standing at computer workstations, including use of a mouse, keyboard, and monitor.

Nice To Haves

  • Master’s degree in computer science or a related field.
  • At least 1 year of healthcare experience.
  • Experience with HL7 data.
  • GCP Cloud Professional Data Engineer certification (a plus, but not required).

Responsibilities

  • Lead a team of data engineers developing, implementing, and maintaining enterprise data pipelines on GCP.
  • Oversee scalable data integration solutions and data exchanges with external vendors, partners, and affiliates.
  • Design, write, test, implement, document, and maintain solutions for GCP cloud enterprise data initiatives.
  • Set technical direction for groups of applications and related technologies, accounting for business, architecture, and technology constraints.
  • Collaborate with architecture, product, data, and business stakeholders to design, build, and monitor scalable data platforms and pipelines.
  • Implement automated workflows, uphold SLAs for timely data delivery, and enable self-service data architecture, query exploration, dashboards, data catalogs, and data discovery.
  • Build and support an enterprise GCP ecosystem for analysis of structured, semi-structured, and unstructured data using medallion architecture.
  • Bring streaming and batch data sources into GCP, transform and load data, and support requests to move data between clusters.
  • Produce modular, reusable, high-quality code with appropriate test coverage and modern abstractions and frameworks.
  • Promote and support data security practices aligned with industry standards and regulatory and legal requirements.
  • Mentor team members on complex projects and Agile practices; share knowledge and encourage team growth.
  • Lead data analysis efforts and proposals for data-related and data-architecture problems.
  • Lead implementation of unit and integration tests and promote or conduct performance testing where appropriate.
  • Evangelize data engineering best practices and standards and participate in or present at data community events.
  • Work independently, prioritize workload, meet deadlines, and adapt to rapidly changing business requirements and priorities.
  • Analyze requirements, design AI agent-based solutions, and integrate them into customer environments.
  • Collaborate with team members to execute development initiatives using Agile practices and principles.
  • Participate in deployment, change, configuration, management, administration, and maintenance of deployment processes and systems.
  • Provide leadership for the team’s continuous integration and continuous delivery automation pipeline.
  • Set expectations, assign and delegate work, monitor engineers’ day-to-day work, discuss performance and development, provide feedback and coaching, and remove barriers to team success.
  • Work with process owners, product owners, and business users to implement best practices for systems integration, security, performance, and data management.
  • Provide after-hours support and perform other duties as assigned.
  • Practice and adhere to the Code of Conduct philosophy and Mission and Value Statement.
© 2026 Teal Labs, Inc
Privacy PolicyTerms of Service