Senior Platform Engineer (DataOps)

Black Canyon Consulting•Bethesda, MD

13d

About The Position

Black Canyon Consulting (BCC) is seeking a Senior Platform Engineer (DataOps) to support work for the National Center for Biotechnology Information (NCBI) at the National Library of Medicine (NLM), an institute of the National Institutes of Health. The role supports NCBI’s new enterprise-wide DataOps Platform team, which applies modern technologies such as Kubernetes, GitOps, and containerization to data-intensive applications. The position is full-time and may be performed onsite at NCBI in Bethesda, Maryland, and/or remotely. The DataOps Platforms team develops and continually improves the DataOps platform, creates and maintains common tools and libraries, evaluates new technologies and practices, and helps NCBI developers adopt the platform. It also supports compliance with federal application security regulations and standards by providing automated solutions and compliance pipelines. The team embraces agile development and continuous improvement and encourages a growth mindset and leadership opportunities at any level.

Requirements

At least 7 years of experience in the field.
Strong coding skills in at least one programming language, such as Python or C++.
Experience with Kubernetes and containerization.
Experience with Google Cloud Platform (GCP), Amazon Web Services (AWS), Microsoft Azure, or equivalent cloud services.
Experience with Apache Kafka, Google Cloud Pub/Sub, or equivalent.
Experience with Apache Airflow or equivalent.
Experience with data-processing applications and modern cloud-based data-processing infrastructure.
Linux command-line skills.
B.S. in a STEM field (Engineering, Computer Science, Mathematics, or Physics), or equivalent industry experience in Systems Engineering.

Nice To Haves

Experience with Google Anthos.
Experience with Docker.
Experience with GitOps tools such as ArgoCD or equivalent.
Experience with infrastructure-as-code tools such as Terraform or equivalent.
Experience with GitLab, GitHub, Bitbucket, TeamCity, Artifactory, or equivalent products for Git source control, CI/CD pipelines, and artifact lifecycle management.
Experience with modern observability and logging tools such as Prometheus, EFK, TIGK, DataDog, Sensu, Jaeger, Sentry, OpsGenie, PagerDuty, Splunk, or equivalent.
Experience with secret-management tools such as HashiCorp Vault, CyberArk, Azure Key Vault, Google Cloud Secret Manager, or equivalent.
Experience with data-transfer tools such as AWS DataSync, Aspera, MinIO, CloudSoda, or equivalent.
Experience with Apache Pulsar, RabbitMQ, Amazon Kinesis, Apache Flume, Apache Storm, Apache Spark Streaming, or Google Cloud Pub/Sub.
Experience with best-practice design patterns in coding and architecture.
Experience working in an agile environment.

Responsibilities

Develop and continuously improve the DataOps platform.
Develop and maintain common tools and libraries.
Evaluate new technologies and practices.
Help NCBI developers adopt the platform.
Provide automated solutions and compliance pipelines to support compliance with federal application security regulations and standards.
Embrace agile development and continuous improvement.