Data Engineer

Black Canyon Consulting•Bethesda, MD

10d

About The Position

Black Canyon Consulting (BCC) is searching for a Data Engineer to support the design, development, and maintenance of data infrastructure that enables artificial intelligence, machine learning, and analytical workloads. The Data Engineer will help ensure that data is reliable, accessible, scalable, and operationally ready throughout the full lifecycle from ingestion through deployment. The Data Engineer will design, implement, maintain, and optimize data pipelines, APIs, data transformation workflows, and supporting infrastructure. This position will work with databases, data lakes, data warehouses, feature stores, and other data platforms to ensure information can be efficiently ingested, transformed, accessed, and used by analytical and AI-enabled systems. The Data Engineer will also develop and maintain ETL/ELT processes, feature engineering workflows, and service layers that expose data or model endpoints to downstream systems and applications.

Requirements

Experience designing, developing, and maintaining data pipelines
Experience building and optimizing ETL and/or ELT workflows
Experience working with relational and/or non-relational databases
Experience integrating databases, data lakes, data warehouses, or similar data platforms
Experience developing data transformation workflows
Experience supporting data used for analytical, AI, or machine learning workloads
Experience developing or integrating APIs and service layers
Strong understanding of data reliability, accessibility, quality, and operational readiness
Experience monitoring, troubleshooting, and optimizing data pipelines
Understanding of data modeling and data architecture principles
Strong programming and problem-solving skills
Ability to work collaboratively with software engineers, data scientists, analysts, and other technical stakeholders

Nice To Haves

Experience supporting machine learning or artificial intelligence environments
Experience developing feature engineering workflows or working with feature stores
Experience designing data infrastructure for high-volume or distributed systems
Experience with cloud-based data platforms and infrastructure
Experience developing APIs that expose data, analytical results, or model endpoints
Experience with automated data pipeline monitoring and alerting
Experience optimizing data pipelines for performance, reliability, and scalability
Experience working with modern data lakehouse or warehouse architectures
Experience implementing data validation, testing, and quality controls

Responsibilities

Design, implement, and maintain scalable data pipelines supporting analytical and AI/ML workloads
Build, monitor, troubleshoot, and optimize ETL and ELT data pipelines
Develop workflows for ingesting, transforming, processing, and delivering data
Configure and integrate databases, data lakes, data warehouses, feature stores, and other data platforms
Develop and maintain data transformation and feature engineering processes
Ensure data reliability, accessibility, quality, and operational readiness
Develop APIs and service layers to expose data or model endpoints to downstream applications and systems
Collaborate with data scientists, software engineers, analysts, and other technical teams to support data requirements
Monitor pipeline performance and resolve data processing or integration issues
Implement processes to validate data accuracy, completeness, and consistency
Support deployment and operation of data infrastructure throughout the system lifecycle
Identify opportunities to improve pipeline scalability, performance, maintainability, and efficiency
Document data pipelines, integrations, transformations, and technical processes