Data Engineer

Cain Watters & Associates, LLC • Frisco, TX

13d • $116,000 - $135,000 • Onsite

About The Position

Reporting to the IT Manager, Data & Analytics, the Data Engineer is responsible for designing, building, and maintaining scalable data pipelines, data platforms, and data warehouse/data lakehouse architectures that support enterprise analytics, reporting, automation, and AI-enabled solutions. This role ensures that data is accurate, reliable, well-governed, secure, and optimized for consumption across Microsoft Fabric, Power BI, generative AI, AI agents, and future technologies. The Data Engineer will enable trusted, AI-ready data foundations and support client- and vendor-facing technology solutions that are critical to business operations, growth, and service delivery.

Requirements

• Bachelor’s degree in Information Systems, Computer Science, Data Engineering, or a related field, or equivalent work experience.
• 5+ years of experience in data engineering, data integration, or data platform development.
• Strong experience with data pipeline development and ETL/ELT processes.
• Proficiency in SQL and data modeling concepts (warehouse/lakehouse).
• Experience working with cloud-based data platforms (e.g., Azure Data Factory, Azure SQL, Microsoft Fabric, or similar).
• Understanding of data governance, data quality, and security principles.
• Experience designing or supporting data assets, integrations, or platform services used by enterprise applications, automation, analytics, or AI-enabled workflows.

Nice To Haves

• Experience with Microsoft Fabric, Lakehouse architecture, and Delta Lake concepts.
• Experience with PySpark or distributed data processing frameworks.
• Familiarity with Power BI, semantic models, and analytics workflows.
• Experience with data cataloging and governance tools (e.g., Microsoft Purview).
• Experience working in regulated environments with compliance requirements (e.g., PCI, financial services).
• Familiarity with generative AI, retrieval-augmented generation, semantic search, AI agent concepts, responsible AI practices, or AI-ready data architecture.
• Experience supporting client- or vendor-facing technology solutions, secure data exchanges, APIs, integrations, or externally consumed reporting platforms.
• DP-600, DP-700, AZ-305

Responsibilities

• Design, build, and maintain robust ETL/ELT data pipelines to ingest, process, and integrate data from multiple source systems (e.g., CRM platforms, APIs, and external data sources) into Azure and Microsoft Fabric.
• Develop and manage the enterprise data platform, including lakehouse architecture (bronze, silver, gold layers), ensuring data is structured, scalable, and optimized for analytics.
• Ensure the quality, integrity, and consistency of data through validation processes, monitoring, and proactive issue resolution.
• Optimize performance and scalability of data workflows, pipelines, and storage to support efficient data processing and reporting.
• Implement and enforce data governance, security, and compliance standards, including data lineage, access controls, and regulatory requirements.
• Enable AI-ready and agentic enterprise capabilities by preparing governed datasets, metadata, lineage, and integration patterns that allow AI agents, automation, and decision-support tools to operate safely and effectively.
• Support the design, delivery, and ongoing operation of client- and vendor-facing data integrations, reporting assets, APIs, and platform services that require reliable, secure, and scalable data engineering practices.
• Build and maintain data pipelines using Azure Data Factory, Fabric Data Pipelines, and other integration tools.
• Develop data transformation logic using SQL, PySpark, or similar technologies to standardize and prepare data for analytics use.
• Design and manage lakehouse structures in Microsoft Fabric, including bronze, silver, and gold data layers.
• Collaborate with BI developers to ensure data is modeled and structured appropriately for Power BI semantic models and reporting.
• Monitor data pipelines and platform performance, troubleshooting failures and optimizing workloads for reliability and efficiency.
• Implement data validation checks and reconciliation processes to ensure data accuracy and completeness.
• Manage data storage strategies, including partitioning, indexing, and lifecycle management for performance and cost optimization.
• Ensure secure data handling practices, including role-based access controls, encryption, and compliance with organizational and regulatory requirements (e.g., PCI considerations).
• Maintain documentation of data pipelines, data models, and system architecture to support transparency, governance, and knowledge sharing.
• Partner with business stakeholders and IT teams to understand data requirements and deliver scalable data solutions.
• Support data governance initiatives, including metadata management, lineage tracking, and data classification (e.g., integration with Purview).
• Prepare and expose curated, governed, and well-documented data assets for AI, generative AI, semantic search, retrieval-augmented generation, and agent-based enterprise workflows.
• Partner with business, IT, and analytics stakeholders to identify where data engineering can enable agentic workflows, automation, and AI-assisted decision-making while maintaining appropriate human oversight and control.
• Design, build, and support data integrations, APIs, secure data exchanges, and reporting components used in client- and vendor-facing technology solutions.
• Establish monitoring, observability, alerting, and support practices for AI-enabled and externally facing data solutions to ensure reliability, traceability, performance, and timely issue resolution.
• Contribute to continuous improvement of the data platform, recommending new tools, practices, and optimizations aligned with industry best practices.

Benefits

• Heavily subsidized medical insurance options
• Generous employer HSA contributions
• 100% employer-paid Short-Term Disability, Long-Term Disability, Accident, Basic Life Insurance, and Telehealth/Virtual Urgent Care/Whole Mental Health
• Dental and Vision plans
• Unlimited PTO
• Organization-paid holidays, including an extended Winter Break
• Safe Harbor 401k Plan
• Employer-paid professional dues, licenses, and renewals
• Continuing Education resources
• Family Planning Assistance Program and Paid Parental Leave
• Subsidized fitness programs

What This Job Offers

• Job Type: Full-time
• Career Level: Mid Level