About The Position

As a Data Platform Engineer on the AI and Technology Solutions team, you will design, build, and maintain scalable data solutions for internal systems and client-facing applications. The work includes hands-on data engineering across Microsoft Fabric, SQL Server, Lakehouse environments, and cloud data platforms, transforming raw data into reliable, secure, usable assets for analytics, reporting, automation, and operational workflows. You will use pipelines, notebooks, PySpark, and C# to orchestrate, process, and optimize data at scale. You will also help build Vastian’s client-facing data ingestion and analytics product. Using Azure technologies such as Event Hubs, Azure Data Explorer, and Azure Functions, you will securely receive, process, and analyze high volumes of data from client laboratory and healthcare systems in near real time.

Requirements

  • Experience or a strong educational background in data engineering, analytics, computer science, or a related technical discipline.
  • Strong analytical and problem-solving skills, including the ability to work confidently with complex structured and semi-structured datasets.
  • Hands-on experience with Azure data ingestion and analytics technologies such as Azure Event Hubs, Azure Data Explorer, Event Grid, Azure Functions, or Stream Analytics.
  • Proficiency in Kusto Query Language (KQL) and experience designing high-volume, time-series, or event-based data solutions.
  • Practical experience using Python, PySpark, and C# for data processing, transformation, automation, or analytics tasks.
  • Strong hands-on experience with SQL Server and writing efficient SQL for data transformation, integration, and analysis.
  • Experience working with JSON and other semi-structured formats, REST APIs, and schema design, validation, and evolution.
  • Familiarity with data modeling, ETL/ELT patterns, pipelines, and best practices for building scalable and maintainable data solutions.
  • Understanding of data security and privacy practices, including encryption, role-based access control, managed identities, and handling sensitive or regulated data.
  • Ability to communicate technical concepts and data outcomes clearly to technical and non-technical stakeholders.

Responsibilities

  • Design and build scalable client-facing data ingestion pipelines using Azure Event Hubs, Azure Data Explorer, Azure Functions, Event Grid, and related serverless services to receive and process client data in batch and near real time.
  • Design, build, and optimize data pipelines, Lakehouse solutions, and engineering workflows using Microsoft Fabric, SQL Server, PySpark, and related cloud data technologies.
  • Develop, maintain, and improve batch and orchestration pipelines that move and transform structured and unstructured data across source systems, Lakehouse layers, and cloud environments.
  • Develop and optimize Kusto Query Language (KQL) queries, ingestion mappings, update policies, materialized views, and retention and caching policies for fast, cost-effective analytics on high-volume data.
  • Work with application and platform engineers to define and support client-facing ingestion APIs, including authentication, payload validation, throttling, and error handling.
  • Handle defined and flexible client data schemas, including semi-structured JSON, and develop approaches for schema evolution, versioning, and normalization into curated, analytics-ready models.
  • Ensure strict tenant isolation and data security across multi-tenant data stores, segregating, access-controlling, and protecting client data in line with healthcare and regulatory requirements.
  • Implement data quality checks, monitoring, alerting, and observability for ingestion and processing pipelines, including dead-letter handling, replay, and recovery processes.

Benefits

  • Medical, dental, and vision insurance
  • Short- and long-term disability coverage
  • Life insurance and AD&D
  • Supplemental life insurance
  • 401(k) savings plan
  • Paid time off
  • Competitive base salary
  • Performance bonus program
© 2026 Teal Labs, Inc
Privacy PolicyTerms of Service