Data Engineer

Bits In Glass

About The Position

As a Data Engineer at Bits In Glass, you will build data pipelines, data warehousing, and data integration solutions for analytics and AI use cases. You will collaborate with Delivery Managers, ML Engineers, and other stakeholders to ensure data flows seamlessly and is accessible, supporting data-driven decision-making for clients. The role calls for a quick learner who is excited about innovation and continually develops expertise in data, AI, and cloud technologies. Bits In Glass is a high-growth cloud consulting firm with offices in Edmonton, Calgary, Toronto, Denver, India, and the United Kingdom, serving clients in Canada, the UK, India, and the US. Its consultants help organizations automate business processes and leverage modern platforms and web and mobile technologies.

Requirements

  • Degree in Computer Science, Engineering, or a related field, or equivalent experience.
  • 3–5 years of experience building production data pipelines.
  • Strong SQL and Python skills, including PySpark.
  • Knowledge of distributed processing, including partitioning, joins, and performance tuning, such as Apache Spark on Databricks.
  • Knowledge of lakehouse architecture, including medallion architecture and open table formats such as Delta Lake or Apache Iceberg.
  • Experience with declarative pipelines and orchestration tools, such as Lakeflow Spark Declarative Pipelines, Lakeflow Jobs, Airflow, or dbt.
  • Knowledge of batch, incremental, CDC, and API data ingestion.
  • Understanding of batch and streaming concepts, including watermarking and late-arriving data.
  • Understanding of OLTP and OLAP systems and how they differ and integrate.
  • Knowledge of relational and dimensional data modeling, including star schemas and slowly changing dimensions.
  • Knowledge of data warehousing platforms such as Databricks SQL, Snowflake, BigQuery, or Redshift.
  • Knowledge of data quality validation, monitoring, and alerting.
  • Knowledge of governance and security, including RBAC/ABAC access control, PII masking, and lineage.
  • Experience with at least one cloud platform: AWS, Azure, or GCP.
  • Experience with Git, automated testing, CI/CD, and infrastructure as code.
  • Strong problem-solving, communication, and collaboration skills.

Nice To Haves

  • Experience with vector databases such as Databricks Vector Search, pgvector, or Pinecone.
  • Experience preparing unstructured data for RAG and AI agents, including tools such as Databricks AI Functions or Agent Bricks.
  • Real-time streaming experience with technologies such as Apache Kafka, Azure Event Hubs, or Amazon Kinesis.
  • Experience serving operational data to applications using tools such as Lakebase, FastAPI, or Databricks Apps.
  • Experience with feature engineering and model tracking, such as MLflow or Feature Store.
  • Experience with NoSQL and graph databases such as MongoDB, Cosmos DB, or Neo4j.
  • Experience with BI and semantic-layer tools such as Unity Catalog metric views, AI/BI and Genie, Power BI, or Tableau.
  • Databricks Data Engineer Associate/Professional certification or a cloud data certification.
  • Client-facing and Agile delivery experience.

Responsibilities

  • Build and maintain scalable batch and streaming ETL/ELT pipelines for analytics and AI.
  • Ingest data from OLTP databases, files, SaaS applications, and APIs into a lakehouse.
  • Prepare data for machine learning and GenAI use cases in collaboration with ML Engineers.
  • Translate client data requirements into tailored solutions.
  • Ensure data quality, security, and governance.
  • Optimize cloud storage and compute for performance and cost.
  • Use Databricks as the primary platform for processing and orchestration.
  • Monitor, troubleshoot, and improve pipeline reliability.
  • Apply best practices for version control, code review, testing, CI/CD, and documentation.
  • Contribute to technical design and mentor junior team members.
© 2026 Teal Labs, Inc
Privacy PolicyTerms of Service