Data Engineer, AI & Business Intelligence- San Diego

Event Network•San Diego, CA

1d•$125,000 - $145,000

About The Position

Event Network is seeking a mid-level Data Engineer to develop and support the pipelines, integrations, curated datasets, models, and services behind our artificial intelligence and business intelligence initiatives. This is a hands-on engineering role for someone who can turn defined business and analytical needs into secure, supportable production solutions. You will partner closely with the Senior Business Analyst leading our AI and BI efforts and with other Information Systems resources. A central part of the role is strengthening our software delivery discipline: organizing work in Azure DevOps repos, applying legitimate Git-based version control, establishing and maintaining CI/CD pipelines, and building automated tests and deployment gates for data solutions.

Requirements

Approximately 3-5 years of relevant experience in data engineering, analytics engineering, database development, or integration development; equivalent practical experience will be considered.
Strong SQL skills, including complex queries, joins, aggregations, window functions, common table expressions, and performance troubleshooting on SQL Server or a comparable relational platform.
Working proficiency in Python and experience building reusable, readable code for structured data, APIs, and common file formats.
Experience building and supporting production ETL/ELT pipelines, including incremental processing, orchestration, logging, error handling, monitoring, and recovery.
Hands-on Git experience and a clear understanding of branches, pull requests, code review, merge practices, release history, and resolving conflicts.
Experience creating or maintaining CI/CD pipelines in Azure DevOps or a comparable platform, including automated validation and controlled promotion between environments.
Experience designing automated unit, integration, data-quality, schema, or regression tests and incorporating them into engineering workflows.
Understanding of development, test, and production separation; secure configuration and secrets management; and repeatable deployment practices.
Experience preparing governed, reusable data for Power BI or a comparable BI platform and working with facts, dimensions, relationships, and semantic models.
Ability to investigate ambiguous data problems, ask focused questions, and collaborate effectively with both business and technical stakeholders.
As a mid-level engineer, the person should independently complete clearly defined assignments, break moderately complex requirements into technical tasks, support production pipelines, troubleshoot routine issues, write readable code, test and document work, and communicate risks and blockers.

Nice To Haves

Azure Data Factory, Azure Databricks, Unity Catalog, Azure Data Lake Storage, Microsoft Fabric or OneLake, and Power BI semantic models.
Azure DevOps Repos and Pipelines, infrastructure as code such as Bicep or Terraform, Databricks Asset Bundles or comparable deployment automation, and test frameworks such as pytest.
Azure Key Vault, Microsoft Entra ID, Azure Monitor, Application Insights, Log Analytics, or Microsoft Purview.
Microsoft Dynamics NAV or Business Central, LS Retail, retail sales and inventory data, e-commerce, SFTP, partner-file integrations, and financial or operational reconciliation.
Guidance is expected for enterprise-wide architecture, company-wide governance, major vendor or infrastructure decisions, highly sensitive data, and unfamiliar advanced AI patterns.

Responsibilities

Design, build, maintain, and support scheduled, incremental, and event-driven data pipelines from databases, APIs, operational applications, files, and approved Microsoft 365 sources.
Develop transformations and curated data products for Azure Databricks, Unity Catalog, Power BI semantic models, analytics, and approved AI applications.
Own code and deployment hygiene in Azure DevOps: organize repositories, use branches and pull requests appropriately, maintain useful commit history, apply peer-review practices, and keep production code out of personal or unmanaged locations.
Establish and maintain CI/CD pipelines that validate, package, and promote data pipelines, notebooks, SQL, infrastructure, and configuration across development, test, and production environments.
Build automated tests for transformation logic, pipeline behavior, schemas, data quality, reconciliations, and regression scenarios; integrate those tests into pull-request and deployment workflows.
Implement configuration management, environment-specific settings, secret handling, approvals, rollback or recovery procedures, and auditable releases.
Create reliable ingestion and retrieval processes for approved documents and structured data used by AI solutions, including metadata, indexing, security-aware access, evaluation data, and telemetry where appropriate.
Implement monitoring, logging, alerts, retry and reprocessing patterns, and production support procedures for assigned pipelines, datasets, integrations, indexes, and services.
Profile and reconcile data, detect missing or duplicate records, validate business rules and control totals, and prevent incomplete loads from being presented as successful.
Maintain technical documentation, source-to-target mappings, deployment instructions, runbooks, and change histories as part of the definition of done.
Communicate progress, tradeoffs, risks, data limitations, and blockers early and clearly to technical and business partners.
Ingest data from SQL Server, Dynamics NAV and LS Retail, point-of-sale and e-commerce systems, inventory and replenishment systems, APIs, SFTP locations, partner files, Microsoft 365 content, and other approved sources.
Build reusable scheduled, incremental, and event-driven workflows with error handling, logging, retry, failure notification, safe reprocessing, and change-impact awareness.
Document source mappings, schedules, dependencies, transformation rules, and operational ownership.
Profile, clean, standardize, and combine source data into governed bronze, silver, and justified gold data products or semantic models.
Create facts, dimensions, historical tracking, conformed identifiers, approved calculations, and reusable datasets for Power BI and other authorized consumers.
Improve query and refresh performance, reduce duplicated report logic, implement row-level security where required, and reconcile outputs to approved sources.
Develop a practical understanding of retail sales, transactions, products, locations, inventory movement, purchasing, pricing, replenishment, vendors, e-commerce, and operating periods.
Prepare structured data and approved SharePoint or other document content for secure AI use.
Build ingestion and refresh processes for text, metadata, indexes, embeddings, and keyword, semantic, vector, or hybrid retrieval when the use case requires them.
Preserve source references and permission-aware access; capture retrieval results, errors, response times, usage, and quality indicators.
Develop representative evaluation and regression datasets, and help determine whether poor results originate in source content, transformation, indexing, retrieval, permissions, or application behavior.
Convert successful prototypes into maintainable implementations without taking ownership of company AI strategy or foundation-model research.
Validate counts, control totals, required fields, value ranges, relationships, timeliness, duplicates, volume changes, and source-to-target reconciliation.
Prevent partial or failed loads from appearing successful and communicate material data limitations promptly.
Monitor assigned services, diagnose routine failures, recover incomplete loads, confirm production changes, participate in root-cause analysis, and escalate architectural, security, vendor, or source-system issues appropriately.