Senior Data Engineer

Gunvor Group Ltd•Houston, TX

13h•Onsite

About The Position

As a Senior Data Engineer specializing in Natural Gas and Power, you will support the design, development, and optimization of advanced data infrastructure and products that support critical trading, risk, and market analytics functions. You will collaborate closely with traders, quantitative analysts, risk managers, and IT stakeholders to ensure data systems are scalable, secure, and aligned with the strategic goals of the energy business. In addition to hands-on engineering, you will play a pivotal role supporting the data architecture roadmap, mentoring engineers, and driving innovation in data-driven decision-making across the Natural Gas and Power value chain. This role reports to the Data Engineer Lead.

Requirements

Master’s or Bachelor’s degree in Computer Science, Data Engineering, Applied Mathematics, or a related technical field.
5+ years of experience in data engineering, with at least 3 years in a senior role.
Proven experience in the energy trading sector, ideally with exposure to Natural Gas and Power markets, balancing mechanisms, and regulatory frameworks (e.g., REMIT, EMIR).
Azure: ADLS Gen2, Event Hubs, Synapse Analytics, Azure Databricks (Spark), Azure Functions, Azure Data Factory/Databricks Workflows, Key Vault, Azure Monitoring/Log Analytics; IaC with Terraform/Bicep; CI/CD with Azure DevOps or GitHub Actions.
Snowflake (on Azure or multi-cloud): Warehousing design, Streams & Tasks, Snowpipe/Snowpipe Streaming, Time Travel & Fail-safe, RBAC & row/column security, external tables over ADLS, performance tuning & cost governance.
Programming & Engineering Practices: Strong OOP in Python and Java/Scala; SDLC leadership, DevOps mindset, TDD/BDD, code reviews, automated testing (unit/integration/contract), packaging and dependency management, API design (REST/gRPC).
Web & Data Acquisition: Robust web scraping and ingestion with Scrapy, Requests, Playwright/Selenium; scheduling, retries/exponential backoff, change-data capture; ethical/legal collection practices (robots.txt, terms).
Orchestration & Quality: Airflow/ADF/Databricks Jobs, data contracts, Great Expectations (or similar), lineage/catalog (e.g., Purview), metrics/observability (Prometheus/Grafana/Application Insights).
Dataframe oriented programming: pandas, spark/snowpark dataframes, SQL data transformation

Nice To Haves

Designing low-latency pipelines for sub-second to minute-level telemetry, weather and market data; tuning Spark Structured Streaming/Flink/Kafka Streams.
Quality & Reconciliation for telemetry and market submissions (gap fill, resampling, deduplication, anomaly detection, schema evolution).
Serving Patterns: time-series stores and query layers (e.g., Delta Lake over ADLS, Iceberg, materialized views in Snowflake), APIs and event streams for downstream consumption.
English (fluent), any additional language is an asset

Responsibilities

Contribute to the definition of the target data architecture (lake/lakehouse, streaming, event-driven) and technology choices for high-volume time-series, market, and transactional data.
Build and operate end-to-end pipelines (ingest → quality → transform → serve) with strong SLAs/SLOs, lineage, and observability.
Establish coding, testing, CI/CD, and infrastructure-as-code standards; drive adoption of data governance, cataloging, and access controls.
Own capacity, performance, reliability, and cost efficiency of data platforms across environments (dev/test/prod).
Ensure data quality, lineage, and observability across all layers of the data stack.
Partner with trading desks, quantitative teams, and risk functions to translate business needs into data solutions that enhance decision-making and operational efficiency.
Act as a senior liaison between engineering and business stakeholders, ensuring alignment on data priorities and delivery timelines.
Prioritize a value-based backlog (e.g., faster close/settlement, improved forecast accuracy, reduced balancing penalties) and measure business impact.
Align data models and domain ownership with business processes (bids/offers, nominations, positions, exposures,outages).
Coordinate with Cybersecurity, Compliance, and Legal on sector-specific controls (e.g., REMIT/NERC-CIP considerations, data retention, segregation).
Incubate and industrialize data products: curated marts, feature stores, real-time decision APIs, and event streams for forecasting and optimization.
Introduce modern patterns (CDC, schema evolution, Delta/Iceberg, stream–batch unification) to improve freshness and resilience.
Evaluate and integrate external data (weather, fundamentals, congestion, capacity postings), internal and external vendor systems (ETRM) safely and at scale.
Collaborate with quantitative analysts to productionize ML pipelines (forecasting load/renewables, anomaly detection, etc.. ) with monitoring and rollback.
Lead incident reviews and architectural forums; provide pragmatic guidance on trade-offs (latency vs. cost, simplicity vs. flexibility).
Develop growth paths and learning plans focused on energy domain fluency and modern data engineering practices.
Implement robust monitoring/alerting, runbooks, and on-call rotations; drive MTTR down and availability up for critical data services.
Enforce data quality contracts (SLAs/SLOs), lineage, and reconciliation for market submissions, settlements, and reporting.
Optimize cloud spend and storage/compute footprints; plan capacity for market events and seasonal peaks.
Ensure security and compliance by design: least-privilege access, secrets management, encryption, auditability, and disaster recovery testing.