Lead Data Engineer

NextEra Energy • Juno Beach, FL

30d

About The Position

We are seeking a hands-on Lead Data Engineer to architect and deliver enterprise-grade data and AI products that power an executive intelligence platform. The role sits within Marketing & Communications but supports broader enterprise use cases. This person will lead technical design, mentor engineers, and quickly move data products and AI applications from development through QA and into production while maintaining reliability, security, governance, explainability, and cost discipline.

Requirements

• Python
• PySpark
• SQL
• Databricks pipelines
• Databricks Jobs
• Delta tables
• Git
• CI/CD
• automated testing
• environment promotion
• release validation
• monitoring
• rollback practices
• secure authentication
• pagination
• rate-limit handling
• retries
• schema management
• data contracts
• lineage
• quality controls
• observability
• recovery procedures
• prompt templates
• versioning
• scheduled or event-triggered prompt execution
• RAG
• vector search
• knowledge graphs
• agent and tool orchestration
• Databricks LLM applications
• business rules
• algorithms
• model outputs
• confidence thresholds
• human review
• evaluation
• guardrails
• tracing
• performance monitoring
• workload right-sizing
• efficient job design
• autoscaling
• auto-termination
• table and file optimization
• retention policies
• spend monitoring

Responsibilities

• Design, build, test, deploy, and operate Databricks pipelines, Jobs, Delta tables, and reusable ingestion frameworks using Python, PySpark, and SQL.
• Lead data architecture, engineering standards, code reviews, and mentoring while rapidly productionizing prototypes through Git, CI/CD, automated testing, environment promotion, release validation, monitoring, and rollback practices.
• Integrate structured, semi-structured, and unstructured data from internal systems, databases, APIs, files, documents, vendor feeds, public sources, and event streams.
• Build resilient, governed integrations with secure authentication, pagination, rate-limit handling, retries, schema management, data contracts, lineage, quality controls, observability, and recovery procedures.
• Engineer production AI workflows using prompt templates and versioning, scheduled or event-triggered prompt execution, RAG, vector search, knowledge graphs, agent and tool orchestration, and Databricks LLM applications.
• Build explainable decision systems that combine business rules, algorithms, model outputs, confidence thresholds, and human review; implement evaluation, guardrails, tracing, and performance monitoring.
• Optimize Databricks compute and cloud storage costs through workload right-sizing, efficient job design, autoscaling and auto-termination, table and file optimization, retention policies, and ongoing spend monitoring.
• Partner across data science, product, DevOps, security, architecture, and business teams; communicate clearly, surface risks early, and drive practical solutions.

What This Job Offers

• Job Type: Full-time
• Career Level: Senior
• Education Level: No Education Listed