Data Engineer

Bridge Specialty Group • Plano, TX

23h

About The Position

Brown & Brown is seeking a Data Engineer to join its growing team in Plano, Texas. This hands-on engineering role advances the company’s Data & AI Platform by designing, building, and operating modern data solutions for AI/ML, analytics, business intelligence, and enterprise insights. The engineer works with a collaborative team and partners with engineers, product owners, architects, and business stakeholders to turn requirements into production-ready data pipelines and products.

The role spans the data-solution lifecycle: design, development, testing, deployment, monitoring, troubleshooting, and ongoing optimization. Solutions use modern cloud and lakehouse technologies, including Databricks, Delta Lake, and related services, with an emphasis on reliability, scalability, governance, quality, and operational support.

Requirements

• Bachelor’s degree in computer science, engineering, mathematics, information systems, or a related STEM field, or equivalent education and professional experience.
• At least 4 years of experience in data engineering, software engineering, or a related technical discipline, including developing and supporting production data solutions.
• Hands-on experience with Apache Spark/PySpark, Delta Lake, Python, and SQL.
• Experience building and supporting production batch and/or streaming data pipelines.
• Experience with Databricks capabilities such as Lakeflow Declarative Pipelines, Databricks Jobs, Auto Loader, Structured Streaming, Delta Lake, and Unity Catalog.
• Experience developing ETL/ELT pipelines to ingest, transform, validate, and publish enterprise data.
• Experience with workflow orchestration, including scheduling, dependencies, parameters, retries, notifications, and monitoring.
• Understanding of Unity Catalog and data governance concepts, including catalogs, schemas, tables, views, lineage, access controls, and permissions.
• Experience with Git and CI/CD practices, and exposure to Infrastructure as Code technologies such as Terraform or Databricks Asset Bundles.
• Strong Python and SQL development skills, including the ability to create maintainable, testable, production-ready data solutions.
• Experience troubleshooting and optimizing Spark workloads using Spark UI, job logs, query plans, partitioning, and join optimization.
• Understanding of relational, dimensional, logical, and physical data modeling concepts.
• Experience across the engineering lifecycle, from requirements and development through testing, deployment, monitoring, and production support.
• Strong problem-solving and analytical skills, including diagnosing and resolving data and production issues.
• Strong written and verbal communication skills and ability to collaborate effectively with technical and business teams.
• Experience working in Agile delivery models such as Scrum, Kanban, or SAFe.
• Advanced experience with Python and PySpark for data processing, automation, and reusable engineering components.
• Experience with cloud platforms such as Microsoft Azure, AWS, or Google Cloud.
• Experience with cloud storage technologies such as Azure Data Lake Storage Gen2, Amazon S3, OneLake, or Google Cloud Storage.
• Experience with modern data technologies such as Snowflake, Apache Iceberg, Apache Hudi, or BigQuery.
• Experience building ingestion pipelines using Databricks Auto Loader.
• Experience developing streaming solutions using Structured Streaming.
• Experience designing or consuming REST APIs and web services.
• Experience implementing data-quality, metadata, governance, or lineage capabilities.
• Experience developing data products for AI, Generative AI, machine learning, semantic models, APIs, advanced analytics, Genie Spaces, or AI agents.
• Experience working with insurance and health-domain data.
• Practical experience using AI-assisted development tools such as GitHub Copilot, Claude, or Databricks AI-assisted development capabilities.
• Strong experience building and supporting production-scale Databricks Lakehouse solutions.
• Strong Python skills applied to reusable data engineering frameworks, automation, libraries, and utilities.
• Experience with Delta Lake and modern lakehouse architectures.
• Experience developing governed datasets and data products for analytics, AI, and Generative AI use cases.
• Experience with automated testing, deployment pipelines, Infrastructure as Code, and modern software engineering practices.
• Demonstrated ability to troubleshoot complex data problems and improve solution reliability, performance, and maintainability.
• A strong engineering mindset emphasizing quality, automation, observability, scalability, and continuous improvement.

Responsibilities

• Design, develop, test, deploy, and support scalable batch and streaming data solutions.
• Build Databricks-native data engineering solutions using Delta Lake, Lakeflow Declarative Pipelines, Databricks Jobs, Auto Loader, Structured Streaming, and Unity Catalog.
• Develop production-grade ingestion, transformation, and data-processing pipelines using Python, PySpark, Apache Spark, and SQL.
• Build reliable and governed data pipelines that improve enterprise data accuracy, consistency, accessibility, and trustworthiness.
• Develop reusable components, libraries, templates, and automation to improve engineering productivity and consistency.
• Implement data-quality validation, monitoring, observability, and remediation processes for critical data assets.
• Integrate governance capabilities, including metadata, lineage, classification, certification, ownership, and access controls.
• Develop and maintain governed data assets supporting AI, machine learning, analytics, semantic models, APIs, Genie Spaces, and AI agents.
• Build and maintain workflow orchestration, including scheduling, dependencies, parameters, retries, notifications, and operational monitoring.
• Troubleshoot and optimize workloads using Spark diagnostics, job logs, query plans, partitioning strategies, join optimization, and Adaptive Query Execution.
• Support production data solutions through monitoring, incident resolution, performance tuning, migrations, upgrades, and cost optimization.
• Participate in technical design discussions, code reviews, testing, and continuous improvement of engineering practices.
• Collaborate with product owners, architects, operations teams, and engineers to resolve technical dependencies and deliver solutions.
• Maintain technical documentation for data pipelines, data models, operational procedures, and engineering solutions.

Benefits

• Medical/Rx, dental, vision, life, and disability insurance.
• Employee stock purchase plan (ESPP) and 401(k).
• Student loan assistance and tuition reimbursement.
• Free mental health and enhanced advocacy services.
• Paid time off and holidays.
• Preferred partner discounts.
• Benefit enrollment waiting periods or eligibility criteria may apply; offerings may vary by subsidiary or geographic location.

What This Job Offers

• Job Type: Full-time
• Career Level: Mid Level