Director, Data Engineer - PGIM Technology (Hybrid - Newark, NJ)

Prudential Financial • Newark, NJ

20h • $170,000 - $225,000 • Hybrid

About The Position

As the Director, Data Lead for the PGIM Product Technology, you will engage in requirements gathering, lead design, architect and partner with business owners, technology leads and engineering teams to support shift to Microsoft cloud platforms for data processing and management. This candidate will work to define and enforce standards while also overseeing end-to-end data engineering including sourcing, modeling, security, governance, and business intelligence functions. You will design and lead engineers to implement solutions in accordance with industry best practices. Your expertise in data processing and data analytics using the Microsoft Fabric platform will be crucial in delivering successful data solutions. The candidate will play a key role to bridge solutions, communication, timelines, and challenges/solutions to project stakeholders for multiple, concurrent overlapping implementation initiatives. This position is hybrid and requires your on-site presence on a reoccurring weekly basis 3 days per week.

Requirements

• Bachelor’s or Master’s degree in Computer Science, Data Engineering or related discipline.
• 10+ years of experience in cloud computing with Microsoft Azure leading migrations, integrations, and deployments.
• 10+ years of experience as a Data Architect, with demonstrated expertise in designing, managing, and deploying client solutions.
• Strong knowledge of Azure services and features including Azure Synapse Analytics, Azure Data Factory, Azure Storage Accounts, Azure Logic Apps, Azure Functions, Azure SQL Database, Azure Active Directory, Azure DevOps, Azure Monitor, etc.
• Strong knowledge of Fabric workloads and features including Notebooks, Deployment Pipelines, Workspace architecture, Workspace Security, Data Warehouse/Lakehouse Architecture, Semantic Model, Data Factor, Data Flow, Data Activator, AI Skills, and Copilot, ultimately leading to AI capabilities in Power BI.
• Build and deploy intelligent applications using Azure AI Foundry, leveraging prompt orchestration, grounding with Azure AI Search, and agent orchestration features for enterprise-scale GenAI use cases.
• Independently implement GenAI workflows using Foundry.
• Proficient in CI/CD pipelines to deploy in azure environment using GitHub Actions.
• Proven ability to implement data governance tools (Purview preferred), data security (OneLake Security preferred), and compliance requirements like data retention schedules and audit.
• Specific experience in .Net architecture, design, and development using Azure platform services.
• Must have the proven ability of delivering multiple concurrent, overlapping projects in a fast paced technology team on-time, on budget and within scope.
• Experience in leading technical teams to drive business outcomes.
• Must come to this position with superior communication, collaboration, and problem-solving skills capable of conveying technical concepts to both technical and non-technical stakeholders.

Nice To Haves

• Financial Services experience working with Asset Management esp on Institutional funds and Mutual funds preferred.
• Microsoft certifications in data-related fields are preferred, such as Azure Solutions Architect Expert, Azure Developer Associate, Azure Administrator Associate, Azure Data Engineer, Fabric Engineer, Power BI Data Ana

Responsibilities

• Design and Implement Configurable Data Ingestion ETL framework including Data Quality, transformation and modelling.
• Implement Lakehouse solutions using Microsoft Fabric and paving the road to PGIM future state using Azure services and Fabric Infrastructure.
• Develop and deploy data integration solutions, including data ingestion, transformation, and delivery to consumers within Microsoft Fabric, while utilizing optimal storage formats and layers.
• Work with Infrastructure/Devops teams to come up with CI/CD pipelines as a framework that can be extended across organization.
• Architect solutions that incorporate BI tools like Power BI ensuring seamless data integration for analytics and reporting.
• Ensure data accuracy, consistency, and integrity across the data warehouse ecosystem by crafting robust solution designs.
• Design and Build AI solutions leveraging search, embeddings, retrieval, ranking and Retrieval-Augmented Generation (RAG).
• Set up roadmap for AI as a service to consuming systems and business stakeholders.
• Establish a workspace architecture strategy through design and implement of data warehouse and lakehouse solutions allowing data sharing between different teams avoiding duplication, yet adhering to security and governance rules.
• Lead a small team to design/build and monitor data and AI solutions.
• Ensure teams follow best practices and standards for security, compliance, scalability, reliability, and performance.
• Serve as SME in the Fabric space to technology partners, provide hands on training to development teams, and mentor business areas to maximize Fabric AI Capabilities.
• Create and maintain comprehensive documentation (Playbooks) of solution architectures to internal and external stakeholders covering Data Fabric architecture, processes, and procedures.
• Collaborate closely with our business and technology partners including data analysts, architects, engineers, and client stakeholders to gather requirements and translate them into scalable Microsoft Fabric solutions.
• Proactively seek out new knowledge and adapt to new trends, technical solutions, and patterns that improve the availability, reliability, efficiency, observability, and performance of products while also driving consistency in monitoring and operations at scale, and share knowledge with other engineers.

Benefits

• Market competitive base salaries, with a yearly bonus potential at every level.
• Medical, dental, vision, life insurance, disability insurance, Paid Time Off (PTO), and leave of absences, such as parental and military leave.
• 401(k) plan with company match (up to 4%).
• Company-funded pension plan.
• Wellness Programs including up to $1,600 a year for reimbursement of items purchased to support personal wellbeing needs.
• Work/Life Resources to help support topics such as parenting, housing, senior care, finances, pets, legal matters, education, emotional and mental health, and career development.
• Education Benefit to help finance traditional college enrollment toward obtaining an approved degree and many accredited certificate programs.
• Employee Stock Purchase Plan: Shares can be purchased at 85% of the lower of two prices (Beginning or End of the purchase period), after one year of service.
• Eligibility to participate in a discretionary annual incentive program is subject to the rules governing the program, whereby an award, if any, depends on various factors including, without limitation, individual and organizational performance.

What This Job Offers

• Job Type: Full-time
• Career Level: Director