Sr Data Engineer

Guild Mortgage•,

2d•$109,000 - $156,000•Onsite

About The Position

The Senior Data Engineer is a key technical position and plays an important role in the organization by leading and performing a number of activities related to the company’s Information Technology functions. The role is primarily responsible, under limited supervision, for executing the data strategy and analysis efforts within the company’s enterprise data platform by expanding and optimizing data and data pipeline architecture, as well as optimizing data flow and collection from cross functional areas. Responsibilities include evaluating, designing and developing optimal solutions that support software developers, database architects, data analysts and data scientists on various data initiatives that accommodate long-term growth, are easy to maintain and audit while also ensuring optimal data delivery architecture.

Requirements

Bachelor's Degree, BS in Statistics, Computer Science, Data Science, or related quantitative field, required.
Minimum five to seven years' experience in Data Engineer related role(s) and at least Minimum of three years spent in a senior technical level role(s) required.
Highly proficient in building and optimizing Big Data pipelines, architecture, and data sets.
Highly proficient with data loading, processing, and data warehouse design techniques: star or snowflake schema designs, etc.
Well versed in Informatica Intelligent Cloud Services, Administration, Environment configuration and set up of various tool suite (Mass Ingestion, CDC, Data Catalog, Data Quality, Data Integration, API Management).
Proven expertise in developing in Informatica Intelligent Cloud Services and various tools.
Proven experience with data lake and warehouse best practices and leading products in the marketplace.
Expert at creating data integrations using Extract, Transform and Load (ETL) tools and modern data pipes.
Strong knowledge of Kafka, Sparks, ESB and/or other messaging queues, real-time data integration & stream processing technologies.
Proven experience with querying and analyzing data from SQL relational databases such as AuroraDB, IBM DB2, MemSQL, MS SQL Server, MySQL, Postgres, Redshift or Snowflake, and NoSQL Databases such as Cassandra, DynamoDB, Redis, or MongoDB.
Experience with data science languages such as R, and Python as well as general purpose languages such as Java, Scala, C# or JavaScript.
Experience collecting structured, semi-structured and unstructured data in various popular formats and sourced from internal core systems as well as 3rd partner providers such as Google Analytics, Facebook Insights, Zillow, CoreLogic, MLS Data, Public Records, and Property Data.
Experience with data pipeline and workflow management tools: Azkaban, Luigi, Airflow, etc.
Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.
Strong analytical skills to build processes supporting data transformation, data structures, metadata, dependency and workload management.
Strong data governance skills to ensure highest quality of data is made available for data analysis and accuracy in reporting.
Problem solver with an ability to guide the team towards a solution.
Ability to prioritize multiple tasks in a deadline-driven environment, strong sense of urgency and responsiveness.
Strong detail orientation and highly organized with proven ability to lead effectively and drive results in a matrixed management environment.
Ability to think critically, including the ability to evaluate facts and data to draw conclusions, determine the downstream impact of decisions and associated risks.
Excellent verbal and written communication skills plus demonstrate strong leadership capabilities.
Strong interpersonal and team building skills.
Self-starter with the demonstrated ability to learn/adapt to new technologies and techniques.
Highly organized and detail-oriented; ability to work in a fast-paced, metrics-driven environment required.
Proficiency in Microsoft Office Suite, Word, Excel, Wiki, collaborative cloud-based programs, and third-party software applications required.
Commitment to company values.
Customer Service - Proactive attention to each person.
Integrity - Do and say what's right.
Respect - Treat others with dignity.
Collaboration - Listen and work together.
Learning - Seek knowledge and strive for improvement.
Excellence – Deliver the unexpected.

Responsibilities

Partner with key stakeholders, including the business unit leaders, Product, Data & Technology teams, to assist with their data-related technical issues and support their data infrastructure needs by collecting data from primary sources and optimizing the data architecture, improving quality of sourced data, and ensuring consistent delivery of data to key stakeholders in a timely manner.
Extract, Load & Transform (ELT) data from various primary data sources (internal & 3rd party) into the data lake to create optimal data pipeline architecture utilizing Snowflake and Informatica tool suites.
Assemble large, complex data sets that meet functional / non-functional business requirements.
Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.
Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and leading ELT tools. This will include set up and configure Informatica Intelligent Cloud Services Environment and Tools.
Design and Implement Processes to govern usage of the tools and platform at enterprise level for performance, security and audit compliance.
Design & Build analytics solutions that utilize the data pipeline to provide actionable insights into the business on Revenue, Operational Efficiency and Expense metrics.
Keep Personally Identifiable Information (PII) data separated and secured through table splitting, data masking, access control restrictions and redundancy (high availability across multiple zones).
Create data tools for data scientist and analytics team members that assist them in building and optimizing data analysis and reporting.
Improve the quality of data used for analysis by assessing the accuracy of new data sources and the effectiveness of new data gathering techniques.
Develop processes and tools to monitor and analyze data warehouse performance and data accuracy.
Contribute to the team of data professionals by using expertise to answer questions and share repetitive design patterns with less experienced teammates, enhancing skillsets and competencies of team members, and sharing technical knowledge throughout the team.
Lead in stakeholder reviews and design sessions.
Provide data, reports, and information to management as needed.
Identify, track, and monitor trends and avoidable technology-related errors; work across functions to develop complex solutions, improvements, and stop-gaps.
Focus on the continual improvement of policies, procedures, and processes falling under the scope of authority.
Use expertise to resolve high level issues that cannot be solved by teammates.
Stay abreast of latest technology trends and participate in high-level decisions impacting the direction of the Information Technology function.
Consistently monitor and model platform usage, database sizes, compute resources and third-party costs to ensure that the data teams' spend is as cost-efficient as possible.
Champion data transformation to new ways of working and generating insights.
Drive strong communications, partnerships, and stakeholder management with senior leaders, functional managers and staff.
Perform other duties as assigned.