EISO-PALDS-FA2 A4 SC3

Novalink Solutions LLC•Harrisburg, PA

About The Position

The Commonwealth of Pennsylvania is seeking a Data Engineer & Insights Analyst to support the Pennsylvania Longitudinal Data System (PALDS) as part of its analytics team. The role transforms complex data into trusted, usable information for program delivery, operational improvements, and executive decision-making across state government agencies. It involves understanding policy-relevant research questions, evaluating data sources, and developing analytics products that deliver actionable insights. The position designs and builds data pipelines, integrates and transforms data from multiple sources, identifies and addresses data quality issues, and establishes controls for responsible data use. The analyst supports large, complex, and confidential datasets; multiple teams, systems, and products; and data conversions related to system modernization or replacement. The role also contributes to optimizing the Commonwealth's longitudinal data architecture for future data initiatives. The successful candidate will create cloud-based data solutions and orchestration workflows, engineer analytical data environments, develop monitoring and data-quality frameworks, and build dashboards, reports, models, and other decision-support products. They will collaborate with data scientists, research analysts, policy analysts, stakeholders, and other analytics professionals, while ensuring compliance with applicable Commonwealth policies, governance standards, privacy requirements, security controls, and interagency data-sharing agreements.

Requirements

Advanced SQL knowledge and proficiency with relational databases and query authoring, plus working familiarity with a variety of databases.
Experience building and optimizing big data pipelines, architectures, and data sets.
Advanced proficiency designing, building, and refining dashboards, reports, analytical models, and decision-support products for executive and non-technical users.
Experience with student-level higher education data, including linking it to other sources such as employment and wage data to measure outcomes or generate decision-making insights.
Experience performing root-cause analysis on internal and external data and processes to answer business questions and identify improvement opportunities.
Strong analytical skills for working with unstructured datasets.
Experience building processes supporting data transformation, data structures, metadata, dependency, and workload management.
A successful history of manipulating, processing, and extracting value from large, disconnected datasets.
Working knowledge of message queuing, stream processing, and highly scalable big data stores.
Strong project management and organizational skills.
Experience supporting and working with cross-functional teams in a dynamic environment.
At least 5 years of hands-on experience with cloud-based data services such as AWS S3, EC2, Glue, EMR, RDS, Redshift, Athena, Lambda, or DynamoDB.
At least 5 years of proficiency in object-oriented or object-function scripting languages such as Python, R, Java, C++, or Scala.
At least 5 years of proficiency in relational SQL and NoSQL databases, including Oracle, MS SQL Server, Postgres, and Cassandra.
At least 5 years of experience with version control systems such as Git and CI/CD pipelines.
At least 5 years of knowledge of data modeling techniques, including dimensional modeling, data vault, and star/snowflake schemas.
At least 5 years of familiarity with data cataloging and metadata management tools such as AWS Glue Data Catalog.
At least 5 years of understanding of data governance, data lineage, and data quality frameworks.
At least 5 years of knowledge of REST API design and integration patterns.
At least 5 years of experience with big data tools such as Hadoop, Spark, or Kafka.
At least 5 years of experience with data pipeline and workflow management tools such as Azkaban, Luigi, or Airflow.
At least 5 years of experience with data integration service solutions from vendors such as Informatica, MuleSoft, Talend, or TIBCO.
At least 5 years of experience with stream-processing systems such as Storm or Spark-Streaming.

Responsibilities

Create and maintain optimal data pipeline architecture.
Design and implement cloud-based data solutions using AWS services such as S3, Glue, Redshift, EMR, Athena, Kinesis, and Lambda.
Build and maintain data orchestration workflows using Apache Airflow, AWS Step Functions, or similar services.
Assemble large, complex data sets that meet functional and non-functional business-area requirements.
Engineer analytical data environments for advanced statistical analysis, predictive modeling, machine learning, and evidence-building activities.
Partner with data scientists, research analysts, and policy analysts to ensure enterprise data assets support reproducible research and statistical modeling.
Identify and implement internal process improvements, including automation, optimized data delivery, and more scalable infrastructure.
Build infrastructure for extracting, transforming, and loading data from varied sources using SQL and big data technologies.
Build analytics tools that provide insights into education and workforce outcomes, operational efficiency, and other business performance metrics.
Develop and maintain data pipeline monitoring, alerting, and logging frameworks.
Implement data quality checks and validation frameworks across pipelines.
Develop dashboards, reports, analytical models, and other decision-support products.
Work with Executive, Data, Design, and Support teams to address data-related technical issues and support data infrastructure needs.
Ensure compliance with Commonwealth policies, data governance standards, privacy requirements, security controls, and interagency data-sharing agreements.
Create data tools that help analytics and data science team members build and optimize data products for ongoing operations and data-informed decision-making.
Collaborate with data and analytics professionals to improve functionality in data systems.
Support data conversions associated with system modernization and replacement efforts.