Data Scientist, Demand Forecasting & Analysis

New York ISO•Rensselaer, NY

8d•Hybrid

About The Position

The New York Independent System Operator (NYISO) manages the efficient flow of electricity on more than 11,000 circuit-miles of high-voltage transmission lines, dispatching power from hundreds of generating units across the state. The NYISO’s Operations department invites applications for a full-time Data Scientist to join the Demand Forecasting and Analysis team. Data Scientists for Demand Forecasting and Analysis develop long-term and short-term peak and energy demand forecasts for the New York Control Area through structured and ad-hoc analyses of electric load, weather and climate trends, and emerging technologies such as behind-the meter solar installations, electric vehicles and cold-weather heat pumps using statistical, engineering, algorithmic, data science, and other techniques. A Data Scientist utilizes both commercially available software packages and high-level software programming tools to manage and extract meaningful information from existing and new data sets and to predict future patterns of electric demand. A Data Scientist will collaborate with NYISO colleagues on all analytical, programming, modelling, and data management activities. A Data Scientist will lead initiatives to innovate on forecasting methods, tools, and software and to develop new data pipelines, software tools and visualizations, A Data Scientist will research and leverage forecasting developments within the research community and communicate complex, technical content to internal and external stakeholders through written reports and presentations. A Data Scientist may be tasked to support one or both required forecasting horizons. Short-Term horizon tasks focus on the support of the NYISO’s Market and Grid Operations departments. The Data Scientist will in this capacity develop real-time and day-ahead forecast models, manage the automated production of real-time and day-ahead forecasts and reports, and maintain the appropriate energy, weather, and solar databases of actual and forecast results. There is particular emphasis on enhancing the situational awareness of the Market and Grid Operations management and staff through the provision of deterministic and probabilistic analysis of current and prospective weather patterns and the associated impact on electric demand. Long-Term horizon tasks focus on the support of the NYISO’s System and Resource Planning, Capacity Market Operations and Finance departments through the development and provision of long-term and annual energy and demand forecasts, and maintenance databases of economic data, weather data, state policy programs, and demand-side technologies such as solar photovoltaics, distributed energy resources, and electric vehicles. A Data Scientist is expected to perform these duties independently with little direct supervision from a Senior Data Scientist. ESSENTIAL DUTIES and RESPONSIBILITIES Apply technical knowledge and experience to assist with the design, development and maintenance of forecast models (e.g., regression, machine learning, and analytical models) and data sets. Implement and maintain rigorous data quality control processes to ensure the cleanest possible datasets are available with which to train the forecast models. Create functional tests to ensure both model and forecasting system configuration changes satisfy the expected forecasting improvements and are reasonably free from defects. Perform forecast verification analyses and exploratory data analyses of system and regional demand patterns. Analyze and summarize large volumes of data using a high-level programming/scripting language. Create high impact charts and visualizations for communicating with internal and external stakeholders. Perform peer model and data set reviews and suggest changes that will increase performance, maintainability, scalability, or standards compliance. Collaborate with Operations, Software Development, other IT teams as needed and Project Managers to manage changes to team forecast models and the supporting applications/systems. Coordinate with neighboring ISOs and with regional and national reliability councils as appropriate. Provide task estimates and assist with the creation of work breakdown structures. Evaluate new technology, tools, and data sets. Incorporate approved technology, data sets and tools to enhance the forecast model development process. Document department tools, designs, data sets and procedures. Resolve stakeholder (both internal and external) complaints with products and respond to suggestions for improvement and enhancements. Mentor and coach Associate Data Scientists. Short-Term Forecasting Perform daily review of Day-Ahead and Real-Time energy and solar forecasts and recommend corrective actions to Operations. Revise forecast models of energy and solar power annually and recommend and develop new approaches periodically, or as circumstances require. Maintain database archives of actual and forecast energy, weather, and solar data. Monitor and evaluate energy and solar model forecast accuracy and weather forecast accuracy monthly. Perform annual assessment of system energy and solar forecasts based upon actual, normal and extreme weather conditions. Evaluate performance of weather and solar forecasting service providers monthly. Assess current and prospective weather conditions and communicate with solar and weather forecasting professionals regarding forecasts and forecast accuracy. Produce and distribute weekly and monthly prospective weather reports. Provide post-production application support as required. Support responsibilities may include consultation, problem identification, data analysis and/or emergency forecasting model changes. Collaborate and support the Release and Control team during forecast system deployments. Contribute to Grid Transition’s Uncertainty Response initiative with innovative data management, analytics, and visualization tools. Long-Term Forecasting Develop long-term econometric and end-use forecasts of system, zonal, or transmission district energy and seasonal peak demands. Develop long-term forecasts of demand side impacts, which may consist of energy efficiency, solar PV, codes and standards, distributed energy, electric vehicles, energy storage, and other technologies. Develop budget forecasts (one to five years) of system energy; monitor performance monthly on an actual and weather-adjusted basis. Update long-term and budget forecast models; recommend and develop new approaches periodically, or as circumstances require. Maintain database archives of actual and forecast energy, peak demand and hourly loads, weather data, economic data, and demand-side data. Perform monthly assessments of long-term and budget forecasts based upon actual, normal and extreme weather conditions. Prepare reports and presentations for meetings with NYISO market participants. Coordinate the preparation of the annual Load and Capacity Data Report.

Requirements

Bachelor’s Degree (BS) in Mathematics, Economics, Data Science, or related discipline
Three or more years of related experience in data science, load forecasting, operations research, load research, applied statistics, mathematics, or meteorology required, or combination of education, experience, and training.
Ability to read, analyze and interpret energy forecasting and statistical concepts and methodologies applicable to NYISO energy and peak demand forecasting activities.
Ability to read, analyze and interpret general business periodicals, professional journals, technical procedures, or governmental regulations. Ability to write reports, business correspondence and procedure manuals.
Ability to work with mathematical concepts such as probability and statistical inference, derivatives and integrals, and fundamentals of geometry / trigonometry. Ability to apply concepts such as fractions, percentages, ratios, and proportions to practical situations.
Ability to define problems, collect data, establish facts, and draw valid conclusions. Ability to interpret technical instructions in mathematical or diagram form and deal with several abstract and concrete variables and apply to other areas.
Proficiency or aptitude to become proficient in the use of MetrixND, SAS/Python, Excel, OSIsoft PI, SQL, VBA, MS Access and PowerPoint, and other custom proprietary programs.
Basic working knowledge of GIS software, primarily the ESRI ecosystem
Competency required in Windows operating environment using MS Office applications, e-mail, and internet programs.
Excellent interpersonal, organizational and teamwork skills needed.

Responsibilities

Apply technical knowledge and experience to assist with the design, development and maintenance of forecast models (e.g., regression, machine learning, and analytical models) and data sets.
Implement and maintain rigorous data quality control processes to ensure the cleanest possible datasets are available with which to train the forecast models.
Create functional tests to ensure both model and forecasting system configuration changes satisfy the expected forecasting improvements and are reasonably free from defects.
Perform forecast verification analyses and exploratory data analyses of system and regional demand patterns.
Analyze and summarize large volumes of data using a high-level programming/scripting language. Create high impact charts and visualizations for communicating with internal and external stakeholders.
Perform peer model and data set reviews and suggest changes that will increase performance, maintainability, scalability, or standards compliance.
Collaborate with Operations, Software Development, other IT teams as needed and Project Managers to manage changes to team forecast models and the supporting applications/systems.
Coordinate with neighboring ISOs and with regional and national reliability councils as appropriate.
Provide task estimates and assist with the creation of work breakdown structures.
Evaluate new technology, tools, and data sets. Incorporate approved technology, data sets and tools to enhance the forecast model development process.
Document department tools, designs, data sets and procedures.
Resolve stakeholder (both internal and external) complaints with products and respond to suggestions for improvement and enhancements.
Mentor and coach Associate Data Scientists.
Perform daily review of Day-Ahead and Real-Time energy and solar forecasts and recommend corrective actions to Operations.
Revise forecast models of energy and solar power annually and recommend and develop new approaches periodically, or as circumstances require.
Maintain database archives of actual and forecast energy, weather, and solar data.
Monitor and evaluate energy and solar model forecast accuracy and weather forecast accuracy monthly.
Perform annual assessment of system energy and solar forecasts based upon actual, normal and extreme weather conditions.
Evaluate performance of weather and solar forecasting service providers monthly.
Assess current and prospective weather conditions and communicate with solar and weather forecasting professionals regarding forecasts and forecast accuracy.
Produce and distribute weekly and monthly prospective weather reports.
Provide post-production application support as required. Support responsibilities may include consultation, problem identification, data analysis and/or emergency forecasting model changes.
Collaborate and support the Release and Control team during forecast system deployments.
Contribute to Grid Transition’s Uncertainty Response initiative with innovative data management, analytics, and visualization tools.
Develop long-term econometric and end-use forecasts of system, zonal, or transmission district energy and seasonal peak demands.
Develop long-term forecasts of demand side impacts, which may consist of energy efficiency, solar PV, codes and standards, distributed energy, electric vehicles, energy storage, and other technologies.
Develop budget forecasts (one to five years) of system energy; monitor performance monthly on an actual and weather-adjusted basis.
Update long-term and budget forecast models; recommend and develop new approaches periodically, or as circumstances require.
Maintain database archives of actual and forecast energy, peak demand and hourly loads, weather data, economic data, and demand-side data.
Perform monthly assessments of long-term and budget forecasts based upon actual, normal and extreme weather conditions.
Prepare reports and presentations for meetings with NYISO market participants.
Coordinate the preparation of the annual Load and Capacity Data Report.