By Lia Webster* with support from EVO’s Focus Group on COVID-19**

COVID has caused changes such as increased use of remote offices and at-home schooling, reduced in-person shopping and a corresponding increase in delivery service, curbed use of public transportation, among others. The shifts in end-use energy consumption patterns are noticeable at both individual sites and on a larger scale, creating complications for measurement and verification (M&V) efforts in both energy efficiency projects and programs, especially for those using meter-based M&V methods (i.e., IPMVP Option C).

Managing the short-term effects are challenging, but the ongoing situation brings into question what the lasting impacts might be on both site-level and sector-level energy consumption patterns (i.e., will there be a new-normal or a return to a pre-COVID normal?). Permanent changes in systems and equipment can impact energy efficiency projects and programs, require adjustments to energy baselines, and potentially invalidate assumptions used in long-term utility planning. Considerations for addressing the impacts of COVID on how energy savings should be quantified vary based on M&V approaches used, the context of M&V efforts, stakeholder preferences, and contractual limitations. Discussions are included for utility funded programs and their evaluation efforts, for energy savings performance contracts (ESPCs), as well as for energy management efforts and focus on meter-based M&V.

Utility Programs

Addressing COVID impacts on savings estimates based on meter-based M&V methods is especially critical since changes in energy consumption directly impact estimated savings. Meter-level usage includes all electric loads in a building, each of which could be affected by COVID differently, to varying degrees, or not at all. A higher-than-normal degree of uncertainty in savings could therefore be introduced for each individual project due to COVID for energy efficiency engagements implemented during the pandemic. If a project is in a utility or publicly funded program allowable responses to manage these COVID effects on energy savings may be defined by the Program Administrator, the state’s Public Utilities Commission (PUC), or others having authority. Before acting, project sponsors should ensure that the PUC or others have not already provided instructions for how to proceed. In some cases, decisions by state regulators or utility program administrators may have adjusted requirements for reporting savings, including accounting for COVID’s impacts.

In many cases, however, the project provider may be expected to make suitable non-routine adjustments (NRAs). In these instances, the provider should report savings as accurately as possible, utilize industry M&V protocols and application guides (i.e., use an IPMVP NRA method suitable as detailed in EVO'sIPMVP Application Guide on Non-Routine Events and Adjustments (IPMVP NRE/A Guide)) where possible, and document all procedures. Program efforts commonly require normalized energy savings based on typical weather data to compare year-to-year results, and findings from program evaluation efforts may be used for long-term power resource planning efforts.

For programs using meter-based methods, NRAs are generally effective in estimating actual savings achieved if projects are impacted by COVID in the reporting period. Programs using ‘deemed’ savings estimates (e.g., prescribed kWh per widget replaced) may consider adjusting default parameters used, (e.g., hours of operation) which are based on ‘typical’ pre-COVID operations often detailed in Technical Reference Manuals (TRMs)¹ ; custom calculations can be similarly adjusted to reflect reporting period savings.

Moving forward, if part of a ‘new normal’ includes significant permanent shifts in energy loads across sectors (i.e., commercial to residential) then energy efficiency programs may need to reconsider underlying assumptions used to estimate savings (e.g., loads, hours of operation, life of measure)².

Examples of Energy Efficiency Programs Using Meter-Based M&V Methods in the US and Canada

Aggregate Methods Using Comparison Groups

Some residential and small commercial programs have adopted lower rigor meter-based calculation methods³ to determine avoided energy consumption where significant changes in individual project conditions (NREs) are ignored and corresponding NRAs are not made⁴. In the face of the pandemic, some of these mass-market programs developed a method to control for program-level COVID impacts using comparison groups⁵. These methods are limited to estimating avoided energy consumption in groups of very similar buildings (i.e., single-family residential and small commercial), and are not suitable for large commercial and industrial programs focused on participants with unique buildings, equipment, and energy use patterns.

Determining energy savings using comparison groups is not an M&V approach since savings adjustments are not based on site-specific conditions as required by IPMVP. Adjustments to estimated savings using comparison groups are typically made in retrospect (i.e., ex-post) program evaluations (EM&V), not in initial program-level savings (ex-ante) reporting.

Utility Program Evaluation M&V (EM&V)

Ultimately, the bottom-line for programs is delivering the targeted level of energy or demand savings while meeting cost-benefit criteria (e.g., TRC>1)⁶ as determined by program impact evaluation, EM&V efforts. Utility program evaluations made after program-level reporting is completed can typically use any of the IPMVP M&V options, including meter-based methods⁷. When minimum savings targets and budgets are met, the exact amount of savings often becomes less of a concern as cost-effectiveness targets have likely been achieved as well. Exceptions include where exceeding program targets trigger a financial performance bonus for the utility, or if analyses for capacity markets is required.

Currently, some jurisdictions are assessing program impacts under ‘normal’ conditions assuming that conditions will revert to those in effect pre-COVID, and the overall impact from the COVID-affected period will be limited. To control for impacts from COVID, many evaluators may prefer isolating specific energy efficiency measures (EEMs), if feasible, and using IPMVP Option B-like methods (sometimes in addition to Option C). This allows more flexibility if savings for a ‘normal’ year (i.e., not impacted by COVID) are needed since both loads and operating hours affecting EEMs savings can be individually adjusted, although estimates may be complicated by changes made to equipment and systems.

Some programs using meter-based methods lack data on specific EEMs and site conditions (i.e., static factors) which can limit evaluators to using an Option C-like method. In these cases, applying IPMVP’s NRA methods can be used to estimate savings achieved during the COVID-impacted reporting period, which are likely less than expected at many facilities due to lowered loads and/or run-hours. In these cases, it may be difficult to estimate savings for a ‘normal’ year (i.e., not impacted by COVID).

PUCs may also consider modifying how estimated first-year savings are applied to determine lifetime savings in programs highly affected by COVID. The ‘effective useful life’ (EUL) values assigned to each EEM may be altered enough and compel a review of assumptions related to savings persistence.

By June of 2020, the Illinois Energy Efficiency Stakeholder Advisory Group’s (IlSAG) had requested input from members on how to approach projects whose savings were affected by COVID. The group of evaluators presented three options:

• Normalize energy savings for all years of Effective Useful Life (EUL)
• Do not normalize energy savings in CY2020; normalize other years of EUL
• Do not normalize energy savings in any year of EULThe evaluation group recommended the first option.

In this option, the evaluators would normalize the 2020 program savings for weather and other conditions, essentially aiming to estimate savings as if the pandemic had not occurred for all years of the project’s life using currently defined EULs. This would put the fully deemed and custom calculated savings on a level playing field since the measures with estimated savings deemed by the Illinois Statewide Technical Reference Manual (TRM) will not include impacts from the pandemic (although the installed measures likely were impacted in at least Year 1). After months of discussions following the recommendations, a Policy Resolution was released with specific guidance

COVID Impacts on Electric Capacity Market Bid Validation

Many utilities in the US also participate in electric capacity markets which rely upon estimations of coincident peak demand availability for regional grid management. Capacity bidding programs are used for grid-reliability, especially in demand constrained regions (e.g., PJM, ISO-NE)⁸. Managed by FERC, all ISO and RTOs except for Electric Reliability Council of Texas (ERCOT) and Southwest Power Pool (SPP) have capacity markets which use daily energy rather than hourly capacity to manage the system.

These bid-based programs pay participating utilities to deliver electric capacity during specific peak hours in future years, which in some cases include demand reductions from energy efficiency projects as part of a package of distributed energy resources. Projects bid into capacity markets require accurate reporting of time-specific demand savings (e.g., specific hours by season), and must be independently validated by a qualified third party.

Demand savings from energy efficiency (EE) projects bid into capacity markets can be subject to more stringent rules than a PUC’s rules for M&V in ratepayer¬-funded programs. For example, if a company goes out of business during the period the utility was counting on avoided demand from their EE project, the demand savings must be zeroed out after the date the company ceases operation. The load would not have existed and therefore the utility EE project did not avoid the specified load bid into the capacity market during peak hours. Where a PUC may allow the utility to claim at least a partial year of energy savings, the coincident demand impact (typically seasonal, usually summer) could be zero, depending on timing of the shut-down. Thus, if COVID caused a building shutdown, the regional operator may decide the savings are not eligible for the capacity market bid that year, or the savings may be greatly reduced. The policy adopted in each transmission region for managing impacts from COVID will need to be determined on a case by case basis.

COVID-induced changes in energy consumption patterns continue to be anticipated for many facilities as permanent measures to limit transmission of COVID are implemented¹⁰ but overall impacts on energy consumption in some regions are waning, such as in the north-eastern US managed by PJM¹¹. Incorporating impacts on measure life and savings from the pandemic, however, need to wait for clarity on longer-term outcomes.

Energy Savings Performance Contracts (ESPCs)

For projects under energy savings performance contracts (ESPCs), the specific contract terms and M&V plan details represent the contractual authority. Are NRAs required or allowable? Are details on making NRAs established, or do they need to be agreed upon? A negotiation may be required since M&V plans have historically lacked details on when and how to make NRAs.

If making an NRA is not anticipated, a reasonable agreed-upon solution should be used. The project’s approach to risk sharing should extend to unforeseen circumstances, such as COVID. In most cases, the actual worth of a project is a mix of ‘savings’ and the acquisition and installation of new equipment. In the face of a short-term savings shortfall, it is important to consider the overall value of the project, including its potential to perform in the future. If NRAs are needed, adjusting investment duration or lease terms may be appropriate.

Retroactively applying common risk mitigation strategies could also be considered, such as defining allowable ranges of values for specific site parameters. For example, a chiller upgrade may specify a minimum number of ton-hours of chilled water to be used per month should actual values decline. Some ESPCs require certain operating details are met; in these cases the M&V plans may include additional details on adjusting savings if site conditions change.

In cases such as municipal ESPC projects, the goal of the project may be to acquire equipment and perform facility upgrades. M&V efforts may be used to ensure savings meet fixed financial performance based on “guaranteed savings” which require a minimum level of saving are achieved. Tolerance for short-term uncertainty in savings can be higher based on a long-term outlook, which for some projects may extend for 10 to 20 years. In some instances, such as in bond-funded projects or for projects in federal facilities, the contract modifications could be a substantial burden. In all cases, however, the long-term technical accuracy of the M&V approach (e.g., baseline energy model) must be considered.

Energy Management

Energy management efforts and owner implemented M&V projects often have no external, contractual, or legislated monetary impacts associated with the outcomes. Goals of M&V efforts may include ensuring ongoing project performance or meeting energy reduction targets. Accurately accounting for the energy impacts from COVID is possible using IPMVP NRA methods. In some cases, resetting the baseline period (and corresponding regression-based energy models) after operations stabilize can allow for a new basis for tracking ongoing improvements using meter-based M&V. In the meantime, improvements and other site-level changes should be tracked and their savings may be estimated using a different M&V approach if needed.

To support users of EVO's IPMVP NRE/A Guide, EVO’s Focus Group on COVID-19¹² has developed several COVID-related resources (see previous article Impacts of COVID-19 on Measurement and Verification (M&V) of Energy Savings: Market Perspective). A white paper intended to act as an addendum to the IPMVP NRE/A Guide offers targeted guidance for meter-based M&V projects and programs in relation to the COVID pandemic will be published by EVO: Impacts of COVID-19 on Measurement and Verification (M&V) of Energy Savings: Options for Meter-Based Methods — IPMVP and Beyond.

REFERENCES

1) Technical Reference Manuals (TRMs) are used in many states to define methods and assumptions for savings calculation; California similarly uses ‘workpapers’ but is transitioning from their current Database for Energy-Efficiency Resources (DEER) to a TRM to reconcile and clarify details unrelated to COVID.

2) A re-evaluation of the expected years in service can be warranted, even without complications from COVID. See Dyson, C., Goldberg, M. , et. al., Measuring the Dead from the Living: Using Existing Equipment Stock to estimate Measure Lives, IEPEC, 2019.

3) Regression models with Cv(RMSE) of up to 1.0 are allowed in programs using CalTRACK, which is significantly higher than the industry standard maximum allowable value for Cv(RMSE) of ~0.25.

4) See discussion in IPMVP’s Snapshot on AM&V, 2020 on ‘Aggregate Methods’.

5) See Comparison Groups for The Coronavirus Era which is based on UPM Chapter: 8 Whole Building Retrofit with Consumption Data Analysis Evaluation Protocol. These methods are limited to single-family residential and small commercial programs.

6) Total Resource Cost (TRC) is one common cost-effectiveness criteria used to evaluate publicly funded programs such as those in California.

7) Ex-post evaluation approaches are often not be strictly adherent to IPMVP since site-specific end-use data may be limited or absent and EM&V efforts are conducted over a short timeline after the reporting period has concluded such that the range of operation of a measure (e.g., over the full range of typical summer operating temperatures) may only be partially represented in the M&V data.

8) Capacity and energy bidding mechanisms are also affected by COVID-19 , as described in Ensuring DER Inclusion in Capacity Markets May Require A Rethink of Resource Adequacy.

9) Pennsylvania-New Jersey-Maryland Interconnection (PJM) is a regional transmission organization (RTO) that coordinates the movement of wholesale electricity in all or parts of 13 states and the District of Columbia.

10) For example, ASHRAE now includes guidance for re‑opening after COVID‑19 closure, including HVAC mitigation strategies for “tactical commissioning of the systems to determine risk areas for the building operating in epidemic conditions.”

11) See analysis by PJM in linked chart.

12) EVO’s Focus Group on COVID-19 met from May 2020 through February 2021. Led by Lia Webster, the group included Todd Amundson, Anna Kelly, Dan Bertini, David Jump, Emily Cross, Colm Otten, Denis Tanguay, Abigail Hotaling, Faith DeBolt, Eva Urbatsch, Steve Kromer, Eliot Crowe, Greg Anderson, Patrick O'Shei, Brian Clippinger, Lionel Metchop, as well as others.

(*) Lia Webster, PE, CCP, is Principal at Facility Energy Solutions.

(**) EVO’s Focus Group on COVID-19 met from May 2020 through February 2021. Led by Lia Webster, the group included Todd Amundson, Anna Kelly, Dan Bertini, David Jump, Emily Cross, Colm Otten, Denis Tanguay, Abigail Hotaling, Faith DeBolt, Eva Urbatsch, Steve Kromer, Eliot Crowe, Greg Anderson, Patrick O'Shei, Brian Clippinger, Lionel Metchop, as well as others.