By Kevin Warren*

This article is in response to Denis Tanguay’s article published in the last issue of M&V Focus and titled “The Exact Science of Deemed Savings.” Denis’ primary point was that energy efficiency investments, including those supported by ratepayers or taxpayers, should be evaluated. We should not look just at the money spent. We should also not simply stipulate or “deem” the savings that were achieved. I agree wholeheartedly with this.

A reader, however, may come away from reading Denis’ article thinking that EVO feels that deemed savings are never an appropriate means of estimating savings. I hope to provide some thoughts on best practices for the appropriate applications for deemed savings, and also to show how M&V can be used to complement and validate deemed savings.

A basic tenet of the IPMVP is that the expense of estimating savings should be commensurate with the level of investment, the risks in the investment, and the contractual needs of the parties involved. In a performance contract, there is typically substantial money involved, and there is a clear contractual need for achieved savings to be calculated and documented. As a result, performance contracts should receive rigorous M&V. Deemed savings should be used very sparingly in performance contracts.

Many energy efficient actions are conducted on a far smaller scale. A homeowner purchases a single LED lamp in a hardware store, partly because the local utility has provided an “upstream” subsidy to that lamp that reduces its cost. An apartment building owner purchases efficient clothes washers partly on the basis of the government-funded energy rating label on the machines. A pizza shop owner installs a higher EER rooftop unit when the old one fails, partly because of a local rebate. In none of these cases do the purchaser or seller feel the need for M&V, and it does not make economic sense to require it. The size of the investment does not warrant the cost of M&V, and rigorous savings estimates are not required at the measure or project level. Instead, savings at the program level are of concern.

The motivation for validating savings for a utility or government-sponsored energy efficiency program in North America is typically to assess if the program or portfolio is cost effective as defined by their enabling legislation. Typically, there is a cost-effectiveness threshold above which the program ”passes” and lives to see another day. In other cases, there are savings targets that must be met. In any case, it is not necessary to know the savings from every measure or project with high certainty. It is not even necessary to know the program savings with absolute certainty (if this were even possible). The goal is typically to ensure the investment was a good one and to possibly identify better ones revealed by the evaluation.

The overheard remark that prompted Denis’ article was that some set of estimates was 100 % accurate. I fully agree that this is a misleading and inaccurate statement; however, is it possible the overheard remarks were intended to mean the cost-effectiveness test outcome was certain (or nearly certain) to be within an acceptable range?

The energy savings achieved by programs should be determined through a program impact evaluation. The impact evaluation is not itself M&V, but M&V plays a very important role within an evaluation and in determining appropriate deemed savings values to use in programming planning and tracking. M&V should not (with some exceptions) normally be required for every project. Let us examine two broad types of programs: ones that support large projects, and ones that support a much larger number of individually small impacts. Both program types can have very substantial budgets and savings, but they require different approaches to verifying the achieved savings.

Some programs support large custom projects, performance contracts, or large single-measure installations. M&V is typically a key element of the evaluation of nonresidential lighting retrofit programs, custom incentive programs, and other programs where average project savings are substantial. Common practice in these programs is to select a representative sample of projects and to conduct M&V on that sample.

The upstream lighting rebate program, an appliance rating program, and HVAC rebate program described earlier each support numerous very small energy-saving projects. Such programs typically use deemed or quasi-deemed savings algorithms when planning and tracking the progress of the program. These same algorithms and deemed values may be used or updated by an impact evaluator in order to determine program achievements.

Consider the task of determining the cost-effectiveness of such a program. A key input to the cost-effectiveness is estimating the savings achieved by the program through an impact evaluation. The impact evaluation is required to determine the aggregate effects of the hundreds or thousands of such installations that they have influenced. An evaluator can look at the critical parameters in a deemed or quasi-deemed savings algorithm and determine strategies for estimating each.

• What fraction of the purchased lamps are installed rather than having failed or been put in storage (in service rate)?
• What fraction of the purchased lamps have left the service territory of the utility (leakage)?
• What is the wattage of the purchased lamps?
• What is the average wattage of the lamps that were replaced or that would have been purchased in lieu of the program?
• What is the average operating hours of the purchased lamps?

These various factors can be determined in a single impact evaluation or through separate studies. Once determined, there is often little need to update them every year. Why should the average hours of operation of a screw-in lamp in a residence change from year to year? A large study conducted four years ago may still be entirely applicable, freeing up evaluation resources to focus on more transient factors. In this case, the lighting hours of use is a “deemed” value for the program.

M&V also includes measurements in support of jurisdictional assumptions for use in deemed or quasi-deemed savings algorithms, beyond site-specific project performance. M&V could mean measuring average hours of use by building type, verifying actual pre-existing baseline technologies on site to validate blended-average deemed baseline assumptions, or researching recent census information to verify that an assumed weighted average of commercial building types for a particular utility service area has been accurately characterized.

Note that deemed savings are meant only to provide a reasonable and cost-effective estimate of the savings for a large number of similar measures. The deemed savings estimate for any specific site may differ substantially from the actual savings for a single given site. If it is important for the savings from a particular installation to be known, then M&V is called for. This is typically the case for the larger projects supported by a program.

In conclusion, deemed savings play an important role in the planning, tracking, and evaluation of energy efficiency programs. There is nothing inherently wrong in their use as long as the assumptions are validated and as long as results are not required at the project level.

To meet the needs of impact evaluators planning, reviewing, or conducting M&V within impact evaluation, EVO formed the Evaluation Measurement and Verification (EM&V) Subcommittee of the IPMVP Committee. We plan to publish an Application Guide in early 2019.

(*) Kevin Warren, P.E.
Principal, Warren Energy Engineering
Chair of EVO’s IPMVP Evaluation, Measurement, and Verification (EM&V) Subcommittee