By Lia Webster et.al. *

(This article is an abstract of the EVO White Paper entitled IPMVP's Snapshot on Advanced Measurement and Verification published in January 2020). In recent years, there has been increased interest in advanced measurement and verification (Advanced M&V) – sometimes referred to as M&V 2.0. Advanced M&V applications are characterized by the use of energy meter data in finer time scales with near real-time access; and the processing capacity of large volumes of data via advanced analytics, to give more accurate and timely feedback on energy performance and savings estimates.

Standard M&V methods have been established for several decades as a means to quantify the impacts of energy efficiency projects. IPMVP establishes the basis of meter-based M&V methods in Option C: Whole Facility. The last update of the IPMVP Core Concepts, released in 2016, includes discussion on advanced meter-based methods, including system-level sub-metering.

The basic IPMVP approach is unchanged for advanced M&V, but the increased availability of interval meter data offers several benefits, such as:

• Verifying savings in a shorter timeframe (e.g., less than three months after efficiency project completion, depending on source and timing of savings).
• Visibility of savings at a lower threshold (e.g., ability to see 5% savings using hourly meter data, as opposed to needing >10% savings if using monthly data).
• Ability to quantitatively characterize energy savings seasonally by time-of-day and/or day-of-week.

These enhancements are largely enabled by software providing powerful analytical and visualization capabilities of metered energy data supporting qualified M&V practitioners and building energy engineers.

The basics of conducting meter-based M&V are the same as published in the first IPMVP in 1997, as are the key technical limitations:

• The savings must be larger than the modeling error, and consistently larger than energy fluctuations at the facility throughout the year.
• The range of reporting period operating conditions should not stray outside of the operating conditions observed in the baseline period.
• Meter-based methods will include the effects of all changes occurring within the facility, with changes unrelated to the targeted project appearing as either increased or decreased savings.

Advanced M&V Drivers

The evolution of advanced M&V over the past decade has been driven by several interrelated factors. In many markets, the improved electric grid management for national security and resiliency requiring quicker feedback and improved monitoring and control of grid assets is a primary driver of smart meter infrastructure investments. Elsewhere, legislative actions, executive orders, and utility commission rulings have driven policy that calls for the use of utility meter data to account for demand-side management program impacts. At the same time, the availability of advanced metering infrastructure (AMI) data has been leveraged by private industry resulting in the rapid development of software and analytical tools, including various Energy Information Management Systems.

Given the various drivers of advanced M&V it is helpful to think of it more as a set of applications rather than as a single approach. Now that the core modeling approaches and methods are more mainstream, the potential of advanced M&V can be realized through various use cases, each having its own emphasis, including: performance tracking & cost reduction for building owners and energy managers, pay-for-performance (P4P), aggregated approach, utility embedded M&V, and third-party embedded EM&V:

While software can automate many of the data analysis steps of the M&V process, the advanced M&V process as a whole cannot be fully automated, and it is not a ‘one size fits all’ approach that makes existing M&V methods obsolete.

Requirements for stable building operations and levels of energy savings sufficient to consistently be seen in the model limits the use of advanced M&V. This method excels in certain projects and program situations (e.g., projects with a high level of savings and accurate baseline models) but, in most cases, requires a “human in the loop” to resolve limitations and contextualize results generated– as do other M&V approaches.

Variety of Models, Savings Uncertainty and Model Accuracy

The savings calculated by advanced M&V tools are based on the type of empirical model used, the interval of energy data used, independent variables included, and the specific technical adjustments made in applying the tool. The savings calculated for a given site, and the uncertainty in the estimates, will vary depending on the tool used and the approach taken by the practitioner.

Although tools vary substantially, they are generally based on two model types – change-point and time-of-week and temperature (TOWT). These models are based on linear regressions of energy use to outdoor air temperature and are popular with practitioners as they have proven effective, are intuitive, and limit overall predictive bias. Machine learning methods are also being developed, with artificial neural network being the most widely used method for building energy modeling.

Modeling approaches vary by project but selecting the most accurate model typically requires evaluating multiple model forms to determine the best option. The selection should be based on both statistical criteria and confirmation of expected data relationships. The relationships between outdoor air temperature and heating and cooling loads in buildings are fundamentally linear, although temperature responses vary by building and operating mode. This tie with the known physics of buildings contributes to the industry’s proclivity towards using these linear models; other mathematical relationships can exist if energy loads are driven by other factors (e.g., production processes with large variable-speed motors). Although these popular model forms have proven effective for most buildings, a “one-model-fits-all” approach is not best-practice nor adherent to IPMVP principles.

The only error that is typically quantified in meter-based M&V methods is the error from the empirical energy model(s), hence the intense focus placed upon proper model assessment. Measurement errors are not usually applied to meter-based methods that use revenue-grade utility meters, and AMI data is considered free of measurement errors once validated by the utility; sampling error would only apply in an evaluation study.

In reality, of course, other sources of error exist and include:

• Missing or irregular energy data.
• Flaws in independent variable data such as the source for local weather data.
• Methods used for addressing missing/anomalous data.
• Extrapolations beyond model limits.
• Model misspecification (e.g., specifying a 3-parameter versus 5-parameter change-point model, omitting an important production variable, overfitting from too little data, or leaving an unexplained residual trend).
• Dates selected for baseline and performance periods.
• Non-routine events and any subsequent adjustments.

One of the key benefits of meter-based methods over other M&V methods has been the ability to compute the uncertainty of the savings estimates based on the statistics from the energy model(s), often using the popular error metric “Fractional Savings Uncertainty” (FSU), or the relative precision of the model. FSU quantifies savings uncertainty for models that are essentially valid. Unfortunately, current FSU calculations are not reliable when using hourly or daily energy use data and tend to underestimate uncertainty.

This gap in uncertainty metrics can be mitigated by ensuring the models are as accurate as possible. Maximizing the accuracy of the models, without overfitting, will minimize uncertainty in savings. Often, a more accurate model is technically achievable, but it has not been optimized through additional analyses and customization. Evaluating additional model types, data increments, and independent variables for model improvements is best practice but can add time.

Non-Routine Events and Non-Routine Adjustments

Non-routine events (NREs) are changes in energy use due to changes in site characteristics or to “static factors” which are not used in the empirical energy models or related to the energy project. Typical changes to static factors at a site include significant changes in the number of occupants and occupancy schedules, significant operational changes, equipment shut-downs or removal, maintenance periods, modifications to tenant spaces, the addition of solar panels, or even changes in facility size. These unexpected changes in energy use are the most significant complication faced by meter-based M&V approaches.

Current strategies to identify NREs include specific data visualization methods as well as analytical approaches such as the analyses of model residuals and the use of specific dissimilarity indices to flag irregularities. Once a potential NRE has been identified, what action is warranted? That depends on when the ‘event’ happened, its duration, and level of impact on energy use. A phone call to the site to inquire about operations can save time deliberating its source. Minor, short-term anomalies are less concerning than significant lasting changes. Similarly, NREs that occur during the baseline period are more readily addressed (if identified during baseline model development) than changes occurring during the performance period. If a potential non-routine event has a ‘significant-enough’ impact, a non-routine adjustment (NRA) may be warranted.

If the impact on the savings warrants action, the root cause of the change should be identified. The ‘event’ must be unrelated to the project being measured to justify an adjustment, which must be evaluated on a case-by-case basis. Although NRA strategies are outlined in IPMVP Option C guidance, this remains a big open issue. Submetering or custom engineering calculations have always been required for NRAs, but explicit examples are somewhat limited. With the wide-spread adoption of advanced meter-based methods, there are impressive new opportunities.

Future Directions for IPMVP

Industry context is evolving rapidly with implications for future applications of advanced M&V methods. Beyond the drivers for reporting accurate time-of-use energy savings, the brisk addition of demand-response (DR) efforts and new distributed generation (DG) resources (e.g., electric vehicles) will complicate known methods. Meter-based energy use is core to all of these efforts and will require the coordination of multiple baselines. Inevitably the need for ‘integrated M&V’ to delineate savings from EE, DR, and DG will require M&V approaches to evolve.

Option C methods using monthly data continue to be popular for natural gas and other fuels. Fuel use data is generally limited in granularity and frequency of collection but is improving over time. Strategies may evolve as natural gas metering advances, but the direction will likely be influenced by emissions accounting and efforts to de-carbonize buildings.

Technical developments in modeling methods and software tools are improving the accuracy of energy models, including the introduction of new and updated open-source methods and the launch of EVO’s tool-testing portal for objectively comparing advanced M&V tools. Technical guidance, pilot program case studies, and regulatory language examples can provide direction to those looking to incorporate advanced M&V into their portfolio of projects or programs.

IPMVP will release an Application Guide on Advanced M&V Approachesand Non-Routine Events in Spring 2020, and updates to the IPMVP Core Concepts will follow in 2021. The Application Guide will provide more specific guidance on issues related to advanced methods for energy efficiency applications. Direction on identifying and characterizing non-routine changes in energy use, quantifying their impacts, making necessary non-routine adjustments in savings, and managing savings uncertainty will be included.

(*) The White Paper IPMVP's Snapshot on Advanced Measurement and Verification, was prepared by Lia Webster, Facility Energy Solutions LLC, with contributions from Jessica Granderson, Samuel Fernandes, Eliot Crowe and Shankar Earni, Lawrence Berkely National Laboratory.