Q: part 1: The first, if I make an optimization in the boiler system to reduce energy consumption in heating and domestic hot water, should I base myself on option C or B?

A: part 1: Either could be acceptable/applicable but depends on considerations such as the following:

Intent and goal for the M&V – is the goal to relate the verified savings ultimately to the utility bill/facility meter or is the goal focused on the boiler retrofit performance itself?
Baseline data availability;
If the boiler and hot water systems are currently supplied by a dedicated meter (natural gas or electric), for which the boiler/heater systems represent a large part of the load, with complete 12 months (or longer) data and a statistically valid regression model can be developed based on available relevant independent variables, then an Option C could be viable. If the savings are less than 10% of the metered data, then refining the data for regression to hourly or weekly intervals could help.

If the boiler and hot water systems have historical data regarding energy consumption such as trend data from a BAS, as well as ideally other operational variables such as temperatures, flow rates, heating load supplied (energy meter) or that data can be collected for an appropriate amount of time to capture full range operational characteristics, then an Option B could be viable. All parameters would then be required to be measured post retrofit for the reporting period Option B savings verification.

Q: - part 2: The other question, if I want to optimize the use of a Water Chiller and I do not have 12 months of electricity bills, how can I measure the savings after the MEE, do I consider option B?

A: part 2: Yes. Similar to the approach above for the boiler system, pertinent baseline data would need to be collected to establish the retrofit isolation baseline and then the corresponding measurement plan in place for the post, reporting period to capture the energy and operational characteristics of the chiller (after the MEE). Considerations should be included regarding where the retrofit isolation measurement boundary is established, ie is it just the chiller itself or is the chilled water plant including the pumping, cooling tower (if applicable) to represent the entire plant and associated capturing of improved performance for total chiller plant savings.

Q: I would like to confirm the IPMVP Option that best aligns with our M&V approach. We provide fully financed, turnkey energy efficiency solutions to our commercial and industrial customers. These solutions can be deployed as either portfolio programs across multiple sites throughout the US or single site projects. We bill our customers based on the post-install energy savings for each "system" we install at a given site, where a "system" in our contracts is defined as the all of the equipment we install, retrofit, or optimize at a given customer site (e.g., replacement chillers, boilers, pumps, and air handling units installed at a single site can constitute a "system" in our contracts). For each contract "system", we install submeters to monitor and track the energy use of the equipment included in that "system" both before and after that equipment is optimized, retrofitted, or replaced. The pre-install metering period can vary system to system based on a variety of factors including the what equipment will be addressed and how local climate and/or site operations impact the energy use of that equipment. To measure the energy savings, we aggregate the post-install submeter interval data for the "system" and compare it to the baseline regression model established using OpenEEmeter and the pre-install submeter data. Internally, we have interpreted our M&V approach as Option C because we measuring savings at the whole building level using our submeter interval data to isolate the contract "system" from the rest of the energy using equipment at a given customer site. Is this interpretation correct? If not, which IPMVP Option best aligns with our standard M&V approach?

A: Based on the above description of your M&V strategy, since you are describing the measurement boundary for the sub-meters at the EEM equipment specific level, ("using our submeter interval data to isolate the contract "system" from the rest of the energy using equipment at a given customer site"), this approach would conceptually align with an IPMVP Option B Retrofit Isolation approach - assuming all parameters are being measured.
Please refer to the current IPMVP Core Concepts Chapter 8 for more guidance in selecting IPMVP Options as well as Chapter 9.2 regarding Option B applications and methodology.

Q: Do you have an example of a methodology to measure and verify the load reduction resulting from mass residential smart meter load limiting from 60Amps to 20Amps?

A: One thought would be to use the same method used for conservation voltage reduction. The home would be run every other day or week, with the smart panel alternately on and off. The two periods would be compared to discenr the level of load reduction.

Q: I have a question about which IPMVP Option most closely aligns with our M&V strategy. To perform M&V at our customer's sites, we install submeters to monitor and track the energy use of the addressed equipment/systems and aggregate that interval data to measure the energy savings. The percentage of whole building energy use captured by our submeters can vary building-to-building based on the impact of the addressed equipment/systems on whole building energy use. Rarely does the estimated energy savings of our projects fall below 5% of baseline whole building energy use. Based on our interpretation of IPMVP, we believe our M&V strategy most closely aligns with Option C because we are performing site-level M&V calculations using our aggregated submetering interval data. Is this interpretation correct? If not, which IPMVP option would most closely align with our M&V strategy?

A: Based on the above description of your M&V strategy, since you are describing the measurement boundary for the sub-meters NOT specifically at the EEM level, this approach would conceptually align with an IPMVP Option C Whole Facility approach - assuming your submeters will also be measuring non-EEM level energy consumption and demand, and associated potential changes, which must be accounted for in the details of the savings calculation methodology.
Please refer to the current IPMVP Core Concepts Chapter 8 for more guidance in selecting IPMVP Options as well as Chapter 9.3 regarding Option C applications and methodology.

Q: I am seeking clarification regarding the evaluation of R² and CV(RMSE) in the M&V guidelines. Specifically, is the evaluation of these metrics intended to be performed on the full training dataset, or should it incorporate the testing dataset or cross-validation techniques?

In machine learning practice, there are well-established reasons to avoid relying solely on the full training dataset for model evaluation, as it can lead to overfitting and does not necessarily reflect the model's performance on unseen data. This approach contrasts with my current understanding of the IPVMP guidelines, which seem to imply an evaluation on the training set alone.

Could you please provide guidance on this matter to align our model evaluation process with the best practices in the field? Thank you for your time and expertise.

A: The evaluation of R2 and CV(RMSE) is based on ordinary least squares regression models, and their values should be calculated and checked for the training period data sets from which they are developed. This is a minimum set of criteria for evaluating models.

IPMVP recognizes that there are additional ways to check model validity, including out-of-sample testing and cross-validation. If done properly, these methods provide additional information about a model’s prediction capabilities and validity. IPMVP currently has no recommendations for how these methods should be performed or what the criteria would be. However, IPMVP does not restrict M&V practitioners from using more advanced methods to demonstrate the validity of models used in the M&V analysis.

IPMVP agrees with your point about machine learning algorithms and encourages you to address this concern using more advanced methods to demonstrate model validity when using such modeling methods. IPMVP is working to assess and include these advanced model validity methods in future guidance documents.

Q:  I have a question about which IPMVP Option best aligns with our M&V strategy. We provide turnkey energy efficiency services to our commercial and industrial customers across their real estate portfolios. Our M&V strategy is to install submeters at each building to monitor and track the energy use of equipment/systems we address during our projects. For example, if we are only replacing 3 of the 5 packaged rooftop units at a retail building, then we will install submeters to only those 3 packaged rooftop units we replaced. To measure energy savings we aggregate our interval submeter data to eliminate potential impacts the non-addresses equipment/systems may have on our measured savings. Based on our interpretation of IPMVP, we believe our M&V strategy most closely aligns with Option C because we are using our aggregated submeter data to measure savings at a building-level. Is this interpretation correct? If not, which IPMVP Option most closely aligns with our M&V strategy?

A: Based on the above description of your M&V strategy, since you are describing the measurement boundary for the sub-meters at the EEM equipment specific level, this approach would conceptually align with an IPMVP Option B Retrofit Isolation approach - assuming all parameters are being measured.
Please refer to the current IPMVP Core Concepts Chapter 8 for more guidance in selecting IPMVP Options as well as Chapter 9.2 regarding Option B applications and methodology.

Q: I want to know how to calculation of specific product energy savings. Can you provide data on the formula for calculating energy savings for individual items?

A: Per your question, if the “specific product” or “individual items” is referring to an EEM, please refer to retrofit isolation Options (option A or B) in IPMVP Core Concepts for associated formulas.

If a “specific product” or “individual items” are production outputs from an industrial process or facility with EEM’s implemented, an appropriate response to the first question could be whole facility ala Option C.   Perhaps an energy use indexing type approach could  be appropriate?   One calculation for energy savings to share from BPA’s M&V Energy Use Indexing Protocol version 2.0, July 2018 (available for download here), is:

 Energy Savings (kWh) = ((kWh/units)base – (kWh/units)post) * units

 where:

Energy savings refers to savings occurring during the reporting period.

Units is the normalizing variable, such as square feet, widgets produced, etc. The number of units must be measured over the same time period (and at the same intervals) as the corresponding energy use.

Each parameter (kWh, units) in the ratio, kWh/units, is either a total or an average aggregated over a period of time. Alternatively, the ratio may be the slope of a regression with units as the independent variable and kWh (or other energy use measure) as the dependent variable.

 Two other formula’s:

Percentage Savings (%) = ((kWh/units)base – (kWh/units)post) / (kWh/units)base

Per-Unit Savings = (kWh/units)base – (kWh/units)post

 With data collected during baseline (base) and post-project (post) time periods, one could develop an energy savings calculation for each “specific product” or an “individual item” output from the process or facility.

Q: "MEASUREMENT&VERIFICATION-ISSUES AND EXAMPLES" 2019 Is the material up to date? If you have document about option examples, please give me that material!

A: The examples in the 2019 document referenced are the latest published examples.  There will be additional, updated examples that will be released in upcoming IPMVP application guides.

Q: I am from the UK and I am having difficulties in obtaining HDD and I wondered if there was a good source of information from this? The only information on the Government website seems to be for average HDD over the entire country and not for each relevant weather station.

A:  Yes, http://www.degreedays.net is a good source for airport weather stations in the UK. 

Q: Can you please confirm which IPMVP Option most closely aligns with the following approach for both single-ECM and multi-ECM projects?

A: Overall – an IPMVP Option approach consists of both a baseline method and a corresponding post-EEM (reporting period) measurement method.

Q: Perform Baseline Energy Metering - Energy meters are installed to the circuits feeding power to the addressed equipment/systems to isolate those systems from the rest of the building and capture pre-ECM(s) energy use. The baseline metering period for a given project is determined based on site local weather variations, the operating characteristics of the addressed equipment/system, and the proposed scope of work. These meters will remain installed during implementation of the ECMs.

A: [i]As stated, this Baseline approach seems to be focused on isolation of the retrofit/EEM with the installed meters. Given this information, this baseline approach for retrofit isolation with potential of all parameter measurement via the electric meters, would align with an Option B method of baseline determination.
[*][/i] [b][/b]

Q: Generate the Energy Baseline - The subhourly baseline period energy meter data in concert with coincident local weather data for the baseline period are used to derive the weather and time-of-week normalized baseline model for isolated systems at the building.

A: Same as above – appears to be stating retrofit isolation via energy meters – Option B – again, assuming the metering is capturing all parameters required for energy consumption and demand.

Q: Perform Post-ECM(s) Energy Metering - The energy meters that were installed for the baseline energy metering will be used to monitor and track the energy use of the addressed equipment/systems post-implementation of the ECMs. The post-ECM(s) energy metering period is one calendar year after the owner's acceptance of the implemented ECMs.

A: Same as above – retrofit isolation via meters which are capturing total retrofit energy use (all parameters) – used during reporting period for EEM savings determination – Option B.

Q: I'm a graduate student at Texas A&M University.
I'm wondering if there is any acceptable calibration tolerance in recent IPMVP publications (for example, MBE & CV-RMSE values on a monthly & hourly basis).
I think the ASHRAE G14-2014 & 2015 FEMP provided these criteria, but I should be back to the 2002 IPMVP to get this criteria.

A:  HI, you are correct. Recommend continuing to refer to ASHRAE, FEMP and other Industry publications regarding acceptable calibration tolerance guidance for different applications.