I am seeking clarification regarding the evaluation of R² and CV(RMSE) in the M&V guidelines. Specifically, is the evaluation of these metrics intended to be performed on the full training dataset, or should it incorporate the testing dataset or cross-validation techniques?
In machine learning practice, there are well-established reasons to avoid relying solely on the full training dataset for model evaluation, as it can lead to overfitting and does not necessarily reflect the model's performance on unseen data. This approach contrasts with my current understanding of the IPVMP guidelines, which seem to imply an evaluation based solely on the training set.
Could you please provide guidance on this matter to align our model evaluation process with the best practices in the field? Thank you for your time and expertise.
In machine learning practice, there are well-established reasons to avoid relying solely on the full training dataset for model evaluation, as it can lead to overfitting and does not necessarily reflect the model's performance on unseen data. This approach contrasts with my current understanding of the IPVMP guidelines, which seem to imply an evaluation based solely on the training set.
Could you please provide guidance on this matter to align our model evaluation process with the best practices in the field? Thank you for your time and expertise.
Copyright ©
1994 - 2025
Efficiency Valuation Organization