Researched by BEAST Library Curator | Verified by Harper | Quality: 8.0/10
Every AI-HVAC vendor pitch in 2026 cites "verified savings" and "IPMVP-aligned M&V." Almost none cite the two numbers that decide whether the verification actually held: CV(RMSE) and NMBE. Worse, the acceptable thresholds for those numbers vary by a factor of two depending on whose program manual you're reading — and the looser variant is what most vendor decks quietly assume.
If you are about to sign or renew an AI-HVAC, M&V-as-a-service, or performance-guarantee contract in the next 90 days, this is the page to read first. The same procurement-team M&V audit works against BrainBox, Trane Mosaic, Johnson Controls Nantum, 75F, Siemens Building X, or any startup pitching outcome-based pricing.
BLUF — The Two-Number Test
An AI-HVAC vendor claiming verified savings should be able to cite, on paper, three numeric floors against a stated IPMVP Option:
- CV(RMSE) of the baseline regression — the coefficient of variation of the root-mean-squared-error. Lower is tighter. ASHRAE Guideline 14 caps it at 30% for monthly data and 30% for hourly. Bonneville Power Administration's 2024 reference guide tightens this to 25% for hourly. If the vendor can't tell you their figure, the rest of the math is decoration.
- NMBE — Normalized Mean Bias Error. ASHRAE Guideline 14 caps at 10% monthly and 10% hourly. BPA tightens to 5%. NMBE close to zero means the regression isn't systematically over- or under-shooting.
- Documented NRA workflow — Non-Routine Adjustment procedure. Static factors (occupancy, schedule, weather extremes, COVID-style step changes) will happen during the performance period. EVO's own NRE-NRA white paper concedes there is no automated silver bullet for detecting and adjusting these. A vendor saying "fully automated M&V" without a documented human-judgment NRA workflow is overstating what the standard actually allows.
The Threshold Table — Three Floors, Same Math, Different Numbers
Here is the variance most procurement teams don't realize exists. Each row is the same statistical concept; each column is a published threshold from a real program or standard. The strictest column is what you should specify in your contract; the loosest is what most vendor decks implicitly assume.
| Statistic | ASHRAE Guideline 14-2014 (industry baseline) | BPA Regression for M&V Reference Guide (2024) | FEMP M&V Guidelines v4.0 (federal contracts) | Comment |
|---|---|---|---|---|
| CV(RMSE) — monthly | ≤ 15% | ≤ 15% | ≤ 15% | Consensus floor. If a vendor's monthly model is wider, the savings claim is statistically noise. |
| CV(RMSE) — hourly / sub-daily | ≤ 30% | ≤ 25% | ≤ 30% | Hourly model is where most AI-HVAC vendors live. BPA's tighter 25% floor is the one to specify if you're paying for high-resolution savings claims. |
| NMBE — monthly | ± 5% | ± 5% | ± 5% | Tight consensus. Wider NMBE means the baseline is systematically biased. |
| NMBE — hourly / sub-daily | ± 10% | ± 5% | ± 10% | BPA again the strictest. Specify the tighter floor for any hourly model claim. |
| R² (goodness of fit) | Not formally specified | ≥ 0.75 preferred (≥ 0.60 minimum) | ≥ 0.75 preferred | ASHRAE leaves a gap here. BPA closes it. If R² is below 0.6, the regression isn't really explaining the load — it's curve-fitting. |
Source roll-up: ASHRAE Guideline 14-2014 §5.2.11 and §B-1; Bonneville Power Administration "Regression for M&V Reference Guide" May 2024 §3.2; FEMP M&V Guidelines v4.0 §4.5.
The Hidden Third Number — Why NRA Is Where Savings Get Inflated
The two-number test catches sloppy regressions. It does not catch the most common mode of savings inflation: non-routine adjustments applied selectively after the baseline is locked.
Lawrence Berkeley National Laboratory's work on standardizing NRA approaches identified the structural bias: when a static factor changes (a tenant moves in, a chiller is rebuilt, a heat wave breaks a 10-year record), the vendor has discretion to either (a) adjust the baseline upward, which preserves the savings claim, or (b) adjust the post-installation period downward, which reduces it. Recurve's published analysis went further: across thousands of M&V projects, adjustments systematically biased toward inflating savings rather than deflating them. The asymmetry is structural — vendors are paid on savings, not on baseline accuracy.
EVO's own 2026 NRE-NRA white paper concedes: "No automated silver bullet exists. Project-level details and professional judgment are required." Any vendor claiming a black-box automated NRA workflow is, by the standard's own admission, claiming something the standard does not permit.
The 6-Question Vendor M&V Audit — Drop This Into Your Next RFP
Practitioner protocol: send these 6 questions to every vendor proposing outcome-based pricing or "verified savings." Treat refusal-to-answer or hand-waving as a no-bid.
- Which IPMVP Option are you using — A, B, C, or D? If they don't lead with this, the M&V plan doesn't exist. (Option C is whole-facility regression — most AI-HVAC vendors. Option D is calibrated simulation — increasingly common for digital-twin vendors; see our Option D piece.)
- What is your baseline model's CV(RMSE) and NMBE at the same data resolution (monthly or hourly) on which you are claiming savings? Both numbers, both stated, in the contract. Specify the BPA-tight thresholds (25% hourly CV(RMSE), 5% NMBE) for any sub-daily claim.
- What baseline period length are you using and why? 12 months minimum is the standard. Shorter baselines mean the regression hasn't seen a full weather cycle.
- Show me your documented NRA workflow. Who decides when an NRA is triggered? On what evidence? Who reviews? Vendor-only sign-off is a red flag; ASHRAE 14 expects independent judgment.
- Which independent variables are in the regression? Outside air temperature and a binary occupancy schedule are the floor. Missing humidity, missing schedule granularity, or missing internal-load proxies means the regression won't generalize.
- Will you commit to an annual third-party M&V audit by an EVO-CMVP certified professional? This is the single procurement clause that aligns vendor incentives with reality. CMVP audit cost is ~$8K-$25K per site per year. If the vendor refuses, you have your answer about the savings claim.
APAC Angle — Singapore Green Mark and the Global IPMVP Stack
Singapore's Building and Construction Authority Green Mark 2021 (current edition: GM:2021, second revision effective 1 June 2024) requires verified energy performance improvement against 2005 baseline as a mandatory prerequisite. The verification step is the final certification gate. While BCA does not require IPMVP-specific certification, the practical effect for any APAC-deployed AI-HVAC vendor is that the M&V plan needs to survive a public-sector audit lens — which is closer to the BPA-tight thresholds than the ASHRAE-loose ones. Hong Kong's BEAM Plus and Japan's CASBEE follow similar verification logic.
If you operate a portfolio that spans US incentive programs (NYSERDA, ConEd C&I, PG&E EE) and APAC certification (Green Mark, BEAM Plus, NABERS), specify the strictest threshold once and apply it everywhere. The cost of two M&V regimes is higher than the cost of meeting the stricter one universally.
What I'd Do This Quarter
If I were running M&V across a 5-building Class-A portfolio with one AI-HVAC pilot live and two more in procurement:
- Week 1: Pull every vendor's existing M&V claim. Compute CV(RMSE) and NMBE from their reported baseline data. Compare against BPA-tight thresholds. Flag any model failing R² ≥ 0.60.
- Week 2-4: Insert the 6-question audit into all open RFPs. Make CMVP third-party audit a contract clause, not a nice-to-have.
- Week 5-8: For the live pilot, document the NRA decision log. Who has signed off on which adjustments since baseline lock? Reconstruct the asymmetry test — were adjustments biased toward inflating savings?
- Week 9-12: Move your strongest baseline-vs-reporting case to Option D calibrated simulation if you have a digital twin in place. Option D is increasingly bankable for digital-twin-equipped portfolios and gives utility programs and ESG raters a more defensible savings figure.
For deeper background, see our IPMVP Verification reference page, M&V 2.0 playbook, and why AI-HVAC savings claims need a verification protocol.
Have a question about this topic? Ask our CRE AI Agent →