Most CRE AI vendors sell you a dashboard and walk away. The seven questions below tell you — in about ten minutes — whether a vendor has real skin in your building's outcomes, or is renting you a black box with a login. Score your own stack against it for free. Or ask our agents to run it on your actual vendor list and reply within 48 hours.
This is not a marketing checklist. It is the diligence frame an operator uses before signing anything, built from the one discipline most proptech contracts quietly avoid: measurement & verification you can defend to a lender, an auditor, or a tenant.
Why "skin in the game" is the only question that scales
The AI capability is commoditizing. In the last twelve months the major BMS platforms pushed generative diagnostics into the mid-market, and at least one incumbent acquired its way to an analytics layer rather than building trust into it. When the model is a commodity, the durable differentiator is no longer "which AI?" — it is whether the vendor's incentives are tied to your verified result. A vendor with skin in the game prices on outcomes you can audit. A vendor without it prices on seats and hopes you never run the numbers.
The 7-question scorecard
Give one point for each green answer. The scoring bands are below the table.
| # | Ask the vendor… | Operator answer (1 pt) | Black-box answer (0 pt) |
|---|---|---|---|
| 1 | M&V method: "Which IPMVP option do you measure savings under, and who sets the baseline?" | Names IPMVP Option C (or B) and co-sets a documented baseline with you | "Our dashboard shows the savings" — no protocol named |
| 2 | Pay-for-performance: "Does any part of your fee move with verified savings?" | Yes — shared-savings or a performance hold-back tied to M&V | Flat SaaS seat fee regardless of outcome |
| 3 | Open vs. locked: "Do you read other vendors' systems, or only your own hardware?" | Cross-vendor / open-protocol (BACnet, Haystack) — reads the whole stack | Single-vendor island; "works best if you replace X" |
| 4 | Audit trail: "Can I export a time-stamped log of every recommendation and action, day one?" | Yes — immutable, exportable, owner-readable | "That's on the roadmap" / screenshots only |
| 5 | Exit ownership: "On termination, who owns the model, the tuning, and the historical data?" | You do — full data + config export, no ransom | Data and model stay with the vendor |
| 6 | Recommissioning: "How do you catch baseline drift and re-tune over time?" | Scheduled recommissioning + drift detection in the contract | Set-and-forget; savings "decay" unexplained after year one |
| 7 | Accountable human: "Who is the named person on the hook for my SLA, and what's the response window?" | Named accountable owner + written SLA | "Submit a ticket" / rotating support queue |
Scoring
- 7 / 7 — Operator. Their incentives are aligned with your result. Negotiate scope, not trust.
- 4–6 — Order-taker with upside. Good tooling, weak accountability. Push questions 1, 2, and 5 into the contract before signing.
- 0–3 — A black box you rent. You are paying for software and absorbing all the outcome risk yourself. Re-scope or walk.
What "verified" looks like in dollars
The scorecard rewards vendors who let you measure. For reference, here are publicly reported, M&V-style outcome bands an operator-grade engagement should be comfortable being held to. These are benchmark-grade reference points, not a simulation of your building — your capture depends on baseline, climate, and asset mix.
| Reference outcome | Reported band | Why it's relevant |
|---|---|---|
| Whole-facility HVAC optimization (Carrier Abound reference) | ~28% energy reduction / ~$1.8M reported savings | Real, attributable savings band — the kind a shared-savings vendor will stand behind |
| Incentive stack (NY smart-building grant program) | $15M pooled grant funding + utility rebates | External dollars that change the payback math — ask a vendor to size the stack for you |
| Consensus CRE AI deployment value | ~14% operating-cost impact (PwC / JLL / Schneider consensus) | The honest middle of the market — anyone promising 40% with no M&V is selling, not measuring |
Energy and savings figures should be expressed under IPMVP Option C (whole-facility) discipline; treat any vendor number without a stated option or baseline as marketing, not measurement.
Run it on your stack — free
This scorecard is free to use and free to share. If you want it run against your actual vendor list — including the awkward follow-up questions most owners don't think to ask — our agents will do it and reply within 48 hours, at no cost. We are a global CRE platform built by an operator inside a Fortune 10 portfolio; the point of putting 100+ agents in the open is to get this discipline out from behind $50K subscriptions.
→ Ask our agents to score your vendor stack (free, 48-hour reply)
Related reading: The proof — third-party verified ROI · Why owner-operator-first beats horizontal AI · Build vs. buy: the mid-market owner's decision frame