Two questions, one context
Capability fit: how well does the selected offering meet this organization’s declared requirements? Operating fit: how well can that organization deploy, staff, integrate, maintain, and afford it?
Each edition fixes the segment, use case, organization profile, product edition, workload, critical requirements, rubric version, publication date, and refresh date. A result belongs to that context. It is not a permanent label on a vendor.
| Both axes at threshold 70 | Interpretation |
|---|---|
| High capability · High operating | Strong fit: investigate a candidate and verify critical requirements. |
| High capability · Lower operating | Operating demands: resolve effort, staffing, or constraints. |
| Lower capability · High operating | Capability gaps: determine whether missing requirements are acceptable. |
| Lower capability · Lower operating | Limited fit for this profile: revisit the approach. |
A published rubric
Ratings use 0–5 anchors: 0 means verified absence or failure; 1 major gaps; 2 partial fit; 3 meets the declared requirement; 4 exceeds it in a useful observed way; 5 substantially exceeds it in the scenario. Segment-specific anchors must be published before assessment. Unknown is null.
| Axis | Criterion | Weight |
|---|---|---|
| capability | Required functions | 40% |
| capability | Representative effectiveness | 30% |
| capability | Evidence and explainability | 15% |
| capability | Required data exchange | 15% |
| operating | Staffing and administration | 30% |
| operating | Environment and deployment | 25% |
| operating | Integration maintenance | 20% |
| operating | Portability and exit | 15% |
| operating | Cost predictability | 10% |
Capability integration asks whether an exchange works. Operating integration asks what it takes to maintain. Keep those observations distinct. Cost evidence records dates, licensing units, workload, and exclusions; negotiated prices that are unknown remain unknown.
Unknowns remain visible
Let W be total applicable weight, K known-evidence weight, and S the sum of weight × rating for known criteria.
Observed score = 100 × S / (5 × K), when K > 0
Evidence coverage = K / W
Possible interval = [100 × S / (5 × W),
100 × (S + 5 × (W − K)) / (5 × W)]For W=100, K=80, and S=300: the observed score is 75, coverage is 80%, and the possible interval is 60–80. Because that interval crosses 70, a definite higher/lower fit label would be misleading. The interval concerns missing evidence, not statistical confidence.
Both axes need at least 80% coverage, and all critical requirements need explicit results before a point is publishable. A verified critical failure blocks a shortlist recommendation. Criteria can be non-applicable only with a rationale applied consistently to the whole comparison profile.
Completeness is not quality. Display vendor documentation, reproducible tests, independent evidence, and unverified statements separately. Documentation alone cannot establish measured effectiveness. Reviewer disagreement and sensitivity to weights need their own notes.
A comparison readers can challenge
- Our axes, weights, ratings, and coordinates are independently developed. Third-party analyst positions are not scoring inputs.
- Products and editions are selected through published inclusion rules. No universal “best vendor” badge.
- Sponsorship, partnerships, and affiliate relationships cannot alter inclusion, scoring, boundaries, or publication timing.
- Factual corrections require evidence, regardless of who submits them. The editor owns the conclusion; vendors do not approve it.
- Every scored edition needs a reviewer other than the scorer, archived input data, dates, and change history.
Launch status: three research protocols and a synthetic scoring demonstration. Actual product profiles remain unassessed until the required independent observations and reviews are complete.