Estimated reading time : 5 min · Published
Write the decision before the criteria
Start with the task: publishing in two languages, managing a catalogue, handling appointments or taking over an existing site. Record volume, actual operators, ongoing budget, accessibility requirements and data needed if the tool is replaced. Without this context, an overall score hides the trade-offs.
Remove candidates that cannot meet a genuine essential requirement, while separating that requirement from a preference. The CMS selection guide proposes a shared task for the people who will actually publish.
- Task and audience stated.
- Blocking requirements separated from preferences.
- Operating and exit costs included.
Use the same trial for every option
Give shortlisted tools the same instructions, starting content and comparable trial time. For a CMS, publish, revise, translate and export a page. For booking software, follow a full appointment and a failure. Observe the work the team will really do, not only the quality of a sales demonstration.
Keep the date, tested version or plan, method and reviewable evidence for each result. A feature mentioned on a marketing page is not necessarily available in the chosen plan or region. Mark items that were not tested.
- Same scenario across candidates.
- Version, plan and date recorded.
- Observed result separated from supplier claims.
Show the trade-offs
A table can bring together five or six decisive criteria: editing effort, rendered accessibility, export, support, needed integrations and total cost. Explain how each was judged. A numerical score without a method gives false precision.
State the strengths and limits of every option, including the recommended one. Disclose an affiliate link, commercial relationship or your own offer if it is involved. The recommendation should follow the use case and the trials, not a product’s position in the table.
Fictional exercise: anonymous solutions A and B must export files and their metadata. This requirement is declared essential before testing. A does not return the files; B passes in the tested scenario. Two preferences are then illustrated on a scale defined for the exercise, from 0 “does not meet the need” to 4 “fully meets it”: editing, weight 3, and support, weight 1. A receives 4 and 4; B receives 3 and 2. These values illustrate reasoning, not an assessment of real products.
Adding the preferences gives A 16/16 and B 11/16. Yet A fails the export prerequisite: its higher total is not enough to select it. Keep the essential result separate from the score, with the test, date and evidence. If a point is unknown, mark “not tested” and arrange the test; do not turn missing evidence into success or failure. For options still eligible, changing weights helps reveal whether the choice depends strongly on the chosen preferences.
- Method beside the result.
- Recommended option’s limits included.
- Commercial interests visible.
Scroll the table horizontally to read every column. With a keyboard, focus the table area and use the arrow keys.
| Exercise criterion | Rule set before testing | Fictional solution A | Fictional solution B |
|---|---|---|---|
| Export files and metadata | Essential, outside the score | Failed: files missing | Passed in the tested scenario |
| Editing | Preference, weight 3 | 4 × 3 = 12 | 3 × 3 = 9 |
| Support | Preference, weight 1 | 4 × 1 = 4 | 2 × 1 = 2 |
| Preference total | Does not replace the prerequisite | 16/16; prerequisite unmet | 11/16; prerequisite met |
Keep the comparison useful
Add a review date and identify criteria that change quickly: price, plan limits, interfaces, export and integrations. When a fact changes, correct it with a proportionate note. If the scope differs substantially, repeat the trial rather than silently replacing a value.
Finish with guidance for a few real situations instead of a universal winner. Ask readers to verify the current offer and try the critical journey with their own content. The source-checking guide helps maintain the claims register.
- Review date and owner.
- Links to sources and trials retained.
- Conditional conclusion instead of absolute ranking.
Decide in a concrete situation
Fictional example: one offer includes hosting and support while another covers creation only. Align scope, period and responsibilities before comparing totals; a lower price may describe a different service.
Use one row per function or expense, marked included, excluded or unknown. Add maintenance, exit and data transfer; an overall score without verifiable criteria hides differences.
- Calculate a website’s total cost over several years
Compare initial delivery, running costs, content, renewals and exit costs using a documented website budget.
Compare a banking cost
Two offers advertise a price but include different services. Compare the cost of your usage rather than the displayed amount alone.
List recurring fees, occasional transactions and conditions. Keep dated documents and flag unknown costs; an incomplete total cannot settle the choice.
- List included services
- Separate fixed and variable fees
- Keep dated conditions
- Compare a banking cost
Proposed practical case
Compare a refurbished purchase
Similar devices are offered by different sellers. Condition, accessories and terms may explain differences between the offers.
Create a record for each device with declared defects, seller and documents. Check applicable arrangements with the relevant source for your country.
- Identify the seller
- Record condition and accessories
- Read stated returns and warranty
- Compare a refurbished purchase
Proposed practical case
Reference documents
- W3C WAI : Selecting Evaluation Tools (a specialist example) · Source consulted
- Scottish Government : Multi-Criteria Analysis Guidance · Source consulted
Content updated on