How to evaluate a data visualisation tool: a buyer's checklist for UK teams

Most visualisation tools survive a demo. Twenty minutes of clean sample data, a filter that responds instantly, a chart that animates as it loads. The trouble arrives in week three, when someone asks why the North West figures disagree with the finance system, or when a colleague using a screen reader tries to read the same chart you were so pleased with.

None of that comes up in a sales call. What follows is the checklist worth working through before you commit budget and, more importantly, your team's time.

Start with the connectors, not the charts

Every tool looks similar once the data is in. The difference is how it gets there, and how much effort it takes to keep it arriving.

List the places your data actually lives before you look at a single interface: a cloud warehouse, a transactional database, a finance package, and the shared drive full of spreadsheets that someone updates by hand each Monday. Then ask the vendor to connect to the awkward one, live, during the trial. Demo environments are built to flatter the tool.

Pay attention to refresh behaviour. A direct connection to a slow source can make a dashboard feel broken at nine in the morning when forty people open it at once. An extract solves that but introduces staleness, and you need to decide how stale is acceptable in your context. If the tool refreshes against a warehouse that bills by data scanned, an inefficient query can quietly add to the monthly bill. Ask whether refreshes are incremental or rebuild everything from scratch.

Connector questions worth putting in writing

• Which of our sources can you connect to today, without custom development?
• Do joins happen in the tool or upstream in the warehouse?
• How are dates, time zones and currencies handled when sources disagree?
• What happens to existing dashboards when a source column is renamed?
• Does the connection use a service account, and who manages the credentials?

Treat roadmap promises as promises, not features. If a connector ships next quarter, plan as though it will not.

Collaboration, permissions and the person who leaves

Visualisation is rarely a solo activity. Someone builds, someone reviews, someone publishes, and a much larger group consumes. Ask how those stages are separated. Can a draft be worked on without appearing on everyone's homepage? Can a chart be certified as trustworthy, so a new joiner knows which version of monthly revenue to use?

Permissions matter as much as charts. Row-level security that follows the warehouse is easier to maintain than a second set of rules defined inside the reporting tool, which will drift out of step within a year. Check whether the audit log tells you who changed a metric definition and when. That is useful during the inevitable argument about why a number moved.

Then think about handover. If the analyst who built the sales dashboard resigns, can a capable colleague edit it without a training course? Tools that need expert knowledge to maintain are a long-term cost, however reasonable the licence looks at the start.

Accessibility: test it rather than assume it

Accessibility is a procurement requirement, not a finishing touch. UK public sector bodies have legal duties around the accessibility of their online services, and the Equality Act places broader obligations on organisations to make reasonable adjustments. Even where neither applies directly to your team, a dashboard a keyboard user cannot operate is a dashboard that excludes colleagues and customers.

Conformance reports are a starting point, not proof. Test in the trial:

• Complete a core task using only a keyboard, with no mouse.
• Check colour contrast, and whether any meaning depends on colour alone. Red and green status indicators are the usual offender.
• Ask someone who uses a screen reader to try one chart, and ask the vendor how data is exposed to assistive technology and whether a table alternative exists.
• Zoom to 200 per cent and note what breaks.
• Export a chart to PDF and check the result is readable and tagged.

Small details count: sensible default font sizes, no text baked into images, and dashboards that still work on a tablet. If a vendor cannot answer accessibility questions clearly, that tells you something about the roadmap.

Total cost of ownership: the lines that never make the quote

Licence cost is the visible part. The rest accumulates quietly.

• Viewer versus creator licences. Pricing usually splits these. Count your real audience, not your pilot group — a tool that is cheap for ten analysts can be expensive for four hundred readers.
• Migration. Count the dashboards you would rebuild and estimate honestly. This is normally the largest single cost.
• Compute. More dashboards usually mean more warehouse queries, and that bill belongs to someone.
• Admin and training. Who manages users, licences and gateways? What does a new starter need?
• Non-production environments. Some vendors charge for them; some do not include them at all.
• Renewal terms. Annual uplift clauses, minimum seat commitments, and what happens if you shrink the team.
• Exit. Can you export dashboards, metric definitions and data in a usable format? What would leaving in three years cost in time?

Build a three-year model with those lines and compare it against the do-nothing option and a plain spreadsheet-plus-extract approach. For any significant budget commitment, run the figures past your finance team before signing.

A two-week trial that tells you something

Demo licences tend to be evaluated by the wrong people on the wrong data. Run it deliberately:

1. Use your real, messy data, including the one source nobody wants to talk about.
2. Pick three tasks someone will genuinely do: a weekly report, an ad-hoc question, an alert that fires when a threshold is crossed.
3. Time how long each takes, and count the steps.
4. Include a sceptic and someone who uses assistive technology in the test group.
5. Test at realistic volume — month-end data, not a tidy sample of two thousand rows.
6. Break something on purpose: revoke a credential, rename a column, add a duplicate record. Watch what happens and how you find out.
7. Raise a support ticket and judge the quality of the answer, not just the speed.

Making the call

Score the tools against your own criteria, and weight those criteria before the trials begin while you are still impartial. Connectors and total cost usually deserve the heaviest weighting. Interface polish deserves the least, because every tool in this market looks good on the first day.

Then start small. Pick one team, one recurring decision and one set of data that matters, and run it for a quarter. If it holds up, expand. If it does not, you have learned something useful at a fraction of the cost of an organisation-wide rollout. And keep the exit plan in the drawer — the best negotiating position at renewal is knowing you could leave.

Photo: Firmbee / Pixabay