I’m doing some contract work assisting on some usability tests of a commercial recipe/cooking site. As we lead study participants through various parts of the site and ask them to complete various tasks, one thing we are attempting to capture is their expectations and desires for how the site should behave, the information they feel should be accessible, etc. In Usability/User-Centered Design/User Experience terms, we want to make sure the site information architecture and navigation matches the users mental model (the biggest design errors happen when there is a mismatch between users’ mental model and system model).
So we ask them questions like, “What do you expect to happen when you click that link, ” or “What do you think will happen when you move the mouse here, etc…” This is fairly typical line of questioning; I’ve used it on a number of similar studies.
But today I noticed something about how the users were responding to these questions which caused me to consider whether these questions are really returning valuable information. It seemed to me that users didn’t really have clearly defined expectations or opinions of what will or should happen when they click on a link. They wanted to just click the link to see what would happen. If the new page seemed to get them closer to their goal, great (Ed Chi or someone else at Xerox PARC would say, if the information scent grew stronger). If not, they were happy to go back to the previous page through whatever means were handy (back button normally, but they’d also use in-page navigation).
To use an economic analogy, it was as though clicking on web page links had become so cheap, easy, automatic, and reversible that people didn’t bother to invest the time, energy, and effort to decide what the link should do. It is cheaper to just click it, and if they didn’t like where they were headed, go back.
This may indicate that the process of asking people their expectations when browsing a website may not be a very valid way of testing site usability, because evaluating the expected or perceived value of a link may not something that people really do. It’s not the way that people use the web site. It may be better to find ways to capture user behavior, rather than their expectations (as has long been shown, what users say and what they do are often different) Do they follow the ‘shortest path’ to find a particular bit of information? What side paths did they take? There are lots of quantitative and qualitative usability/user experience metrics out there, but I suspect each particular project team will have to define the particular metrics of interest for their project.
I also wonder if users of a transactional type site, or users doing some type of significant task, might show different behavior. Say, for example, we’re talking about users of an internal company management application. They may make real expectation or value judgements on links as they go about company business. Who knows…
I wonder if there is some way to test this? Maybe evaluate usage of two types of sites — one business transactional, and one commercial informational. Somehow look at reaction times of deciding what to do next? If there is a difference, then I would expect to see longer reaction times on the business site than the commercial site.