Paul Marty and Michael Twidale’s article A conceptual framework for analyzing the usability flaws of museum web sites is very clearly written and pretty much delivers what its title promises.

It reports evaluations of 36 museum web sites (I’m guessing that most, if not all, were for US museums), on the basis of which usability issues common to the museums sector are identified. The evaluation approach is based in the sound principles of user scenarios, though the authors implicitly concede that their application of it might be termed ‘quick and dirty’. Whether or not you want to pick holes in the methodology, some of the results are certainly interesting, and at least plausible, not to say provocative.

The strength of the scenario-based approach is that it is based on taking the users’ perspective and understanding their context. In this sense it is consistent with the principles of ‘human-centred’ International Standards like ISO 13407. Marty and Twidale’s article reports that “usability evaluators spend ten minutes assessing a previously unknown interface and developing representative tasks” (my italics) which seems very little time to develop a true understanding of users’ perspective. Nevertheless the scenarios and tasks they conjured have some face validity. For example:

• “You are a teacher hoping to use this museum’s web site with your class. You have heard that it is possible to upload student works of art to this site, and you would like to find out how that is done so you can teach your students how to do it.”
• “Having just returned home from this museum’s physical installation, you are thinking about a certain painting by a certain artist that you saw in the museum’s galleries. You are visiting the museum’s web site to find out as much information as you can about that painting and that artist.”

Usability evaluators then observe one or two “representative users” enacting one or more of these scenarios using one of the museum web sites. Areas where the user gets stuck, makes an error, or expresses frustration are noted, and then these notes are analysed to reach conclusions.

The article lists five common “characteristics” of museum web sites that lead to usability problems, and between two and four “dimensions” of each of these. I won’t repeat these here: the article is the best place to read them. But here are some quotes I picked out that relate to some of the characteristics, together with my own comments and illustrations.

Users of museum web sites may be confused by the sheer number of choices presented to them, make wrong decisions, become frustrated, and find themselves unable to use the museum’s web site to complete simple tasks.

This is a growing problem, not just for museum web sites. Anything that has a significant archive has to address how best to provide the most usable access to that set of items. Frankly, the usability of this site — the one you’re reading now —, with its crude means of browsing my date, by category, or searching by word, is questionable. And that’s with only just over 150 items. (The software that runs the site provides limited options.) For a slightly different spin on this issue, see my posting on the usability of audio archives.

“This is very pretty,” our test users would often say, “but I don’t know what to do with it.”

Have a look at this page, which users arrive at with one click from the site home page, and see how long it takes to figure out how to get the most from it. (I feel a bit mean about this because the site was created by some prospective new friends, but I hope they’ll forgive me.)

Pressures to improve the accessibility of museum web sites should help address this problem (the prettiness of navigation is largely lost to screen-reader software, after all).

Museum web sites that encourage exploration often end up discouraging visitors who come to the web site with specific interests. These web sites need to support both guided and unguided exploration.

Consider the example of the second scenario given above: someone looking for all the details of a painting and its painter. The traditional solution to this would be a simple search facility, but support for guided vs. unguided exploration is not as simple as browse vs. search.

Museum web sites designed primarily by museum professionals… often require users to have “insider knowledge” about how museums work if they are to make full use of the online features provided. Users may need familiarity with organizational schemes used by museum professionals or special knowledge of specialized vocabulary used in museums.

A fairly self-explanatory point. The art of classification is remaining aware that other classifications are possible.

Also related in a very general way is my earlier posting on the usability of online content.