LarryRoth.net

Just my thoughts

What does the user think is a success?

When conducting a usability test, it is common practice to follow a procedure similar to this:

I have always found one of the toughest tasks to be defining what is a success and what is a failure—it's not always cut and dry. And this is defining success/failure from the test perspective. But I recently began to think it is even more difficult to track what is a success/failure from a user's perspective. Why would that be? Why wouldn't they be the same? Don't we ask users if they feel they have completed a task? Well, when we define a success, it is really limited to a task, but the user may see that task as a larger process. Let's take an example.

I recently booked a room at a national hotel chain using their on-line reservation system. When I arrived at the hotel, I found out that both the rate and the number of guests differed from my confirmation sheet. It turns out that their reservation system faxes the information to the hotel property and that information is then manually keyed in—leaving obvious room for error. In my case, the error was rectified politely and professionally, but it made me think about what makes a usable experience. It made me wonder about a successful task vs. a successful process and what the perception of the user may be.

In the example of a hotel reservation system, if I had been asked to conduct a usability study I would have focused on the reservation form(s) and made sure that users could complete the task of reserving a room for a specific property on a specific day. If they could, I would have thought of it as a success. And that may be fine for the purposes of the usability test. But stepping back and examining the experience from a users perspective, the task may not be a success until they are actually in their room!

I know what your are thinking. First off, there are a lot of "what ifs..." here, and secondly, it certainly isn't the goal of a usability test to identify such broadly scoped successes or failures. I agree with both of those statements, and I am certainly not proposing larger, more involved testing—unless someone wants to fly me around the country trying out reservations at 5 star hotels!

Seriously, my point is that as usability experts, it's our goal to remind the client that we are testing very granular tasks to a sometimes much larger process. While we may help to remove all the barriers a users encounters, the user will be judging the process as a whole, and there may be some hidden barriers that can't easily be tested. Thoughts?

blog comments powered by Disqus