LarryRoth.net

Just my thoughts

What does the user think is a success?

July 1, 2006

When conducting a usability test, it is common practice to follow a procedure similar to this:

  • Create user scenarios
  • Prepare tasks
  • Observe various users while executing these tasks
  • Mark successes and failures
  • Look for patterns and areas of concerns and provide feedback to the application owner
I have always found one of the toughest tasks to be defining what is a success and what is a failure—it's not always cut and dry. And this is defining success/failure from the test perspective. But I recently began to think it is even more difficult to track what is a success/failure from a user's perspective. Why would that be? Why wouldn't they be the same? Don't we ask users if they feel they have completed a task? Well, when we define a success, it is really limited to a task, but the user may see that task as a larger process. Let's take an example.

I recently booked a room at a national hotel chain using their on-line reservation system. When I arrived at the hotel, I found out that both the rate and the number of guests differed from my confirmation sheet. It turns out that their reservation system faxes the information to the hotel property and that information is then manually keyed in—leaving obvious room for error. In my case, the error was rectified politely and professionally, but it made me think about what makes a usable experience. It made me wonder about a successful task vs. a successful process and what the perception of the user may be.

In the example of a hotel reservation system, if I had been asked to conduct a usability study I would have focused on the reservation form(s) and made sure that users could complete the task of reserving a room for a specific property on a specific day. If they could, I would have thought of it as a success. And that may be fine for the purposes of the usability test. But stepping back and examining the experience from a users perspective, the task may not be a success until they are actually in their room!

I know what your are thinking. First off, there are a lot of "what ifs..." here, and secondly, it certainly isn't the goal of a usability test to identify such broadly scoped successes or failures. I agree with both of those statements, and I am certainly not proposing larger, more involved testing—unless someone wants to fly me around the country trying out reservations at 5 star hotels!

Seriously, my point is that as usability experts, it's our goal to remind the client that we are testing very granular tasks to a sometimes much larger process. While we may help to remove all the barriers a users encounters, the user will be judging the process as a whole, and there may be some hidden barriers that can't easily be tested. Thoughts?


What does the user think is a success?

A crash course in Software Engineering

June 28, 2006

During one episode of the Java Posse, the gang mentioned a new podcast called Software Engineering Radio. I had subscribed to the feed (via iTunes), but had not really had chance to listen until a recent trip.

After listening to 10 episodes, I must say it's a wonderful show. It is very informative, with a pace that is not at all slow, but instead moves you along quite quickly. The basic gist of the show is that they cover high level software engineering topics, providing enough information to give you a conversational knowledge of the subject—and perhaps help you decide if it's a topic you want to delve into further. They provide a good balance between high level information and more concrete examples. And they also provide resources for further information.

As an example, in a recent episode about Remoting they give a guideline for when to use messaging vs. RPC style calls. Here is a short excerpt:

...use a messaging system in case you have an event drive application, when you have state machines driving the application logic, and use an RPC style communication when you have to integrate the result into your regular control flow. [1]
It seems obvious. But for me it was a question that I had wondered about for some time—and they answered it quite concisely.

The show tries very hard to be language neutral and platform agnostic. It's a great show and I really recommend it to anyone who wants to dive deeper into software engineering.

meta-footnote-1=Episode 9: Remoting Pt.1 and Listener Feedback, http://media.libsyn.com/media/seradio/seradio-episode9-remoting_pt1.mp3, posted March 20, 2006, comment ~ at 12 minutes, 35 seconds
A crash course in Software Engineering

New features coming to Java SE

May 19, 2006

Ed Burnette has a concise and tantalizing summary of Graham Hamilton's presentation during the Sun general Technical session at JavaOne (2006). You can see a Web cast of the presentation via Sun's site. Two really interesting things popped out for me.

First, it looks as if Java 7 will have some important language changes. For instance, there will (perhaps) be direct support for XML. If you have ever accessed an XML file using Java, you know that there are many lines of code required just to open the file—let alone pull out a value.

Second, Project Semplice. Project Semplice is an attempt to allow Visual Basic programmers to use their VB skill in Java. From the article:

The goal is to enable VB developers to use Java platform. It's not for porting existing applications, it just lets you use your VB skills. Compile from VB source into Java classfiles, make VB source code calls into Java platform APIs, etc. It's a new language for the Java latform. It will support standard VB.net concepts.
This is exciting because it removes a barrier to entry for many programmers. In my opinion, there have been two major barriers for VB programmers to get into Java. The first has been a lack of solid IDE. There are a lot of IDE's for Java and have been from some time, but if you were interested in trying Java the price of the IDE's was somewhat prohibitive. Eclipse and Netbeans solved this problem. Both are excellent IDE's and both are free. The second problem was the complexity of the Java APIs. Java itself is quite a simple language, but compared to using an ActiveX control in VB or calling a DLL, the Java API's can seem quite complex and confusing to a newbie. Once you get past the learning curve you have quite a bit of power at your finger tips, but the curve is steep. Allowing someone to dip there toes by using code they are used to will help to at least entice them to the platform.

In our shop, I watched my team make the transition from ASP to .Net. It was interesting because instead of sticking with Visual Basic (the core ASP language), our developers quickly switched to C#. A familiarity with the development environment and the Framework helped them get over the hurdle of learning a new language, and once they did, they never looked back.

I can't wait to try it out. It will be fun to dust off my old VB knowledge.


New features coming to Java SE

Checking out the neighbors

April 28, 2006

Space.com has a good article on catching Mars, Jupiter, and Saturn in the evening sky. You don't need a telescope, they are all quite visible with the naked eye and you can even see a little detail with binoculars.

However, I think checking out a plant through a telescope—even an inexpensive one—is quite an experienence. I remember the first time I saw Jupiter with two moons lined up on each side. The minute I could recognize the banding, it all sank in. I am looking at another world! It was enough to keep me outside in 5 degree weather for several hours.

If you are thinking astronomy as a hobby, this is a great time to start, and planets are great objects to start with.


Checking out the neighbors