LarryRoth.net

When do you test?

When planning out a project schedule for a new or redesigned Web site, one common question is when do we run usability tests? I don't believe there is an one easy answer to that question.

My preferred answer is: If there is budget for it, test early and often! Setting up a test plan early on will allow you to not only incrementally test and tweak your designs, but if you are engaged in a site redesign you can also use the same repeatable testing procedures to get a baseline from the existing site. Not only will you be able to the new design is solid, but also you can show improvement over the existing site.

Testing could occur as follows:

• Baseline test of the existing site
• Low fidelity test of approved wireframes
• Test of graphic comps
• Test of completed site on staging server

Of course, the biggest concern is the cost—in time and money—of running so many test. Rather than give the overused response of how testing up front saves more money down the road, I prefer to point out that setting up the test and creating the report is the lion's share of the time required, not recruitment and subsequent execution. It's amazing how efficiently you can write up a report for an existing test the second time around.

Another issue is that clients will often be concerned that poor test results will affect the project schedule. Instead of jumping into a discussion about what is most important, I have found it helpful to let them know that the results of testing are usually minor tweaks that can have major impacts. It is usually the case that they feel poor results mean we will mean starting all over.

Despite the fact that I believe in testing early and often, at BrandLogic we have only every had one client that would allow us to engage in this level of testing. And it's interesting, they were ranked by an independent third party to be number 1 in their vertical market with regards to usability. hmmm.....