Posted by | January 04, 2011

Four Steps to Great Usability Testing (Without Breaking the Bank)

I’m involved in something called the Product Design Guild in San Francisco. It’s pretty cool. We talk about ideas and do guerrilla usability testing of concepts. There’s a really smart group of people there, and they know great products only come out of testing an idea over and over again.

At one of the events, I’m “training” developers and product types in usability testing for a number of startups. The document I wrote I think is good enough to share and explains the basics of conducting usability tests. I’m convinced anyone can be trained to do it or at least write the test. Usability tests don’t have to be really fancy — only digital agencies looking to bill or drain the retainer would do eye-tracking on a half-baked idea — and the feedback you get is incredible.

Here’s the document for download in Word format.

There’s nothing worse that running a usability test and the user (read: idiot) using the mouse gushes about what they like, and they still can’t figure out where the login button is. The best suggestions I’ve gotten have been from users that provide more negative feedback (“You know, this is done better on this other site”) than constantly good feedback.

Step 1: Write The Usability Test (With Tasks that Actually Work)

  • Create five tasks you want the user to perform.
    The task should require at least six to nine clicks to complete the task. The tasks could be signing into the system or uploading photos and setting permissions.
  • Write scenarios for each task.
    You should explain the task without telling the user what to do. An example would be, “You were at a family picnic where you took photos of your nephews and nieces. You want to share those photos with your family on the Internet on Flickr. Where would you go to sign in, upload those photos, and set the preferences so only your family can view them?”
  • List follow-up questions you might want to add.
    The magic of usability tests are the answers you get outside of the scenarios. So you might want to come up with additional questions within the screens you are testing.

Step 2: Find Participants of All Sizes (Sort of)

  • Find people that fit your target audience.
    Very few sites have a target audience of “everyone.” Ideally, you would test people that would use your site. In the example above, you want to use people that liked sharing photos with their family and friends.
  • Find people with a range of experience.
    If you find five professional photographers that share photos constantly, you’ll get a narrow view of how your site is performing. It’s best to find users that could be both expert users (a professional photographer) and users than might not have as much experience (an uncle that just bought a camera and uses Facebook sporadically).
  • Find people that will HATE your product.
    There’s nothing worse that running a usability test and the user (read: idiot) using the mouse gushes about what they like, and they still can’t figure out where the login button is. The best suggestions I’ve gotten have been from users that provide more negative feedback (“You know, this is done better on this other site”) than constantly good feedback.
Use words that explain what they need to do, but are not contained within the site. Say “sign in” instead of “login.” Use “find files” on the hard drive instead of “browse.” Remember that you are testing the terminology the site uses as much as the design of the site.

Step 3: Perform the Usability Test (In 60 Minutes or Less)

  • Do a dry run.
    There’s nothing worse than finding out on the first test that your script doesn’t work. Before you jump into the testing, you should run through the test before performing it in front of test subjects. Be sure to work out any software or site issues before doing it in front of users.
  • Vary the tasks and keep the tests in the 15-30 minute range.
    For the purposes of most applications, you’ll be testing three of five tasks with each user. The longer the test, the more often the user will get bored with doing the test. Keep the tests short and snappy.
  • Do not guide the user.
    Use words that explain what they need to do, but are not contained within the site. Say “sign in” instead of “login.” Use “find files” on the hard drive instead of “browse.” Remember that you are testing the terminology the site uses as much as the design of the site.
  • Ask questions.
    The beauty of usability testing is that you can ask questions that are also outside of the test. You don’t necessarily know where the user is going to go. Instead of simple yes/no questions, ask open-ended questions, like “What do you think this would do?”
  • Let them talk.
    People are amazing — once they start talking. They’ll tell you all sorts of things. I’ve had participants tell me their complete business process, including profit margins, during tests. The best feedback is sometimes outside of the what I was asking.
  • Record the sessions with some kind of screen recorder and take notes.
    I’ve used everything from Camstudio to WebEx to record results, but sometimes the best way to take notes is simply pen and paper. Don’t get hung up on the tools. Eye tracking software is ideal but also expensive. You can get close to the same results just by recording what the users do with screen capture software.
  • Record the results on a range of pass/fail.
    In the past, I’ve used the pass/fail method on a task but found on a scale of 0 (they passed) to 3 (they couldn’t find anything) was a better approach to grading tasks on a step by step basis and in aggregate.

Step 4: Evaluating the Results (By Reading the Tea Leaves)

  • Look for patterns.
    A focus group of one is not a good thing — too many usability professionals and product managers do this. I tend to discount those as preferences for that particular user. If two or three users say close to the same thing (“The button is hard to find.”), there’s a usability issue you may want to look at closer.
  • Highlight comments that can make your site better.
    The best feedback from testing is something that you didn’t expect. Highlight some of the comments that might be outside of the tasks but make great sound bites for describing issues with your site.
  • Remember results are subjective, so discuss them with the team.
    Put a team of three usability professionals together, and sometimes they’ll come to three different conclusions about the test. Ideally, you could combine qualitative results with quantitive results (i.e. stats from watching the conversion funnel).
  • Combine testing with other data collection methods.
    I sometimes use a site called Attention Wizard that calls itself a visual attention prediction tool to see where users might click based on contrast of color values. If you combine testing with other validation methods, you’ll get better results.
  • Test your assumptions again.
    Testing is not a one-time thing. You should test as much as possible (I’ve tested as much as every two to three weeks in an engagement), because the more you test, the more you get to refine your product. I’ve had a couple of clients where we tested over 15 users with different tasks, creating a great amount of data that helped in product design.

Extra Credit: Other Resources

Other Blogs

Free Software

Posted by | November 29, 2010

Silicon.com: Expedia On How One Extra Data Field Can Cost $12 Million

So what was the problem?

Expedia analysts realised the site needed to be changed after investigating why many customers who clicked the ‘Buy Now’ button on the company’s site did not complete the transaction.

“This is someone who was on our site, found the right location and hotel, put in all their billing and travel information and clicked the ‘Buy Now’ button,” Megibow said.

“As far as leading indicators of purchase intent go, this is as good as it gets and yet we weren’t taking the money.”

Analysts began examining and correlating information about these failed transactions to identify what traits they had in common.

The answer, it turns out, was quite simple: “We had an optional field on the site under ‘Name’, which was ‘Company’,” Megibow said.

“It confused some customers who filled out the ‘Company’ field with their bank name.”

What did they try?

“After we realised that we just went onto the site and deleted that field – overnight there was a step function [change], resulting in $12m of profit a year, simply by deleting a field.

“We have found 50 or 60 of these kinds of things by using analytics and paying attention to the customer.”

I love data.

Posted by | November 19, 2010

Kicker Studio: Why You Want (But Won’t Like) A Minority Report-style Interface

From Kicker Studio: Big, huge gestural interfaces aren’t all they are cracked up to be:

Oh sure, they look really cool, and certainly companies are doggedly trying to create them. Bloggers and tech geeks love them. (Any time you put Minority Report into a title, you’re bound to get traffic.) But I guarantee you very few of those people have actually used one for any length of time. If not done well (and sometimes ever if they are), they are exhausting. Human beings aren’t meant to hold their arms out in front of their bodies making gestures for long periods of time. It creates a condition called Gorilla Arm (aching muscles, stiffness, a swollen feeling) because it violates basic human ergonomics. Tom Cruise (who, let’s be honest, is in better shape than 95% of us) was reportedly tired from just acting out the scenes in the movie.

I loved the Surface when it came out, but the first question was — how do I use it?

It makes sense in a mobile device to have a lot of gestural interfaces, or the iPad for example. But in a bigger stage, waving your hands around just doesn’t seem like fun. Like Dan Saffer pointed out in the article, it’s the places where a subtle gestural interaction can be used it makes sense.

Posted by | November 10, 2010

Elezea: How To Stay Sane As A Product Manager

About design:

f you haven’t seen the Oatmeal comic How A Web Design Goes Straight To Hell, you need to go read that first, and then come back here — it’s a perfect summary of the problem. The ultimate post on this problem has also already been written, so I’m not going to spend too much time on it — just go read every word in Why Design by Committee Must Die in Smashing Magazine. I do want to highlight a couple of areas in that article, and add some of my own comments.

One of the main problems we have in web design today is that everyone thinks they’re a designer. With coding it’s different — not everyone can code. But design is different. Like art, everyone has an opinion on design. You like it or you don’t. And because you have this immediate visceral reaction to a design, it’s tempting to confuse that with knowing what makes a design good. But that’s simply not true.

As posts like Designing for the Mind and  Gestalt Principles Applied to Design have shown, what makes a design “good” has very little to do with taste, and everything to do with the proven psychology of visual perception. “Pretty” is a small part of design compared to applying the principles of solid user experience design to an interface. So please, let’s leave design to the people who are trained in this stuff. Have I mentioned the importance of trust?

Posted by | November 05, 2010

Web Designer Depot: Usability Resources to Win Arguments

Great article.

Usability Findings:

  • Form labels work best above the field
  • Users focus on faces
  • Quality of design is an indicator of credibility
  • Most users do know how to scroll
  • Blue is the best color for links
  • The ideal search box is 27 characters wide
  • White space improves comprehension
  • Effective user testing doesn’t have to be extensive
  • Informative product pages stand out
  • Most users are blind to advertising
Posted by | October 31, 2010

Kicker Studio: Design Titles and Levels of Experience

Great Article.

Junior designers are those with some training and (hopefully) some talent. They’ve gone to design school or done a handful of professional projects. They can create deliverables, although not without supervision. They can often explain design decisions, and can handle problems of moderate complexity. Junior designers should always be paired with a more senior designer.

Senior designers are those with the magic 10,000 hours of experience that make one a master. For those who are math challenged, that’s 250 40-hour work weeks, or over five years being a designer. In my experience, this is more like seven to 10 years of being a working designer to reach this level of mastery, where one is able to tackle complex problems in a variety of domains.

I would love to see this standard applied — there are many people that have a false idea of where they are in the industry (including myself ;) ). Read the complete article to see Dan Saffer’s thoughts.

Posted by | October 27, 2010

UXMovement: Applying the Golden Ratio to Web Layouts and Objects

From UXMovement:

1.618 is a number all serious designers should know. It’s known as the golden ratio found through­out nature, art and architecture. Seashells, the Mona Lisa and the Parthenon all show the golden ratio. Our faces and bodies are also proportional to the golden ratio. It’s so omnipresent that it’s even found in sounds and intervals of time. If there ever was a mathematical way to explain and express natural beauty, the golden ratio is it.

The golden ratio gives us a divine sense of aesthetic that we can apply to web design. By applying the golden ratio properly, we can make our web sites visually compelling. The most common way to use the golden ratio is with layout. Using the golden ratio we can divide the width of our web site to create a “golden layout” of two columns.

For example, take your typical 960 pixel width layout and divide 960 by 1.618. You’ll get about 594, which is the width of your main content column. To get the width of your sidebar, subtract 594 from 960 and you get 366. Put it all together and you have a two column layout with widths of the golden ratio.

There’s another completely different reason I also go with this structure — studies have shown that sites are easier to read when line length is limited.

The length? Between 550 and 600 pixels wide.

Posted by | October 21, 2010

Four Essential Elements For Creating Captivating Websites

Guillermo Cedillo is a UX designer that works for Sieena, a development firm in Mexico. This is a great article that talks about design. The article originally appeared on Website Magazine, but we have permission to republish it. Cheers.

The four principles of design are balance, rhythm, emphasis and unity. Each one of them is essential for bringing together the different visual elements that are necessary to achieving a strong design, which, in turn, is imperative for a website to succeed on any level. What follows is an examination of each principle, with insights about how to incorporate them into your own Web design for optimal results.

Balance

Different colors, shapes and sizes can create different degrees of what is called “visual interest” on a Web page. It is important that pages are designed to hold a user’s interest without overwhelming them or causing distraction away from the elements most important to conversion goals. As such, distribution of this interest needs to be controlled and balanced by considering each element in a layout and its “visual weight” — determined by its size, shade and thickness of lines.

Symmetrical balance is achieved by placing elements in the design evenly. If you place a large, heavy element on the right side, you will have a matching heavy element on the left. Centering is the easiest way to get a symmetrically balanced page. But be careful, as it can be difficult to create a centered design that doesn’t look flat. For symmetrically balanced design, it is better to create the balance with different elements — an image on the left and a large block of text to the right of it, for example.

Asymmetrical balance is an arrangement of unlike objects of equal weight on each side of the page. Color, value, size, shape and texture can be used as balancing elements. However, asymmetrically balanced pages can be more challenging to design, as elements are not matched across the centerline of the design.

For example, you might have a large element placed very close to the centerline of the design. To balance it asymmetrically, you could place a small element farther away from the centerline. If you think of your design as being on a teeter-totter or seesaw, a lighter element can balance a heavier one by being further away from the center of gravity. You can also use color or texture to balance an asymmetrical design.

Sometimes the purpose of the website itself makes an off-balance design the right choice. Designs that are off-balance suggest motion and action. They make people uncomfortable or uneasy. If the content of your design is also intended to be uncomfortable or make people think, a discordantly balanced design can work well.

Rhythm

Rhythm in design is also known as repetition — a pattern created by repeating elements that are varied, allowing your designs to develop an internal consistency that makes it easier for your customers to understand. Once the brain recognizes the pattern in the rhythm it can relax and understand the whole design.

Repetition (repeating similar elements in a consistent manner) and variation (a change in the form, size or position of the elements) are the keys to visual rhythm. Placing elements in a layout at regular intervals creates a smooth, even rhythm and calm, relaxing mood. Sudden changes in the size and spacing of elements creates a fast, lively rhythm and an exciting mood.

Gestalt is a general description for concepts that make unity and variety possible in design. The mind has the ability to see unified “wholes” from the sum of complex visual parts. Some principles of gestalt are proximity, similarity, continuance, closure, uniform connectedness and 1+1=3 effects.

Emphasis

Emphasis (or dominance) in design provides the focal point for the piece, enabling the most important design element to stand out. To draw the reader to the important part of the piece, every layout needs a focal point.

Generally, a focal point is created when one element is different from the rest. However, to maximize emphasis, it is necessary to avoid too many focal points, so as not to dilute the dominant effect. When all elements are given equal emphasis, it can make the piece appear busy, at best, or even boring and unappealing.

Emphasis can be achieved in the following ways:

Using semantic markup to provide some emphasis, even without styles.

  • Changing the size of fonts or images to emphasize or de-emphasize them in the design.
  • Using bold, black type for headings and subheads and much lighter text for all other content. Placing a large picture next to a small bit of text.
  • Using contrasting colors. For example, using a series of evenly spaced, square photographs next to an outlined photograph with an unusual shape.
  • Placing an important piece of text on a curve or an angle while keeping all of the other type in straight columns.
  • Using colored type or an unusual font for the most important information.

Unity

Unity (or proximity) helps all the elements look like they belong together. Readers need visual cues to let them know an article is one unit — the text, headline, photographs, graphic images and captions all go together. Elements that are positioned close to one another are related while elements that are farther apart are less so.

Unity can be accomplished through the following methods:

  • Being consistent with the type font, sizes and styles for headings, subheads, captions, headers and footers throughout the website.
  • Positioning elements so that those close to one another are related, while elements that are farther apart have less of a relationship.
  • Using only one or two type styles and various size or weight for contrast throughout the site.
  • Repeating a color, shape or texture in different areas throughout.
  • Choosing visuals that share a similar color, theme or shape.

Web users rely heavily on visual clues when making decisions about a website — whether to click and explore, consider a purchase or sign up for a service. This is even more pronounced for first-time visitors when the decision to stay on-site or abandon is made in just a few seconds. Follow these four design principles and you can be sure that your users and new visitors will stay engaged with your website.

Guillermo Cedillo is responsible for the design and implementation of modifications of different Web, desktop and mobile applications as a User Interface Designer for Sieena. Sieena is a Nearshore software development firm specializing in Microsoft technologies, with operations in Los Angeles and Monterrey, Mexico.

 

About Usability Counts

Patrick NeemanPatrick Neeman is Director of User Experience at Jobvite, a social recruiting platform and runs both the UX Drinking Game and Startup Drinking Game | More | Contact

If you're a UX Designer in San Francisco, ping me at Twitter. I want to add you to a list I have there.


 

Alltop. I don't know how I got there either.