Professional and Technical Writing/Design/Usability
Usability Testing of Technical Communications
What is Usability Testing?
Usability testing is a tool used to evaluate a set of instructions or product. It tests to see if these instructions or product do what it is designed to do. Usability tests are a necessary part of the process of writing instructions or developing a product.
What is the Purpose of Usability Testing?
Usability testing involves test readers who read and/or use a draft in a similar situation as the intended audience would be in. After the test, you gather important information by observing and questioning the test reader. This allows you to get a good sense of the overall response of your draft so you can revise areas that need improvement. Usability testing can be simple or complex depending on the situation and importance.
Why is usability testing such a big deal and who uses it?
You may think your writing is easy to understand but more importantly, does your reader? Does your draft work the way it is intended to? The only accurate way to answer this question is to have a test subject read and evaluate your draft. Primary draft testing is the best method to increase the quality of a final draft that will be read and studied by thousands of people. Test materials range from drafts of websites, to print instructions, to informational documents.
Everybody from fortune 500 companies to college students use usability testing in order to improve the efficiency of their final draft. Big corporations treat testing as such a big deal that they build special buildings for the sole purpose of usability testing. These special facilities can be equipped with video cameras, monitored computers, as well as one-way mirrors so testing specialist employees can observe customers' responses to drafts. College students can use this strategy with a roommate or classmates. Whatever the situation, usability testing is proven to be a valuable tool in the writing world.
Preparing for Usability Testing
Establish your test objectives Create questions or concerns that you want answered as a result of your usability testing. Be prepared however, often times questions arise during the testing process. Pay close attention to what your tester is telling you, this is valuable information to help improve your instructions. Make sure to annotate these questions/concerns in your observations, because this should be red flag that issues exist in a particular section that must be addressed. Once you realize what you want to learn, you can customize the test so it evaluates and gives you the specific information you are looking to obtain. The two most general questions usability testing seeks to answer are:
- How can my draft be improved?
- Is my communication sufficient?
Other examples of more specific question are:
- How well is the page formatted?
- Does the draft influence the test readers’ attitude the way I intended it to?
- Are my tasks easy to follow?
To help evaluate this criterion you may create measurable criteria. Measurable criteria can involve having a test reader read an informative draft and having him answer questions. Make sure the criteria is highly specific to your objectives. Criteria that is too broad will prove to be invaluable when assessing the problem later on. If he gets a certain percentage correct, the draft is adequate. It can also be time related. For instance, can the test reader navigate to the website link I provided within twenty seconds?
Pick suitable target readers that fit with the testing situation This is a very important part of the testing because if a test reader is not a part of the target audience, then they may influence you to make ineffective adjustments to your paper. A vital rule for most cases is to pick a dentist for dentistry or an electrician for electrical instructions. Furthermore, the knowledge levels of your target readers on the subject you want to test them on should varied according to the the objective of your testing. For example, you are constructing an origami instruction for individuals with intermediate origami level, and you want to test the clearly of your instruction with a usability testing. If you select test readers with no origami experience, your instructions will seen overly complicated to the readers and the outcome of the usability testing will send you a message that additional clarification is needed where in actuality your the amount of information/explanation in your instruction might just be sufficient for your target audience. Similarly, testing the clearly of your instruction with test readers in advanced origami can lead you to over simplify your instruction. This is to say that you have to make sure the test reader is not too familiar with the subject so that he already knows what to expect. If the reader is already familiar with the project, he may not have any difficulty understanding it, but your target audiences may be perplexed by how complex the writing is. The number of test readers used can range and play a role in determining the usability of your final draft. The number of test readers used depends on the type of draft and the availability of test subjects. For example, sometimes ten or more test readers are needed in testing the overall comprehension of a draft. The purpose of needing more test readers is that each test reader have a unique perspective, and thus can contribute different opinions and ideas. On the other hand, in case of a more structured drafts such as the step-by-step instructions, fewer testers are needed because all you need from them is for them to test for you the clearly of your instruction. The key here is to make sure that your test readers use your draft the same way your users will.
Major categories for readers' tasks Generally speaking, the tasks done by the test readers can be categorized into three categories.
- Perform a procedure (i.e. performance test), such as going through a step-by-step instruction.
- Locate information (i.e. location test), such as finding information in a Reference Manual.
- Understand and remember content (i.e. understandability test), such as learning something through reading.
In case of testing for a newly designed website, all three categories are needed to ensure accurate results.
There are three main types of usability tests:
A performance draft is also known as instructions. It is writing in order to help the reader perform an action. The way to test a performance draft is to give the draft to tester in an environment that has the same tools, information, and references as your target audience would have. As they read the instructions, observe them without interference in order to see if they can perform the process as smooth as possible without any complications.
Here are the four major elements of performance tests:
- Tasks: Ask testers to perform the same tasks your readers will perform.
- Location: Use the same setting for testing that your readers would use.
- Resources: Provide testers with the same tools, equipment, and other resources that your readers will have but not additional ones.
- Information Gathering: Gather information in ways that will enable you to observe the details of the testers' experiences without interfering in their testing of your draft.
Location tests ask readers to locate information in reference manuals or websites. One way to test how valuable the layout of your writing is is to ask the test readers questions and have them locate information in the text as fast as they can. This type of test is very valuable in terms of how good your headings, topic sentences, front matter, back matter, and links are at guiding a reader through the text or website.
Understandability tests are extremely important just for the fact that if your reader does not understand or comprehend what you are communicating then the writing is useless to them. Understandability testing should be utilized in every writing test. The main way to conduct understandability tests is to have the test reader read the information and then answer strategic questions in order to get responses that will help guide your editing process.
- Note: Conducting many small usability test drafts is far more efficient than conducting one large test after the finished draft is complete. If you are testing a web page, sketch the design on paper to be tested so you get a sense of what is needed before starting the website creation. Testing drafts more frequently will allow for many small easier revisions instead of one big overhaul of the draft.
After Usability Testing
After the test subject reads and uses your draft, the first step is to interview the person. This is used to gain more crucial information in addition to the information you observed during the testing. The questions vary according to the type of draft and the criterion of the usability test, but here are some general questions that could be asked:
- What was the biggest problem you encountered in the draft?
- What do you suggest?
- What did you like about the draft?
- Was the format user friendly?
After usability testing, the second step would be to analyze the data you collected. A very helpful way to analyze data is to input your raw data into an application such as Microsoft Excel from which graphs and tables can be constructed. From the test and the interview, you should be able to determine specific areas of your draft that need improvement or revamping. Examples could be anything from simply rewording to make more sense or possibly even a complete revamping of your draft. For this reason, usability testing is a great tool for determining the effectiveness of your draft.
The third step would be to implement the necessary changes to your draft. After the changes are made, further testing is often carried out to see whether or not the changes worked. It is important to realize that your first draft of instructions are usually not perfect and multiple revisions and testings are often necessary.
How to Make Usability Testing More Effective
Usability testing can often be a lengthy process. For this reason you want it to be as effective as possible. Listed here are some ideas to make your usability testing more effective.
- Make sure you are prepared to take notes on your testing. Have documents with charts that can be filled with text during the testing process to make data more easy to analyze
- Ensure testers are part of your target audience.
- ex. If you have a set of instructions designed for adolescent men, do not perform usability testing on women or men outside the specified age group
- Have all the materials for the tester be within reach, or easily available
- Since you will be testing multiple people on a given day, bring enough testing material so you can perform all the tests you wanted to do
- Be sure be descriptive about what criteria you will be recording. You can make a number system to rank experience or possibly overall clarity of whatever is being tested.
- Also leave room for a short descriptive interview at the end of the usability sheet to get the users thoughts and concerns. This will be valuable when you are revising your document to ensure it is of the highest quality and free of complications.
Avoiding Bias in the Testing Process
Avoiding bias is a big thing that can help save a lot of time spent revising. Test readers' responses need to be natural and must be similar to how the target audience would see the draft. Five simple things can minimize biased test results are:
- Downplay the fact that the writing is yours. This will allow the test readers to be more honest because they will not be afraid of offending or criticizing you. Do not show expression when the tester is critiquing the draft because the tester may not be as honest if they see that the work is yours.
- Pay attention to how you phrase questions. Make the questions so the reader has an open path to answer either way. If the format of your questions creates pressure to pick the ideal answer for the test reader, your results could be considered biased.
- Only intervene into the testing phase if it is absolutely necessary. Letting the test reader struggle through issues in the draft himself will allow for pure results that are unbiased. Of course, intervention should be used if the test reader is in danger of injury.
- Make sure the environment is as distraction-free as possible for the test reader. Keep your distance! You do not want to impede in the testing process distracting the subject.
- Ideally, if possible, watch the test from a video camera, one-way mirror, or afar while somebody else conducts the test. However, if this person conducts the interview also, he probably will not summarize the answers the same way you would.
Cross Cultural Testing
If you are going to have your writing read by people around the world or in different cultures, you are going to have some extra issues to consider. This is because every culture has different beliefs and norms. Writing in one culture that is clear and simple can be complex and inappropriate in another culture, unless special measures are taken when conducting usability testing. For example, when Breathe Right(R) nasal strips were going global, their box contained dark coloring because the product was suppose to be used to clear up the nasal passage during the night time. However, this dark package theme did not work well in Europe. They saw it as dark and gloomy and they wanted the package to be light and clear. The key to obtaining global technical communication is to have usability testers from that culture.
Ethics in Usability Testing
Due to the fact that technical communication is lawsuit prone, appropriate measures should be taken even when usability testing is used. Technical writing testing has the same loose guidelines as research in medicine and psychology:
- The volunteer must be informed and fully aware of the process.
- The volunteer must have agreed and given approval to participate in the testing.
For some types of tests, this is only an ethical issue. However, there are situations where there are legal restrictions for the researcher to inform the volunteer of the situation in writing.