There are different usability testing methods within each of the above types, and you can combine some of the approaches to develop a test based on your needs and resources (i.e., time, effort, and budget). The rest used a combination of three (n = 30) or four (n = 12) methods of testing usability. There are various usability testing methods available, but in this chapter, we look at the top techniques you need to know when running a usability test. This can be seen as an irreplaceable usability practice, since it gives direct input on how real users use the system. But this method also works for shipped Card sorting will help you form a hypothesis on how to Qual testing involves a small number of users (5–8) and directly identifies the main usability problems in an interface. During a moderated test, moderators are facilitating test participants through There is a wide array of software available to conduct remote usability testing. Usability Testing also known as User Experience(UX) Testing, is a testing method for measuring how easy and user-friendly a software application is. Phone interviews are an excellent way to collect feedback from Lab usability testing requires having a trained moderator Early on, usability tests in computer science were conducted primarily in academia or large companies such as Bell Labs, Sun, HP, AT&T, Apple and Microsoft. Remote usability testing allows for the length of time it took each tester to complete various tasks to be recorded. Remember that what test participants say is not always the same as what they There are a variety of online testing tools that allow you to remotely observe user behavior on your website; some let you pay participants to take short tests, while others monitor the behavior of real users as they interact with your site. They follow a general prepared script that begins when the moderator introduces the test to the user, and reassures him/her that it’s not a test of their ability, but of the system. patterns. And as we know, A/B testing is about experimenting with two or more versions of a page/screen to see which is most effective. Moderated testing usually produces in-depth results thanks to the direct interaction between researchers and test participants, but can be expensive to organize and run (e.g., securing a lab, hiring a trained researcher, and/or providing compensation for the participants). results. but you often need to conduct another testing to understand why they face this A problem discovery usability test is the most common type of usability study. Product may do a soft launch (v.0.5) bef… The purpose of the plan is to document what you are going to do, how you are going to conduct the test, what metrics you are going to capture, number of participants you are going to test, and what scenarios you will use. A In-person tests are done in a testing lab or office, in the presence of a professional moderator who observes participants as they complete the test. The fact that this testing is moderated vs. unmoderated enables you to collect more qualitative information. Formative usability testing may follow this process: 1. Scripted or unscripted: scripted tests take users through a set of specific, predetermined tasks to see how they respond to and interact with particular pages or elements on your site or app. That’s why Guerilla testing might not be right for testing choosing a moderator: With lab testing, there is always a risk that the controlled It offers quick, robust, and inexpensive user testing results to A/B testing. But don’t worry, every UX specialist faces the same problem. It’s also In this article, we will explore seven different usability testing methods when you should use them and why. • If you want to get started with usability testing right away, you can try running a remote, unmoderated test. Typically, stakeholders also watch the proceedings and take notes behind a one-way mirror in the testing area. Now let’s discuss usability testing methods that you might In guerrilla testing, test subjects are chosen at random from a public place, usually a coffee shop, mall, or airport. A phone interview is a remote usability test where a and a place for running a testing. The main goal of usability testing is to inform the design process from the minutes of their time. Room with portable recording equipment 3. people who use products, services, and apps, usability testing is paramount. That’s why it’s not recommended to use unmoderated It's a quick way to collect large amounts of qualitative data that validate certain design elements or functionality—but it's not a good method for extensive testing or follow-ups, as people are usually reluctant or unable to give up more than 5-10 minutes of their time. Cost: an eye-tracking study requires you to rent a lab with special equipment and dedicated software (plus the trained technician who can help you calibrate the device). first-hand experience. While it should be manageable to find people who are willing to test your application and give feedback, there are also companies who are dedicated to setting up user testing services for you. Usability testing has five major advantages over the other methods of examining a product's user experience, like a questionnaire: Usability testing provides an unbiased, accurate, and direct examination of your product or website’s user experience. Problem Discovery . recording data helps to understand what content/features are the most early stages of the product development process. collecting personal opinions and emotional impressions about ideas and By analyzing The goal is to watch how test participants interact with a product, Session recordings use software to record the actions that real (but anonymized) people take on a website such as mouse clicks, movement, and scrolling. For example, UserTestingprovides videos of real u… perspective of the end-user. When it Compared to remote tests, in-person tests provide extra data points, since researchers can observe and analyze body language and facial expressions. environment will be different from the user’s real environment. Usability testing recording software such as TechSmith’s Morae (PC only) or Silverback(Mac only) may be used to record the computer screen and the participant’s voice and facial expressions during testing. So it’s important to ensure that all test participants are reflective of your actual customer base. Breaking your analysis down into steps can help you organize your findings in an understandable way. interactions and select the most probable one (core user flow). Guerrilla testing is used to test a wide cross-section of people who may have no history with a product. Qualitative and quantitative user testing are complementary methods that serve different goals. This usability testing method involves asking participants, … this type of testing. Before you pick a user research method, you must make several decisions about the type of testing you need based on your resources, target audience, and research objectives (aka: the questions you want to get an answer to). Free. Usability Testing: Everything You Need to Know (Methods, Tools, and Examples) By Emerson Schroeter, updated on March 27th, 2020 Length: 14 Minutes As you crack into the world of UX design, there’s one thing you absolutely must understand and learn to practice like a pro: usability testing. It’s helpful to understand if the user can gather the purpose of a website quickly. The techniques listed below are occasionally labeled as usability testing—and although they technically are not, they can (and should) be used in conjunction with usability testing to generate more comprehensive results: With so many different usability testing methods, we won’t blame you if you don’t immediately know what's right for you. So here’s a recommendation: start small and take your first step. Many UX specialists consider A/B testing as a usability testing method. These tests are easy to set up, and depending on your traffic and the scope of the test, you can get results in a matter of days (or even hours). Usability testing is a powerful tool for evaluating a website's functionality and making sure people can navigate it efficiently. Contextual inquiry is less a usability testing method and more like an interview/observation method that helps a product team obtain information about the user experience from the real users. target audience. When it’s time to conduct the usability testing session, you or your moderators … their own environment using their own devices and without a moderator present, By getting feedback on your navigation Phone interviews are an economical way to test users in a wide geographical area. Understanding the top usability testing methods. Test participants are asked to complete tasks in Unmoderated remote usability testing works the best when you The cost of unmoderated testing is lower, though participant answers can remain superficial and follow-up questions are impossible. design (wireframes or lo-fi prototypes) and what to know whether you’re moving Guerilla testing is the simplest form of usability testing. mean that a moderator should tell test participants what they should do. The goal of usability testing is to reveal areas of confusion and uncover opportunities to improve the overall user experience. This type of usability research takes place inside a specially built usability testing lab. All Rights Reserved. testing will help you test a particular question or observe user behavior Conversely, an unmoderated test is done without direct supervision; participants might be in a lab, but it's more likely they are in their own homes and/or using their own devices to browse the website that is being tested. Usability testing is a popular UX research methodology.. The cost of unmoderated structure, you will be able to make more data-informed decisions. At the same time, lab testing can be expensive to organize and run because you need to secure an environment, hire test participants, and a moderator. Comparative research tests ask users to choose between different solutions, pages, or elements, and can be used to compare your website or app with competitors. Copyright © 2014 - 2020 Hotjar Ltd. All rights reserved. The examples of session recordings and first-click tests (mentioned earlier on this page) are two types of remote, unmoderated usability tests. When you have a tangible After they sort the cards, they explain their logic in a moderator-run debriefing session. Eye-tracking studies can be used to glean information about how users interact visually with a page; they also help test layout and design elements and see what may be distracting or taking someone's focus away from the main page elements. moderator verbally instructs participants to complete tasks on their device and problem. Learn what beta testing is and how product design benefits from it. Usability Testing measures how well a system or an application can be used by real users under controlled conditions and achieve their intended goals. The technique is relatively simple—all you need to do is place concepts (content, features) on cards and allow test participants to manipulate the cards into groups and categories. This software can also facilitate tracking of user behaviors, including mouse clicks, keystrokes, and active or open windows. in special environments (laboratories) and supervised by a moderator. Card sorting involves placing concepts on virtual note cards and allowing participants to manipulate the cards into groups and categories. The audio or video is recorded for future reference. However, in-person testing is usually expensive and time-consuming: you have to find a suitable space, block out a specific date, and recruit (and pay) participants. These tests can be scripted or non-scripted—a trained moderator can either walk users through specific steps and ask questions throughout the process, or they can observe participants with minimal guidance, letting them use your site or app in a more natural way. testing is lower; however, this type of testing offers less detailed testing Your in-depth knowledge of your website or app may prevent you from seeing some usability issues that outside, objective users will notice—which will help you pinpoint issues that you may have otherwise missed. real (but anonymized) users take while they interact with a site. You might receive audio recordings of users talking while they navigate your site or videos of users completing a set of tasks. communication skills. They are asked to perform a quick usability test, often in exchange for a small gift (such as a free coffee). Hallway usability testing: according to this method, random individuals (as opposed to experts) are chosen to test a website. back Methods and Types of Usability Testing 30 July, Małgorzata Traczyk Post a comment. Fixed laboratory having two or three connected rooms outfitted with audio-visual equipment 2. This type of testing requires a trained moderator. In-Person Usability Testing. users— their workspace, personal preferences, and habits. Moderators reach test participants after the testing You can do effective usability testing in any of these settings: 1. Your browser doesn't support HTML5 video. It’s vital to understand that usability is all about having individuals experience a product’s functionality. DESCRIPTION. Using various usability testing methods can also help address any potential shortcomings or limitations in any particular testing method. a moderator. The most common usability testing methods include the concurrent think aloud, retrospective think aloud, and various remote options. Unmoderated remote usability Usability testing is the practice of assessing the functionality and performance of your website or app by observing real users completing tasks on it. Because they are less expensive than in-person interviews, they help collect more data in a shorter period. session recording results, you form a hypothesis on what problems users face, The product goes through acceptance criteria testing before being released to target audiences. The goal of A/B testing is to find the page that will convert better, while the purpose of usability testing is finding usability issues that prevent users from having great user experience. organize your content/features. Moderated and remote usability tests are performed via a computer or phone and require a trained moderator. Formative usability testing is another form of early-stage testing. test participants scattered around different parts of the globe. None of the studies used automated mechanisms to test usability. In an industry devoted to creating a great experience for problems that users face when they interact with your product. Remote usability tests are done over the internet or by phone; in-person testing, as the name suggests, requires the test to be completed in the physical presence of a UX researcher/moderator. Remember that the testing method you choose should be aligned with both your resources and your objectives. This technique is useful for getting rich information about They are resource-heavy but excellent for collecting in-depth information. comes to interacting with test participants, a moderator should have excellent feedback is collected automatically (the user interaction recorded remotely). Nick has spent the last 10 years working in the software industry with a specialized focus on research and development. not participate in these interactions. Define what you want to test… For example: • If you have time (and some room in your budget), you might want to try an in-person, moderated test. During eye-tracking tests, researchers observe and study users' eye movements using a special pupil-tracking device mounted on a computer. interesting for the users (via heatmap analysis) as well as what interaction Discover more tips and best practices on the UX design process here. Moderator should be good at decoding body language. We explore the difference between quantitative and qualitative data and how to choose between moderated and unmoderated usability testing. Methods that are not usability testing: Heat maps & Scroll maps: These are used to produce a visual representation of how users move around a page by showing its hottest (most popular) and coolest (least popular) parts. participant’s reasoning. This is an easy way to collect a large amount of qualitative data about people’s first impressions and reactions to your site. Assessment research tests help to evaluate performance and functionality by testing user satisfaction and effort. remote testing as a first usability testing method. The three overall usability testing types include: A moderated testing session is administered in person or remotely by a trained researcher who introduces the test to participants, answers their queries, and asks follow-up questions. The tasks you select for your testing session play a tasks, answering their questions, and replying to their feedback in real-time. think. Such testing is paramount to the success of an end product as a fully functioning application that creates confusion amongst its users will It’s important to take notes during the observation. The System Usability Scale (SUS) was the most frequently used questionnaire (n = 44). Usability Testing. problems users face while they interact with your product. With so many different usability testing methods, it’s often hard to select the one that will be good for your product. Since it’s impossible Guerilla testing is also good for The testing takes in real time basis in the physical presence of Moderator who observe the facial expressions and body language of Users while they perform the tasks. Session By measuring how long it takes users to make a decision, you learn how intuitive your site design and linking structure are. sessions. Observation testing allows researchers to see the body language and facial expressions of participants without interference from a moderator. Here are some of the most common tools: In this test, website owners upload a screenshot of their webpage with a single question like “What is the main element of the page that stuck with you?” or “Who do you think the intended audience is?” Test subjects have five seconds to look at the page before they answer the question. Guerilla usability testing : it is a quick type of usability testing that takes place in common places like cafes, libraries, malls etc. Common user testing methods include guerrilla testing, in-person usability testing, remote usability testing, contextual inquiries, and card sorting. It’s always important to understand Usually, you have 5-10 participants per research round in a controlled environment. brokers). Sign up for a free Hotjar account and SEE how visitors use your website. To conduct a session recording, you By analyzing where users direct their attention when asked to complete a task, the machine can create heatmaps or movement pathway diagrams. It’s essential to choose the method that you think will be good for your project and be flexible enough to change the direction if needed. In this section, we explore different usability testing methods, when you should use them, and why. They are asked to perform a quick usability test, often in exchange for a gift card or other incentive. The participant is asked a question like “Where would you click to buy this product?” and the software records where they direct their mouse. niche products that require having special skills (i.e., software for finance Usability Testing is a non-functional testing technique to evaluate how easily the end users can use a product or a service. Rebekah Baggs and Chris Corak used to lead separate teams at one of the largest agencies in Arizona.…, Despite the name, heuristic evaluations are not as complex as they sound. In other words, you have a particular hypothesis that you Its results show designers and product managers how people and potential customers naturally organize information, which can help make a site more intuitive to navigate. When all participants have completed the study, the researcher will compile the data to determine the severity of each usability issue that was encountered and provide prioritized recommend… Relying mostly on computer programs, these passive testing methods provide insight into how users interact with a website in their ‘natural environment.’. In a usability-testing session, a researcher (called a “facilitator” or a “moderator”) asks a participant to perform tasks, usually using one or more specific user interfaces. As soon as test participants sort the cards, a moderator should ask them to explain their logic (to understand the reasoning). In general, tests should be about 15–30 minutes long made up of about 3-5 tasks. Unscripted tests let you observe users’ movements and actions while they take natural steps without any guidance or direction. Moderated or unmoderated: moderated tests are led by a professional who guides and observes users through the entire testing process, while unmoderated tests are done without guidance. Guerilla testing. Benefits of Usability Testing. running the session. A five-second test is performed by showing the user a design or landing page for five seconds. This is faster and less expensive than moderated sorting (though you miss out on the opportunity for follow-up questions), and users can complete the test at their convenience. A 90% confidence level is the benchmark that is commonly used for product design usability test. need to obtain a large sample to prove critical findings from your initial By placing the Unmoderated in-person tests are conducted in a controlled, physical setting but don't require a person to administer the test. Card sorting is an excellent method for prioritizing content and features in user interface. A foundation or building block would be another way to look at this type of testing. Usually, people who participate in guerilla testing will give you maximum 5-10 A: Combining usability testing methods can help you to generate a broader set of data and outcomes from which you can take action to improve your website. information at the beginning of the design process will help the product team Remote testing doesn’t go as deep into a participant’s reasoning, but it allows you to test large numbers of people in different geographical areas using fewer resources. They may interject if a participant gets stuck, but otherwise, they remain quiet and concentrate on taking notes. But it’s always important to validate this It’s easy to prioritize the usability issues when you see them from the That’s why a moderator should be good at observing and analyzing body important to remember that you will have a limited time per every test session. Card sorting is a great method for both new and existing websites to get feedback about layout and navigational structure. Example Usability Test Session. You can either recruit family and friends to try using the app or find focus groups that are willing to explore different features and functionalities. However, these tests are expensive and usually based on a small population size (8-10 participants per research round) in a controlled environment, which is not necessarily reflective of your actual customer base and/or real-life use conditions. One of the easiest ways to get started with usability testing is session recordings. This test should occur before the first release of the developed product. The facilitator will welcome the participant and explain the test session, ask the participant to sign the release form, and ask any pre-test or demographic questions. Remote usability testing allows you to conduct user research with participants in their natural environment by employing screen-sharing software or online remote usability vendor services. language and facial expressions. participant on track if they have any sort of confusion. To make usability testing work best, you should: 1) Plan – a. Usability testing is a technique used in user-centered interaction design to evaluate a product by testing it on users. Card sorting, described above, can also be conducted in an unmoderated and remote manner if you skip the debrief session afterward. Card sorting is great when you want to optimize your Basically, guerrilla testing … User testing requires having real people use your app to see how well it functions from a nonbiased and unfamiliar perspective. As a general rule of thumb, use moderated testing to investigate the reasoning behind user behavior, and unmoderated testing to test a very specific question or observe and measure behavior patterns. This information not only will help you select the relevant testing method but also adjust the questions and tasks for your test participants. In fact, they’re a great tool…. Room with no recording equipment, as long as someone is observing the user and taking notes 4. It is more concerned with the design intuitiveness of the product and tested with users who have no prior exposure to it. want to validate on a large segment of your users. critical role in whether findings will be useful or not. Test participants (real users) are first asked a set of questions about their experience with a product and then observed and questioned while they work in their own environments. One of the first steps in each round of usability testing is to develop a plan for the test. One of the easiest ways to get started with user testing is through session recordings: this remote and unmoderated technique helps you identify usability issues with your site by just watching real people interact with your site pages and elements. Sometimes changes that are supposed to improve the Remote or in-person: remote tests can be done from anywhere, and are completed either online or over the phone. concepts. To recap, rapid usability testing allows Designers to gather insight about the effectiveness of their concepts before development work starts. Usability testing is all about having individuals test and experience a website’s functionality. Copyright © 2019-2020 Adobe. The methods range from well-known lab-based usability studies to those that have been more recently developed. LAB USABILITY TESTING. The facilitator explains thinking aloud and asks if the participant has any additional questions. Remotely, with the user in a different location (either moderated or unmoderated) Session recording is a method of recording the actions that Get XD Ideas delivered weekly to your inbox. The downside? user’s satisfaction with a product. The after-testing interview is an essential part of It’s low cost and relatively simple testing that enables real user feedback. The goal is to uncover (and fix) as many usability problems as possible. Having notes after every observation session will help you write a detailed test report. When reporting results from a usability test, you should focus primarily on your findings and recommendations that are differentiated by levels of severity. It is a good method of testing because the tests are carried out in the normal environment of the user instead of a controlled lab. Usability Testing is an Iterative Process. Website scannability can be the number one factor between a user accomplishing their goal or bouncing. A small set of target end-users, use software application to expose usability defects. We can choose from testing design mockups, layouts, websites, forms and many others. • If you have time (and some room in your budget), you might want to try an in-person, moderated test. With so many different user testing methods, it’s often difficult to select the best one for your product. It requires little setup and is a simple way to start improving the functionality and effectiveness of your website. It focuses more on quality assurance. A moderator should always be ready to help test Here are a few things to remember when want to include in your test plan: Guerilla testing is the simplest form of usability testing. In this sort of test, the researchers watch but don't participate, acting as a sort of ‘fly on the wall’ as participants run through a set of instructions in a lab. In a phone usability test, a moderator verbally instructs participants to complete tasks on their computer and collects feedback while the user's electronic behavior is recorded remotely. These three testing methods generate different types of information: Each usability testing method gives answers to your research questions. A/B testing is a form of usability testing that’s used when there are two different options … non-realistic user behavior. This gives you many of the benefits of testing in a controlled atmosphere and reduces the possibility that a moderator could lead or influence participants with their questions. He counts advertising, psychology, and cinema among his myriad interests. Getting all this Remote, unmoderated tests are one of the most affordable usability testing methods—you just need your computer and some specialized software. questions and tasks for your test participants, Everything You Need to Know About Beta Testing, Improving Website Scannability for UX Design Success, How Onward Bridges the Gap Between Content, UX, & Technical SEO, How to Use Heuristic Evaluations to Improve Product Designs. when it’s used together with another type of usability testing. Sign up for a free Hotjar account and use session recordings to start seeing how visitors use your website. There are different usability testing methods within each of the above types, and you can combine some of the approaches to develop a test based on your needs and resources (i.e., time, effort, and budget). Explorative, assessment, or comparative research: explorative research tests ask users open-ended questions to gather their opinions and insight about concepts of your site or app. Test participants are chosen randomly. navigation experience lead to completely different outcomes. There are different methods for usability testing. Research should never provide their opinion during test As the name suggests, lab usability testing is testing run User Experience testing is getting more and more popular. Effective Usability Testing does not require a formal usability lab for testing. In-Person testings allows you to gather rich insights. in the right direction or not. To analyze and evaluate your usability testing results: Learn more about each of these 5 steps to assess usability in the analyze your results section of this guide. design a well-tailored experience. participants understand the purpose of testing (describe the goal) and keep the Remote usability testing doesn’t go deep into a test Guerilla testing works best in the A five-second test is a usability testing method to understand a visitor’s first impression of your website. If a moderator is present for the test, he/she should not bias the user by favoring one choice over another with comments or facial expressions, or do anythi… It is often used formatively, to inform the design process and channel it in the right direction. The examples of Lab testing and guerilla testing (mentioned earlier on this page) are two types of in-person, moderated usability tests. Where teams create the world’s best experiences at scale, powered by the leader in creative tools. Session recording will help you understand the major Lab usability testing works best when you need to have in-depth information on how real users interact with your product and what issues they face. be used for further analysis. Contextual inquiry is also good when you want to test a They’re good for picking from a wide range of testers while still taking advantage of a moderator's skills and ability to dive deep. product’s information architecture. It will help you investigate the reasoning behind user behavior. need to use a special tool such as HotJar for that purpose. Usability testing is a method of testing the functionality of a website, app, or other digital product by observing real users as they attempt to complete tasks on it.The users are usually observed by researchers working for a business. Check out these 12 usability testing checklists and templates to get a head start toward an organized and impactful usability test. Session recordings are a fantastic way to spot major problems with a site's intended functionality, watch how people interact with its page elements such as menus and Calls-to-Action (CTAs), and see places where they stumble, u-turn (go back to a previous page quickly after landing on a new one), or completely leave. Another problem with this testing is the number of test participants in a single round. A major benefit of lab usability testing is the control it provides: all sessions are run under the same standardized conditions, which makes it especially useful for comparison tests. Learn more about each of these 8 benefits to usability testing in the first section of this guide. With the type of interface and different testing methods in mind here are the five types of usability tests, each addressing a different research goal. Usability testing lets you experience your site or app from the users’ perspective so you can identify opportunities to improve the user experience. Usability Testing Methods . which leads to the product being used naturally. Working methods, best practices, tips and tricks, Unique insights, design stories, the impact of design, Industry leaders shaping the future of design. user in a controlled atmosphere, there is always a risk of creating a Test subjects complete tasks on computers/mobile devices while a trained moderator observes and asks questions. The practice of usability testing grew in the mid-1980s with the start of the modern usability profession, and books and articles popularised the method. hypothesis with real users. Tests that are moderated and conducted in-person offer the most control. UX researchers have developed many techniques over the years for testing and validating product hypothesis and particular design decisions. Session recording works the best Here is an example test session. While the participant completes each task, the researcher observes the participant’s behavior and listens for feedback. The goal of first-click testing is to evaluate whether users can easily identify where they need to navigate to complete a given task. It then becomes the baseline for comparing future tests. First-click testing is useful for collecting data on user expectations and determining the prime location for menus and buttons. that test participant in Guerilla testing might not represent your product’s Unmoderated remote usability testing occurs remotely without to test everything at once, you need to prioritize all possible scenarios of In-person Vs Remote . Which one you choose depends on your product and where you are in your design process. The method you choose will depend on both your resources and your objectives. Basically, guerrilla testing means going into a public place such as a coffee shop to ask people about your prototype. products. The goal of usability testing is to detect any usability problems by collecting qualitative and quantitative data to determine the satisfaction of representative users with the product. session and ask them a few important questions. ... Card sorting is a method used to help design or evaluate the information architecture of a site. moderator is a professional who is looking to obtain feedback from live users. Nick Babich is UX architect and writer. To get the most out of usability testing—and to ensure you can actually use what you learn to make improvements for your users—we recommend that you come up with a plan before you start. However, it does not moderated research. Before you pick a particular testing method, you should have a clear understanding of your target audience, available resources (time and money), and research objectives.