U.S. Servas Usability Testing
U.S. Servas is a nonprofit organization that connects people domestically and around the world. It facilitates programs where people of diverse backgrounds meet and visit people in their homes and learn about different cultures. The organization's members are travelers and hosts, who participate in traveling and hosting travelers.
We screened and recruited 8 participants from two generational groups (Generation X and Y, with a 50% split) that also had the following characteristics:
- Intermediate or advanced computer experience
- All participants not familiar with U.S. Servas
- All participants interested in traveling and/or hosting diverse group of people and learning about other cultures
USABILITY STUDY OBJECTIVE
The goal of this project was to evaluate the navigation and information discoverability of the new version of the US Servas website via a usability study. We wanted to understand a new user’s experience and identify any pain points or usability issues associated with the new design. Working hand-in-hand with the client, our team undertook the following steps as part of the bigger usability study we designed:
- Conducted collaborative walkthrough on the U.S. Servas website
- Conducted in-person moderated usability testing sessions with 8 participants who did not have experience with the U.S. Servas organization or their website
- Usability testing sessions held on the University of Washington campus over the course of two days
- Think-aloud protocol applied while participants worked through the tasks
- Within-subject design applied and each participant worked through all tasks
- All tasks were randomized, except those tasks that had to be executed in a sequential order to preserve their understandability
- Laptop with Morae software used during testing for recording user clicks as well as their comments
- Debriefed with participants after tasks to gather qualitative nuances
- Overall, participants liked the U.S. Servas concept to meet people, travel and host for cultural exchange
- Required steps, to register and apply with U.S. Servas as a traveler and a host were difficult to find
- Participants liked information located in one place (FAQs)
- Participants struggled with the verbose content
- Participants were unsure about the membership fees even when they found them
- Participants were confused with various entry points to update their personal information
How did we get there?
First, we set some goals - the questions we wanted to find the answers for during our usability testing:
- Will users understand traveler responsibilities?
- Will users understand the application process timeline for traveler and host?
- Will users understand the cost associated with the traveler and host roles?
- Will users be able to identify other programs with U.S. Servas?
- Will users be able to identify the required steps to become a traveler or host?
- Will users be able to complete the registration process with Servas?
- Will users understand how to update their personal information?
Then, we ran our usability test, during which each participant, working individually, was asked to complete a set of tasks that would help us answer the questions above.
Through the usability test, we observed user actions, conducted short interviews, took notes, recorded time and video of the participants, as well as their website navigation via screen recording. That helped as gather quantitative and qualitative data to arrive at the conclusions we did.
Here are a few snippets of our QUANTITATIVE data, visualized on graphs:
And here's some QUALITATIVE stuff, followed by our team's recommendations:
Overall, most of the participants were intrigued by the U.S. Servas programs and expressed their interest to become a member of this organization and participate either as a traveler or host to support cultural exchange.
“I am very fascinated …[with U.S. Servars programs] … I cant think about anything like it. It kind of remindes me of studying abroad where you have a host family but this seems ment for people of other ages … I would keep clicking … I would be interested.” (P1 Kerry)
“This is really good and cool … it would be interesting experience [to join them] sometimes …” (P5 Yury)
Team up with colleges and universities and share the U.S. Servas concept and programs more broadly
The website provided a false positive feeling for participants that they are informed due to content overload, yet participants also experienced difficulty completing tasks caused by the lack of important information. The content hierarchy and formatting of content does not support scanning of information leading participants to look elsewhere. Both of these issues lead participants feeling not confident.
- For 3 out of 10 tasks, participants took longer paths than expected to locate the correct information.
- All participants successfully located the responsibilities of a traveler
- 2 out of 8 participants took more steps than necessary to complete this task
- All participants were overwhelmed by the size of the Traveler Responsibilities PDF document
- It was not intuitive for most participants to go to Community to find Programs
- 6 out of 8 participants’ first inclination was to go to About to find the additional programs
“Do I have to read this whole thing?” (P7, Anne)
“It was sort of a pain...how many clicks I went through to get to this document.” (P6, Eliza)
- Use consistent terminology throughout the site
- Make the content more concise so it is easier to read and comprehend
- Surface and emphasize the top-priority content (e.g. costs, responsibilities, application process, etc.) so users don’t have to go out of their way to find information
- Reorganize content hierarchy
- Remove redundant and unnecessary links
We expected participants to use the homepage to locate the necessary links to complete tasks. It was not abundantly clear to our participants that they could scroll down the homepage past what was visible on the screen. In the post-test discussion, users were surprised that there were more direct links below the main picture.
- Most participants did not look for information on the home page
- 1 out of 8 participants accidently scrolled down the homepage and was surprised by additional information
- None of our participants completed 3 of the tasks following the predicted path via the links on the lower part of the homepage.
“I didn’t know if there was anything underneath the picture cause the picture was right to the bottom. And I think I was a couple questions in when I realized that there were additional links and items underneath it.”(P8, Darby )
Homepages are the most valuable real estate on a website and U.S. Servas has the opportunity to optimize this by doing the following:
- Remove the static picture at the top
- Provide a one-sentence tagline that summarizes what U.S. Servas does
- Emphasize the site’s top high-priority tasks
- For example, move the Become a Traveler/Become a Host links from the bottom to the top of the page so they are immediately visible
We expected participants to understand they could click on the top-level navigation links, Travelers and Hosts, to find the required steps to become a traveler or host. Entry points to the travelers and hosts landing pages were undiscoverable. Additionally, Ways to Travel and Ways to Host pages did not provide sufficient information so participants resorted to the FAQs pages.
- Only 1 out of 8 participants successfully uncovered the Hosts landing pages via the top-level navigation link
- Only 1 out of 8 participants found the Travelers landing page through a breadcrumb on the Ways to Travel page
- None of our participants completed 3 of the tasks following the predicted path via the top-level navigation links.
- Only 1 out of 8 participants considered going to Ways to Travel as one of the predicted paths to complete the task
“...I did not expect that cause when this drop down popped up, I thought, oh, this is what I’m picking, so I didn’t click on.. just the word itself - the Travelers, the Hosts because I got this pop-up, and I was like, oh...these are the choices under this organization...“ (P6, Eliza)
- Make the information for becoming a traveler and/or host more accessible
- Provide access to Become a Traveler and Become a Host pages by
- Emphasizing the respective links and content on the homepage
- Adding them to the respective menu dropdowns
- Redesign the travelers and hosts landing pages
- For example, remove the images and relocate the steps to become a traveler/host to the top of the page
While we observed participants successfully sign up and log in, all of the participants displayed confusion when seeing the "I have an account" and "I want to create an account" buttons on the same page. All participants were confident in completing the signup tasks because the system provided expected confirmation at the end.
- 8 out of 8 participants completed the task successfully
- 3 out of 8 participants looked to sign up by going to content pages (e.g. Travelers, Hosts, or About)
- Most of the participants clicked on the orange I want to create an account button not realizing they were already on the Create new account page
- Most participants were surprised and did not expect the content regarding settings and survey on the Create new account page
- 2 out of 8 participants looked elsewhere to sign up (i.e. Travelers, Hosts, About ) and were confused by the Sign Up link in the upper right corner
“It wasn’t clear to me that sign up - what that meant there...What threw me off the most? I think it would be nice to say ‘Hey, are you interested?.. Here is where you can register to become a, you know, something more. Sign up/login is on every single website everywhere...I would have expected something under travelers or hosts...” (p7, Anne)
- Use consistent terminology (e.g. Create new account vs. Signup vs. Registration etc.) across the site
- Eliminate confusion by removing the “You are not logged in” message when trying to create an account and log into their account
- Eliminate confusion around the I have an account and I want to create an account buttons by doing the following:
- Remove both buttons when the user is already on the Signup or Login pages (i.e. remove I have an account and I want to create an account buttons)
- Remove the message referencing the old site when user trying to log into their account and instead walk users through updating their password
- Eliminate confusion and remove the Locale settings on the Signup page
While participants were successful in updating personal information, we observed that all participants were confused by the multiple entry points to edit the same information and the elements on the account page. Although most participants clicked on the Create, update, or renew your Servas profiles link, it was not immediately clear to them that it was a link until they hovered over it.
- 8 out of 8 participants completed the Update your email task
- 6 out of 8 took the expected path to complete the task
- 2 out of 8 found an alternative way to complete the task via the Edit page
- Although they completed the task, these two participants were confused about the purpose of the page and surprised by its content
- ⅞ participants successfully completed the Update your mailing address task
- 2 out of 8 participants clicked on the Edit page, thinking they could update their mailing address from there, but could not
- 2 out of 8 participants clicked on the Address Book, thinking they could update their mailing address, but could not
- Most participants were confused about the numbers, arrow, and x’s for each piece of information on the Create Your Contact Information page
- None of the participants thought the Edit page was consistent with the remainder of the website in terms of look and feel.
“And when it said ‘update contact information, the first one’ I thought oh, i should probably click that….’wait a minute, i needed to scroll down instead…. I thought these would hyperlink to something else...I think this is a completely confusing website right now. This page is confusing. I don’t know why all this is here.”(P7, Anne)
- Clarify the difference between a profile and an account
- Remove redundant links - there are 3 redundant links in the upper right hand corner of the page
- Create clarity around the profile steps on the Create, update, or renew your Servas profiles page
- Clarify the purpose of the Address Book
- Create consistent design for the Edit page so it matches the overall look and feel of the U.S. Servas website
- Clarify the purpose of the Edit page
- Give the Edit page a title so the users understand its purpose (i.e. update email and password)
- Remove the survey from the Edit page
With all the findings in mind, we suggested that the client takes the following steps to further evaluate the website's usability and improve user effectiveness (are users able to find the information they are looking for?) and efficiency (can users find the information they need quickly?):
- Apply recommendations from the usability study
- Focus on information architecture by tree testing to evaluate findability of topics on the U.S. SERVAS website
- Evaluate content
- Add missing content
- Use consistent terminology
- Remove unnecessary content
- Conduct additional usability testing focusing on the Create, update, and renew your Servas profiles section