In this article, we’ll describe two creative techniques that generate a wonderful range of usability testing participants and still meet Canadian requirements to store survey data on data servers in Canada. Better recruiting techniques help web teams get faster and more representative results from their usability testing studies.
We’ve had great success using Ethnio for recruiting on the web sites of our large U.S. clients. Ethnio’s web service enables us to recruit, screen and even offer live intercepts – connecting to watch and hear a user as they perform their real task. However, Ethnio stores personal data in the U.S. and therefore, at least for our Government of Canada clients and usability colleagues, using it doesn’t meet privacy regulations.
The first technique recruits participants after they complete an un-moderated, online research study. Our second technique offers a Canadian alternative to Ethnio.
1. Recruit and qualify participants after an online study
The first time we used online recruiting in Canada was during an information architecture project for a federal government site. The plan was to test website navigation by alternating between moderated (qualitative) and automated (quantitative) test sessions. The automated test sessions were done with TreeJack – a tool for testing hierarchical navigation. But it was going to be very tough to recruit representative participants for the moderated sessions when we were on such a tight schedule.
The ‘aha’ moment hit when we saw the Redirect URL option provided by TreeJack. People who had completed the TreeJack study had shown their interest in our research – why not recruit from their ranks? So, at the end of each TreeJack session we redirected the person to a brief survey set up on FluidSurveys. FluidSurveys stores their data in Canada, thus helping government web teams meet privacy regulations.
In the follow-on recruit survey, we asked a few screening questions like the person’s role, and the frequency of their visits to the site under study. Then we collected their name, province, email address, and phone number. We added some language to the bottom of the survey about privacy rights from the Agency’s legal team, and turned on the survey.
An hour of people’s time performing a usability test session is valuable. We made sure that the incentive was prominently promised for successful recruits. And since we knew we might attract more people than we needed, we warned applicants that they must meet a certain set of criteria to be contacted.
What a fantastic field of potential participants! They came from all over Canada, willing to contribute to improving the site. We were able to schedule English and French participants from a wide range of provinces and with varied demographics – for example: first-time, daily, weekly and monthly visitors; topic experts versus students, etc.
Neo Insight has continued to refine this recruiting method over the course of this year. We now have a standard methodology that is very effective at ensuring our usability testing participants are real users of the site under study. We have since used the method to collect many responses on university, high tech, and intranet projects. In one variant, we even started the usability test session by having them recreate their last task on the public version of the site. That way we were able to observe them doing the task they were performing when they responded to the TreeJack invitation, except on the new site we were testing.
2. Use an online survey – a Canadian alternative to Ethnio
Once the value of online recruiting was proven to us in such a powerful way, we wanted to use it in a straight-up usability test that we’d been asked to perform. In this case, the federal client had launched their new site, and in addition to the planned usability testing, they were conducting a series of benchmark surveys on their own FluidSurveys account.
We designed a recruiting and screening survey that met the needs of the test design. After a few test run-throughs, and another pass by the legal team, the invitation went live on the client’s FluidSurveys account. Recruits poured in – we were able to turn off the invitation within a few days with over 100 potential participants covering a range of business and consumer roles. When time came for the next round of benchmarking to start, we were able to turn on the recruiting survey the same day we signed the contract. A few days later we had all the recruits we needed scheduled into sessions. What a great tool to ease and speed up a critical step in a usability testing project.
Since our introduction of the online recruiting technique, and the design of their recruit invitation and survey, the federal agency has been applying the new technique to all of their planned usability tests. Before the project even begins, they can collect a pool of willing participants, speeding up the entire usability testing process.
We recommend you try online recruiting for your next usability test. Some tips:
- In Canada, make sure data will be stored within the country, to meet privacy regulations regarding the people who want to participate – we use FluidSurveys
- If you can use Ethnio, do so! It has invitations, screening, live intercepts and many other features.
- Qualify your participants by redirecting to a recruit survey after a Treejack or Chalkmark study
- Offer a suitable incentive
- Keep your recruit survey very short
- Have processes in place to manage and then completely delete the personal information of the applicants after the study