When we conduct usability testing, we always have a plan – a discussion guide or testing script that has the entire session mapped out to the finest detail (how to greet the participants, what tasks to give them, etc.). However, sometimes, you need to change your plans in the middle of testing, as I found out myself.
I recently finished a usability testing project for a mobile website. It was a pretty awesome-looking site with lots of features from its desktop cousin. After the first 2-3 sessions, I started noticing a clear pattern: everyone was having trouble finding things on the mobile site. There were so many features on the site that users were getting lost trying to find things.
It was evident that the issue was with the information architecture of the site (i.e. how information was structured in the site). I felt like there was no point getting the participants to do the same tasks and uncovering the same issues over and over again. So, I decided to conduct a card-sorting activity during the remaining sessions. During a short break between sessions, I quickly printed out cards with labels from the existing site.
Before I showed the site to the participants and asked them to conduct the tasks, I gave them the pile of cards and asked them to group them into categories they deemed logical. After the grouping exercise, I then asked them to give a name to each of the groups that they came up with. The whole process just took 10 minutes and after asking a few participants to conduct the task I could see a clear pattern about how participants were sorting the cards. Not so surprisingly, the IA of the mobile site was very different to what the participants had come up with. No wonder they were having difficulty finding things, the site IA was totally different to what the participants had expected.
The clients were fascinated by the findings from the ad-hoc card sorting activity. It helped them get into the heads of their users and see the system from a users’ point of view. The clients were pleased to receive the extra deliverable: a new IA based on actual user input, and the good thing from a project management point of view is that the card-sorting activity did not cost us (or the client) any additional time or budget (10 minutes for the activity and a couple of hours to analyse the findings and restructure the IA).
So it got me thinking…what would I have done differently next time? Maybe I could have prepared cards before the first session just in case I needed them or perhaps utilising a digital card-sorting tool like OptimalSort. Online tools are useful as participants would be able to sort the cards on their computer screens and the clients would be able to watch the results live via screen sharing.
Moral of the story: Don’t be afraid to change your course if you’re heading in the wrong direction. Always keep asking yourself whether you’re getting rich insights to answer the research questions and also are these the most important questions to be asking? If not, what other methods can you use to find the answers? Planning and being adaptable is more important than a plan (that doesn’t work).
Had you had to change course in the middle of a usability testing project? What would you have done? Feel free to share your experiences and thoughts in the comments below.