We are currently running through initial user testing for one of our projects. We held a live and online review so far (and more to come) of what is still a storyboard of the end product with the journey for end-users described. We want to validate that our design reaches the intended objective. Hence we took our project to our end-customer to receive valuable input. Here is a short list of what we would recommend doing with user research.
First level feedback
What a user test does it gives you first level feedback. Some of the comments made at the meetings were constructive and very useful. For us, it is also the first time the design goes beyond the initial project team and the decision makers from the client. It is the first time or target group will experience it, hence what they say is important to us.
Listen and probe for more information
It is always tempting when you receive feedback to explain your way out of it, but that is not the purpose of a user test. A user test is typically aimed at gathering all good, bad and ugly thoughts and ideas from your intended end-users. Probing to understand what would be better and what they really mean with their feedback is good. Defending and over explaining not so much.
How to decide what will change as a result?
When I look at user-feedback I always think about what will enhance the objective we are trying to reach and what will hinder it. The second question then is, is it feasible in our budget and timeframe to incorporate the changes suggested? If at all feasible then we should consider doing it, if not then we keep it for a later deployment or future use.
What must you implement?
Any point where you see a user losing interest or getting stuck purely because of your design, you must tweak to improve the experience. It may be a simple thing or more complex, but if they are already getting stuck at this early stage then we will for sure run into trouble later. If your user group has comments that more than one person brings forward, pay attention and check with other testers if they see the same issue. If they do, then you must consider changing it.
What can you discard?
I would find it really annoying if I gave valuable feedback but it didn’t at all get heard or incorporated. The virtue of inviting people for their opinion will give them a bigger interest in how your gamification design looks in the end. So when you are leaving things aside, I would say err on the side of caution and prioritise what is most to least important from a user experience perspective and that way decide on what you can discard or move to a later version. Items such as character looks, colours, specific words, are often the trivia that decision makers fall over but your end-user has no issue with and could be left aside.
No matter what you change or not, keep a respectful tone and environment so that everyone feels comfortable contributing. It is always a little scary to bring your precious design that probably took some time to be developed to a discerning group of potential critics. Take it from the perspective that their constructive feedback will make your end-result better.
