Many human resource professionals and academia, often ask the question has gamification been validated. There are several ways of looking at this question.
Commercial validation
When we look at the different ways of validating a product or service, we could look at commercial validation first of all. Commercially the first indicator is whether there are buyers for gamification services and gamified products, secondly are they satisfied with results and third are they repeat buyers. I can say with evidence from my own business that commercially we have a market and there are repeat buyers. The more prepared the buyer is to allow us to do our best work, the better the results. Our best work happens when rigorous user research, creative gamification design and implementation followed by small iterations culminate to achieve the results.
Business result success
If we look at it from a business results perspective, then the achievement of business goals is most important. In our way of working with clients, the goals should be measurable and clearly related to a specific business process. The more refined this is, the easier it becomes to work towards them. If we increase learner engagement or employee engagement, then this should also translate into some end-user behaviours and actions that we can measure such as retention of knowledge, reported application of knowledge, increased job satisfaction, etc.
I know from judging projects in competitions, that the measures to prove project effectiveness and results vary widely. Until we have standards across the board it is just my word against the word of another person. We can use return on investment, sales increases, net promoter scores, happiness index and blend them all together, but linking it directly purely to gamification is not easy to prove. Most businesses don’t operate in a vacuum.
Research validation
In research terms validation needs to produce consistent outcomes over time, in a variety of conditions and for a variety of people with different backgrounds, environmental circumstances and mental states. If we cannot replicate results, then the validity is not proven and makes it hard to be applicable in multiple places. Unless the results are over 70% consistent, the tools are not considered adequately validated.
The studies dedicated to proving gamification struggle to find approaches to be similar enough to start with. It seems to me that it is in fact easier to find evidence about game effectiveness than there is proof on gamification effectiveness. I could be wrong but that is just my observation.
Gamification requires more validation and research
I would love to come to a place where every project also has an associated piece of research that can validate everything we do and suggest over time and aim for the 80 to 90% consistency mark. I can say that to date we cannot make this claim. Firstly because research is not widely available even if it is increasing and secondly because gamification (especially in our experience) has taken many different forms for different clients, so finding the commonality may not be so easy, hence replicability and consistency are in question. We may have good reasons to differentiate, but we may then not have a given formula.
In a way, I think this is the dilemma for growth in our industry unless we can prove what works consistently in given business processes with some sense of security then we will keep stumbling over these questions. I like to build an evidence-based gamification design and track it the data on behaviours in order to come to the best outcomes for all our clients. With personalisation high on the agenda and very relevant to end-users in an on-demand world, there may also be a need to review validation to be more granular and personalised too.
At company level and specific client level, I can prove the results of gamification and we can measure how they are valid for a larger group of people. Because our projects are bespoke and vastly different in industry, goals and client culture, I would say even comparability can at times be an issue. It may be worth grouping evidence of a variety of sources to find the common ground, which is something we will be looking into for 2019 and beyond.
Leave a Reply