With student, parent and staff voice you’ll get really important feedback from your questionnaire if it’s done right – but you’ve got to really careful in your design otherwise you’ll get bad results that will set you on the wrong track.
To avoid yourself falling into this trap, here’s a brief guide for how to put together effective questionnaires.
A questionnaire is a powerful tool, but it’s not the only tool to get meaningful feedback on things. Structured interviews, focus groups and data analysis can also provide many meaningful insights – you need to pick the right tool for the job. Generally data analysis is best when the data is reliable – it shows what people actually do rather than what they say they do. But data isn’t always available. Structured interviews can be a really interesting way of exploring a topic, and focus groups work well if there’s a strong social dimension to the question you’re asking.
Let’s take an example. You want to know what activities your students would like to do during lunch break. A classic question would then be to ask if they wanted to do Football, Basketball, Rounders or Cricket. Classic, but also potentially misleading. Those might be the only options we could think of, but it’s certain that the students will have other things they’d like to do as well. You can get around this by having an Other box (and I’d recommend you do this anyway). But to some extent that destroys the power of questionnaires – to give you the actual numbers of people who think certain things. Much better to do a bit of research before you write your questionnaire – chat to a few students and make sure you have an idea of the options they’d like, then make it scientific by getting exact data on how many people want what. Then you don’t fall into the trap of You don’t know what you don’t know.
Most people will know not to write leading questions like “Don’t you think homework’s rubbish?”, but it’s equally important to force people into making a choice. Otherwise you’ll just have hordes of people plumping for the middle option. If you use a point scale in your questions make it 1 to 4 rather than 1 to 5 so there’s no middle ground.
Longer answers are often neglected on questionnaires, and while they don’t provide easily analysable data they do give you an insight into why people think things, rather than just what they think. So only 80% of people hand in their homework. That’s interesting in and of itself, but it becomes actionable by senior management when you start to understand why that 20% don’t. Once you know it’s because they forget, or because they do too much outside school, or there isn’t a comfortable place to work it becomes something you can actually fix.
The classic technique of cosmetics firms “90% of women thought Pantene gave them shinier hair” … based on a survey of 20. You find me enough sets of 20 or 30 people and I reckon I can get the questionnaire to say pretty much anything. The data gets more reliable the more people you ask. If you don’t get many responses it will still give you valuable information on the kinds of things people can think, but not what they actually do think and in what proportions.
I remember overhearing at a governing training session about a governor from a primary school who was trying to find out the best time for their parents evening. So they interviewed parents at the school gates, and they found the best time was during the day. They scheduled it for then and had a small revolt on their hands. The working parents who perhaps picked their children up a little later or had siblings walk them home weren’t present, the school got a skewed result and ended up drawing the wrong conclusion. You can never quite rule out a biased sample (and there’s so many ways to be biased – can all the parents even speak enough English to take part?) but you just have to try and to avoid it wherever possible.
Who would have thought a questionnaire could be so complicated? But even after all this work people can still interpret questions in unexpected ways. The best way to go is to test the questionnaire on a small sample first (say 10 or 20 people) just to make sure you’re getting the kinds of answers you were looking for.
That’s it! Questionnaire nirvana!
For those looking for a bit more information there’s a really good (and short) book on the topic you can get here.