Mon, 01 Mar 2010

How to make a good survey...

...or at least not annoy those attempting to respond to it.
For whatever reason, I've been asked to complete an unusually large number of surveys recently. I get customer surveys from suppliers I deal with at work and as a student I frequently get asked to participate in questionnaires to assist other students in their research. If I feel a survey is reasonable I'll happily respond to it - but I find myself getting annoyed with surveys for the same reasons over and over again. Some surveys have been so annoying I've abandoned them half way through, wasting my time and probably not providing any feedback to the surveyor.
Remember that most people who fill in surveys are giving of their limited free time to provide you with some useful feedback. Respect their time and good will and try to make things easy for them. Here, for reference of anyone who would like me to respond to their survey, are my top tips for making your survey more likely to be completed and returned:

Be up front and open about what information you are trying to gather and why you are trying to gather it. While multiple-choice answers are unlikely to change depending on the reason for the questionnaire, free-form text answers are likely to be quite different. I'll except those doing psychological research from this rule, since they typically have their own reasons for asking certain things in certain ways.
Be honest about how long it will take. Give an estimate at the start so people can decide if (a) they want to devote that much time and (b) if they have that much time available in one block now, or should wait until later to start. Estimate on the high side. Ask three friends of yours to take the survey and time them - see how long it takes them to fill it in and use that as a gauge. If it's a multi-page survey, provide a progress bar. Make sure the progress bar is honest and realistic about progress at every page.
Use less pages, particularly on web surveys. Remember that ever time I have to click next wastes time as my browser has to download a new page, render it, etc., etc. In particular, stop the foolishness of asking one question per page. If I wanted to click next all day, I'd install Windows software for a living.
Don't make questions compulsory. Allow me to progress without answering a question. It might be that I don't have an answer, don't want to give an answer, or that I want to come back to it later.
Don't ask for personal details, in particular things like date of birth. If there is a genuine reason you need such information, state why you need it and don't make it compulsory. Any survey I get asking me for date of birth is ignored. If you are building up a statistical model that requires the person's age, ask them that - or ask them their year of birth, or even the month and year. If you need such information, explain why and detail how you are going to protect such personal information from abuse (e.g. By only storing anonymised data, or similar.)
Get someone else to check your questions. For preference, get someone who does not intimately understand your research or line of work to check your questions. See if they can understand all your questions, and indeed that their understanding is the same as yours.
Make it clear whether you expect one or multiple answers to each question. For online surveys, radio buttons should be used to allow selection of a single option, and check boxes allow selection of multiple options.
Do not make free-form text answers compulsory, ever. By all means allow free-form text answers or extra 'any other comments' style questions at the end, but do not make them compulsory. In particular, do not echo one recent survey which asked "What made you pick option X?" after each question - and forced some answer to be given before proceeding. This is a classic example of how to erode any good will the survey respondent feels towards you.
Thank the respondents and most importantly, tell them what's going to happen to the information and how it will help you. A poor survey will thank the respondent for answering their questions and leave them wondering what, if anything, will happen next. Consider offering to collect an e-mail address and send them a copy of your research when it's ready for publication. If you're a commercial company, offer to provide an update in a reasonable time frame on the results of the survey and what improvements customers can expect because of it. If the respondent declines to give an e-mail address or contact details, consider giving them a URL where they can find more information about what will happen with the information they've just given you. People like feeling useful - make your survey respondents feel useful!

These tips are not all encompassing - far from it. They do capture some of the most irritating and repeated failures I've seen in a sequence of recent surveys. For some more general tips on writing good surveys check here, here or use the power of your favourite internet search engine to find innumerable other resources on the subject. Consider reading a book, made of paper on the topic; there are many. Stick to the guidelines above though and you're at least likely to get a prompt response to your survey from me.
Happy surveying!
posted at: 23:23 | path: | permanent link to this entry