Survey design tips & tricks – Part 2

After we had published our “Survey design tips & tricks” recommendations we received a lot of feedback asking if we can continue the list and give some more examples of what one needs to take care of when creating a Limesurvey survey.

So we got some coffee and had a team meeting discussing further issues worth mentioning. Here are 10 more tips & tricks:

11. Be careful when using matrix questions

Matrix – also called array questions – should be used carefully. They are hard to answer for the user if you add too many sub-questions. Recommendation: Repeat headers at larger matrix questions to help users answer such complex questions and keep the sub-questions as short as possible. This setting can be found at the advanced question settings of each array question.

12. Mark keywords in bold

Within lengthy question or answer texts you should mark important keywords in bold. Users are lazy and don’t like reading lengthy texts or explanations. Highlighting the most important parts helps them and improves the completion rate.

13. Force users to agree to terms and conditions

Some surveys may require that the user agrees to certain terms and conditions. There may also be surveys for which you may want them to mark that they have understood important fill-out notes like “I have understood that…X, Y, Z”.

For such cases use multiple choice questions and set the “max answers” question attribute accordingly to force users to check the box of each item (“Please mark all”).

Alternatively, you can add the terms and conditions at the start page and rename the Limesurvey navigation buttons to “I agree / I do NOT agree”.

14. Use conditions/branching

Limesurvey support complex conditions and you should make use of it. No male people want to be asked for their favorite make-up brand and probably only some very few women want to share their opinion about their favorite beer. So how about adding a gender question and only ask such questions the appropriate target group?!

You can even use conditions based on additional token attributes if you are designing a closed survey and information about gender or age is available up front.

Asking users questions they can’t/don’t want to answer is a survey design worst case.

15. Use screen-outs

If you are interested in data of e.g. young people only, don’t waste other peoples’ time. Add a question asking for their age at the very beginning and set a “zero-quota” to screen out people selecting an answer out of range of your preferred age.

16. Use real time data validation

Instead of cleaning up data later make use of Limesurvey’s real time data validation features:

  1. Email addresses can be checked for valid syntax using a regular expression.
  2. Date ranges can be limited by setting a min/max date. Such date ranges can even be set dynamically. Using the Expression Manager you can e.g. set the max date to today’s date.
  3. For numeric input questions you can also set reasonable min/max limits e.g. when asking for height/weight.
  4. When asking for percentage values, set a max value of 100 and make sure the sum is not greater than 100.
  5. You can reference previous answers to set the max of the current question to a value that was entered at a previous question

17. Always use similar scales

You should always use similar scales within a survey. Users will be confused if question 2 uses a 1-7 scale, question 5 uses a 1-5 scale and question 10 uses a scale ranging from 1 to 10. It also makes comparing data far more difficult. So when designing your survey think of scales which you can use for all questions.

In addition to that you should also make sure to never change the direction of scales. Only for certain psychological experiments that may make sense but users may not even notice that so when doing so show them according hints.

18. Use an even number of items for scales

Scales should always use an even number of answers to force users to give a tendency. At a 1-5 scale people tend to marking the “3” answer in the middle which doesn’t really tell you much.

Instead, use an even number of items at your scale like:

“very good” – “good” – “bad” – “very bad”

19. Offer “save and resume later” option

Especially for lengthy surveys you should enable the “save and return later” feature at “Survey Settings” -> “Notification & Data Management”. Users may be interrupted, may have to leave suddenly or may receive a phone call. Offering an option to save their answers prevents them from just closing their browser or suffering from a session time out.

20. Give feedback about the user’s progress

The user should always be aware of his/her progress when answering a (lengthy) survey. Thus, you should always show a progress bar and also offer additional information like “Just some final questions about XYZ and then we are nearly done.” or “This is the last page of questions, we are nearly there.”