By Greg Lipper • April 6, 2018Comments

Mobile Survey Optimization Best Practices

If you are reading this, you already understand that mobile phone delivery of surveys offers significant advantages in terms of speed, ongoing engagement, time/location sensitivity, and ease (for both the participant and data consumer).

However, mobile is not a silver bullet to use for all surveys, as discussed in When to Use Mobile Surveys. For cases where mobile is appropriate, success depends on you following a few basic best practices. 

Consider when television came on the scene in the 1950s. Successful programs were made in a shorter format, and with more frequent viewing and enhanced camera views in mind. They didn’t just film theatre or movies. They made shows suitable to be shown on TV. The same analogy can be made with surveys designed for F2F translated for use on mobile. The platform has changed, so the way surveys are designed must too. 

If you take a survey written for a traditional Face to Face (F2F) or even online panel survey and cram it onto a mobile phone, it will annoy responders, which will result in high incompletion rates and low quality responses.

The major differentiating factors are time and space. Screens are smaller (and of a huge array of sizes and resolutions). Remember, too, that when people are using mobile devices, they are often on the move, so a survey must be designed for frequent brief interactions. 

These two factors demand simplicity of design. Neither mobile screens nor mobile users handle complexity well. Questions must be answerable without deep and lengthy thought or complex input. 

Your 5-point mobile survey design checklist: 

Here are our suggestions for mobile optimization for surveys: 

  • Mobile only. When possible, start from scratch developing a questionnaire that is not just mobile-first in perspective, but mobile only. Subjecting F2F or online panelists to mobile-centric surveys is just as problematic as forcing a non-mobile survey onto a mobile platform. 
  • Brevity. The industry trend is to cut survey length to 10 minutes. We think that this is still 10 times too long. Ask 100 people for 10 minutes, and maybe 10 will comply. Ask 100 people for 1 minute and maybe 50 will comply. If you make it fun and rewarding, they will do it 60 times a month. Break your survey down into one-minute interactions – roughly 5 questions. This is the format we use on Happi. A “Smile” on Happi is the act of a user selecting a prize they want a chance to win and responding to a 5-question survey. Then they select the same prize (or another) for another chance to win and answer another 5 questions. Those questions can be the next section of the same survey campaign or an entirely different one. The point is, give them doses of only 5 questions at a time. 
  • Use images and video sparingly. Yes, people watch videos and images on their phones all the time. More time, in fact, than they actually speak on their phone. However, unpredictable network conditions, the huge array of screen conditions, and responders’ eagerness to get back to their entertainment content must be considered. On Happi, each question can be embedded with a hyperlink, which can display an image or video in a new window within the app. Each image has to be optimized for mobile display and the videos should be no longer than a minute. Many users will refuse to download this content when not on wireless, so the more content you force a user to download, the slower the completion time will be.
  •  Simple formatting. Italicsbold altering font size ancolor present differently across devices and are perceived differently across cultures. At best, they can be distracting. At worst, they can annoy or offend. Keep formatting to one font in one size and in one color. 
  • Keep open-ended questions simple. How much do you like typing with your thumbs, on a train, with a bag in your hand? Think of that when designing your survey. This is not an F2F interview. The responder might not be seated or have a standard keyboard. You need to make it harder on yourself to design a survey that tells you what you need to know while keeping it easy on the responder. They are in control, not you. They can switch to another app or activity with the slightest annoyance. Don’t annoy them. 

The wrong way to do a mobile survey 

This survey was designed for an F2F interview and, as we will go through later, it doesn’t translate to mobile use. 

1- Firstly, can you tell me whether you or any of your family work in any of the following industries?

2 - Who is in charge of cooking for your household?

3 - Who usually decides what brand of ingredients to be used in your household?

4 - Which of the following ingredients do you use for cooking?

5 - Which of the following products do you currently use for cooking?

6 - Suppose you can buy 11 packs of any of the below cooking products. Which ones will you buy and how many packs will you buy?

7 - Please tap here to see a product description. Please read it carefully. Then, suppose you can buy 11 packs of any of the below cooking products. Which ones will you buy and how many packs will you buy?

8 - Which of these phrases best describes how likely you are to buy this product if it is available in the store? You can tap here to is the ad again if you wish.

9 Can you let me know why would you not consider buying this product if it is available in the store? You can tap here to see the ad again if you wish.

10 - Can you let me know why would you consider buying this product if it is available in the store? You can tap here to see the ad again if you wish.

11 - What will be the role of this product in your cooking? You can tap here to see the ad again if you wish.

12 - Which seasoning ingredient is it going to replace? You can tap here to see the ad again, if you wish.

 First off, this is way too long for a mobile survey. And, as you will see below, unnecessarily long.  

Don’t include unnecessary questions

The first question is probably unnecessary for a mobile survey as the user has already been profiled for employment status and industry. But even if not, beginning with “firstly” presumes that this is the first question of the day the user receives and “tell me” presumes a 1-to-1 discussion. It is better to word it as “Please confirm if you or a relative work in any of these industries”. 

Break the survey into two campaigns: qualify and query

There are many qualification questions to this survey and the client was interested only in responses from people who answered one of the questions in a specific way. Even though the campaign was structured to disqualify responders along the way, until the survey is run, it is impossible to predict the incidence rate, and, therefore, completion time or number of interviews or responses necessary to complete the campaign.

 Don’t make questions too complex

Question 7 is extremely mobile unfriendly. First, the instructions are complex. Second, you are asking the respondent to type 11 products and the number of packets they will buy. Question 8 then asks the user to read a long product concept and, while remembering the concept, again enter how they would allocate the 11 packs. Even with survey application guidance and controls, it’s a hassle for the user. This approach begs for high drop-out rates and low-quality responses. Yes, if it were an F2F interview, the interviewer could coach the user and ensure compliant responses, but this isn’t F2F. If you make it hard on the user, they will move on to something easier.

 The right way to do a mobile survey

 Here is how the same insight could be gained through a mobile optimized survey: 

1- Have you purchased [product] in the past 3 months? Yes/No

2 - Please see this product concept. Would you be likely to buy this? Yes/No

3 - Would you buy it in addition to or instead of other products? addition / replacement 

This campaign results in three segments of interest:

1) Those who would not buy

2) Those who would buy as a replacement

3) Those who would buy as an addition

 Segment your audience

Now, we can analyze the demographic profile of each segment to determine if we want to create sub-segments of each. We can then send very specific questions to each segment. 

This keeps the process short, sweet, and relevant to the user. It also lets the client know the available segment size, estimate completion times and costs, and, if they wish, keep the first campaign going to continuously grow the qualified panel for future studies. 

Book a free consultation

We hope this information was helpful on a conceptual level. If you want to get tactical and develop a mobile optimized strategy for your next survey, please fill in our contact form to request a free consultation.