Survey Design

Surveys are used in a variety of ways to determine the perceptions, attitudes, and buying preferences of customers as well as to assess employee issues like job satisfaction.  When properly constructed and administered surveys can be a highly reliable and valid source of information.  So much so in fact, that survey is often admissible in court.  The survey design process consists of several steps that must be properly executed in order for the survey to be valid and reliable.

Establishing the Goals and Objectives of the Survey

Begin by asking the question, why are we conducting a survey?  Is it to determine customer or employee satisfaction?  Is it to assess customer or employee retention?  Is it to determine if your marketing materials are communicating the right message about your products?  Is it to assess the attitudes about a product or service?  List out your objectives for the survey.  There may be several related objectives you hope to address within a single survey.  Once you have your objectives identified the next step is to determine the types of information you need to assess.  Set a timeline to design, administer, and analyze the survey.  Make sure there is also a time point for all reporting to be complete for presentation.

Determining the Sample and the Sample Size

The sample is that specific portion of your target market population that you will survey.  The sample size is based on either the population size, to ensure you have a representative sampling of the population, or based on a power analysis to make certain the statistical analyses represents a good chance or likelihood to detect relationships between your survey items.  Care must be taken not to select biased samples like snowball sampling or samples of convenience.

Choosing the Mode of Survey Administration

There are several different ways to administer a survey.  Surveys can be administered over the phone, in person, by email, or direct observation.  You need to consider what modes will deliver the best results and then assess those modes for cost, ease of administration and how the data can be recorded.

Designing the Survey and Creating the Questions

Begin your survey with a brief summary of why the survey is being done, what is being assessed, how long it will take and the fact it is completely voluntary to participate.  Survey questions can take several different forms.  These include:

  • Multiple choice responses
  • Open-ended questions
  • Closed Yes/No/NA questions
  • Essay style questions (free form responses)
  • Open ended numeric as in a ratings scale (on a scale of 1-10…..)

The best way to structure a survey is to follow the four accepted criteria for question design:

  • Be brief
  • Be objective
  • Be simple
  • Be specific

It is important to understand that the type of questions and the way responses are measured (ratings scales) have a direct impact on the type of analyses that can be conducted on the data.  That means it is critical to choose your questions and rating scales very carefully.  It is also important to think carefully about the order of the questions and how the questions are asked.  Framing or phrasing the question positively can elicit a different response then phrasing the same question negatively.

Pilot Testing the Survey

Before spending a lot of money to implement your survey test pilot it first to make sure participant’s understand the questions, the data you are collecting is reflective of your objectives, and sample the results to make sure the data can be downloaded in the form you want to receive the data.  Correcting mistakes early will ensure the right results and save you from wasting time and money.

Analyzing the Survey and Generating A Report

After the survey is administered have competent statisticians analyze the responses, with both descriptive and inferential statistics.  The analyses should include relevant tables and graphs so the findings can be easily visualized and understood.  There are several good online survey administration tools like Survey Monkey, Qualtrics, and Fluid Surveys that can do basic descriptive reporting (means, frequencies and percentages).