Sampling: The Importance of Good Sample
11 Sep 2019

What is Survey Sample? 

A sample is a subset of a population selected for a research study. A population is the total number of people in a group that researchers are interested in examining. For example, if you want to understand the attitudes of people who invest in the stock market in the United States, it would be too difficult and costly to question every person who trades this asset. As a result, quantitative research only takes a “sample” of the full population. 

Quality

The quality of your sample determines the quality of your results. As researchers and brands try to interpret insights, they must never compromise the quality of their methodology. We know that errors can come from poor survey writing. A good researcher also knows that a “bad sample” leads to inaccurate and misleading results.

Sample Size

So, how much sample is needed to project the attitudes of all US investors accurately? The key is to reduce the number of errors when mirroring your ideal population. The more respondents you have taking the survey minimizes the margin of error (confidence interval). However, at a certain point, increasing the number of respondents only reduces the margin of error slightly and means it is not worth adding the additional respondents. 

When there are too few respondents, there is a higher likelihood that outliers will impact the survey result, and your conclusions will be inaccurate. 

What is that magic number of respondents? Every ideal sample size is different because not all populations are the same. The best way to figure out the sample is to calculate the margin error.  Click here to use a margin of error calculator. 

Bias

It is also crucial to make sure that your sample is unbiased. Sampling bias occurs when the respondents selected are not representative of the population. The best way to avoid any discrepancies is to choose your sample randomly. 

Bias could also come in the form of a “bad respondent.” Someone might fit the population demographics, but fill out their surveys incorrectly to finish the study as quickly as possible or not be paying attention. As a result, it is pertinent to monitor your sample in the field and carefully review it after finishing the research study. 

Monitoring Sample

For online surveys, there are a couple of ways that you can check the quality of a respondent:

  1. Review individual survey results for patterns like straight lines and diagonal/patterned lines. These patterns indicate inadequate responses. 
  2. Make sure to monitor open-ended questions for gibberish or irrelevant answers. If respondents are not taking the survey seriously, they aren’t accurately portraying the population’s views and thus muddying the results. 
  3. How quickly a respondent finishes, the study is another indicator if the respondent was speeding through and took the survey accurately. 

Screeners

Given that you may be targeting a specific audience, screeners are necessary to determine that the appropriate people are coming into your survey. If you want your results to reflect the views of the population you are seeking, it is essential to impose restrictions on who is taking the questionnaire. For example, you need to make sure a study among stock traders does not include those who do not invest. Add questions to your survey that confirm that the consumer trades stocks.

If your audience is supposed to reflect a specific population in regards to certain criteria (e.g., gender, age, etc.), it is critical to monitor these attributes. A great way to control this is by implementing quotas. Putting these criteria questions in the screener will allow you to quickly determine if you need more sample for a quota or screen out full quotas.

Want to learn more about Market Research? Here are some blog posts to check out!

  1. Blockchain & the Market Research Industry
  2. I’m Just Not That Into You: Exclusivity or Bad Marketing?
  3. Is Your Research Stuck in the 1980s? Update Your Brand Tracker!
  4. Market Research Doesn’t Need to Be Boring – Improve Data Visualization

Sign up for our newsletters here!