The No BS Guide to Market Research Sample Sizes
The use of well-formulated and actionable market research is often the difference between success and failure with product line expansions, new businesses and entry to new markets. The benefits of a carefully thought out market research study are plentiful including:
- Better understanding your target customers
- Sizing your market
- Measuring how well your product or service idea might perform in the market
- Reducing your risk profile for a big decision
However, understanding how to conduct an effective study and deciding on the appropriate market research sample sizes is paramount.
In this article we will pull back the curtains on sample size and provide a comprehensive overview of sample size, what makes something statistically significant, and how you can apply these learnings in your next market research project. We will demystify the terminology and provide useful guidance so you can plan your next market research study with confidence in your choice of how many people you should survey.
Market research sample sizes lingo - what do the terms mean?
Market research surveys are a great tool to help you understand your potential customer bases behaviors, interests and willingness to purchase.
A survey with an appropriate sample size or number of respondents is exactly the proof you need to make critical decisions when entering new markets but there are a number of elements that make a survey valid.
Here are a few of the most important terms you need to know:
What is a Sample Size?
The first thing that many people will want to know when you present a study is the sample size in market research. This refers to the number of people who participated in your study.
Having a good sample size in market research allows you to draw appropriate conclusions about how the larger population would behave. But keep in mind that having a larger sample size doesn't necessarily correlate to more accurate findings.
Many leadership figures will look for as high a number as possible when it comes to sample size, but this is not always necessary.
Market Research Survey Cell
A cell or sampling cell is quite simply one particular group of people who make up part of your sample target market.
For example, let’s say you are creating a new hair tie that you expect to be popular with women with long hair and 20-30 year old men with man buns. The 20-30 year old men with man buns would be one cell and the long haired women would be the other. Every business critical cell that you want to analyze data by after your study is over should be considered up front so you can build your sample plan from 'the bottom up'.
Statistical Significance
What does it mean when something is statistically significant? The very purpose of running market research is to reduce chance for failure to occur. This is the same for the market research survey itself.
You typically cannot survey every single member of your target audience to know with absolute certainty that the opinions collected reflect the market. The aim is to minimize the amount of people you survey that do not echo the sentiment of the majority, but you can be unlucky.
A study with high statistical significance means that the findings obtained were not as a result of unluckily surveying the outliers of your target market.
In more common terms, the higher the statistical significance, the more likely it is that the opinions of the majority are captured in your study.
For example, with a 95% statistical significance level, I can assuredly say my survey results are real and will tell me what my target customers would do in a given scenario.
Margin of Error
The vast majority of studies in research are given credibility by the ability to repeat them and arrive at the exact same or a very similar result. How close the result you would get is where your margin of error lies.
For example if I had a margin of error of 5%and conducted a survey 100 times, I would expect my results to be within 5%every time. The lower your margin for error, the more certain you can be that the results reflect the entire possible population.
Confidence Levels
Statisticians can sometimes communicate as though they are in The Matrix without realizing that not everyone speaks with the same degree of understanding as they do. Often times when a poll is reported, the confidence level is included in the results. A survey might report a 95 percent confidence level. What does this mean, though? A confidence level tells you exactly how sure you can be that the results from the cell surveyed accurately reflect the overall market.
Confidence levels are quite important in research and you should typically be looking for a 90-99% confidence level in your study results.
The most commonly used and accepted confidence level is 95%. Why? Well, because many scientists use it in academic writing, and because it is generally considered 'accurate enough'.
What are some rules of thumb for sample size calculation?
The world of research and academia is entrenched with many conflicting rules and myths. We are aiming to simplify these and address what benefits the end user the most. Here are some sample size calculation rules of thumb.
1) Size matters, but so does business reality
A larger sample carries more power to predict the actual behaviors of your target market. In black and white, if you had the opinions of every customer you would obviously know all the possible outcomes with your launch.
Furthermore, market research for management will often come with pressure to have the highest participant rate for your sample size so they can have more certainty.
However, 100% participation is almost never possible so avoid obsessing over the largest sample size you can possibly get. The early bird catches the worm and timing is important. Most businesses have time + budget constraints that will not allow for market research that drags on. Making achievable, realistic and time conscious goals are vital.
2) At a certain point, spending more money will not drastically improve accuracy
The law of diminishing returns is the point at which your level of benefits begins to reduce for every further dollar spent. This is true for market research sample sizes.
The cost-benefits ratio for sample sizes off begins to narrow at a certain point. The intersection of confidence levels, margin of error and costs is generally the point to confidently accept that you have captured the response of your target market.
Continuing to spend money beyond this point is very much personal preference but as can be noted from the infographic below, it is not always necessary or financially advisable.
3) Your risk of error/business loss should guide your choice on sample size
Budgeting is a major part of any business and it is no different when it comes to market research sample sizes.
Certain demographics are more likely to be expensive to survey than others. For example, a large sample size of dentists will be less accessible than surveying the general grocery store using population. Understanding the accessibility to these populations will give your study costs a foundation.
However, more importantly a good rule of thumb to follow is that scale and risk of the decisions dependent on the study should be comparable to the market research sample size. If you are on the verge of making the most crucial business decisions, you should do as precise a job on market research as possible. If you are just looking to quickly check your gut on a minor decision, then you can likely get away with a smaller sample size.
4) When in doubt, a safe sample size formula is ~400 per cell
As we have shared in this article, there simply is no hard and fast rule when it comes to the correct sample size in market research and it all depends on the participants required, aims of your study, the scale and risk attached to the outcomes and the budget set aside for research.
At Starlight, we have found that market research sample sizes of 300-400 for each cell command a great balance between the cost-benefit trade off with low margins of error and high confidence levels.
In Conclusion
Deciding on the right sample size for your market research survey requires care and logical thinking, but it's not rocket science.
You only need to have as many people take your survey as necessary. For most purposes a sample of between 300-400 will do the trick.
Generally, there is not a need for an excessive amount of sample unless the choice you are making requires a very high degree of accuracy, or you are doing specialized research like a market segmentation.
Still have a question? Need help scoping a study of your own? We'd love to hear from you in the comments below or feel free to contact us directly.