Setting Confidence Levels Based on your Information Needs

Confidence levels are statistical measures of how well the results of a quantitative study represent the population, given the parameters surrounding the study. These measures are often used as a short-hand method of describing how "accurate" a research project is in addressing a particular issue. Higher confidence levels are more costly to obtain than lower confidence levels, a particularly important consideration in times of constrained budgets. Given the trade-off, how can one efficiently establish a suitable confidence level?

There are three key elements to consider when determining the appropriate confidence level for a quantitative study. The first involves determining how the results will be utilized. If extensive, costly decisions could be made based on the results, a higher confidence level is recommended. If subtle adjustments or internal confirmation is the goal, a lower confidence level is appropriate.

The second consideration is the speed with which action is to be taken using the results. If immediate action is required, a lower confidence level is acceptable. General Colin Powell once wrote that he needed to be approximately 70% confident in his information before making a military decision during the Gulf War. Any less was too risky; any more meant too much time would be required, thus rendering the decision useless. Many American corporations always use 95% as their minimum threshold, without any consideration of timing. This can obviously be detrimental if time is of the essence with a project.

The third element considers corporate attitude toward risk, as established by senior management. Some managers are more enterprising while others are more risk adverse. The more intrepid decision-maker may be willing to trade the level of accuracy for timing. The more conservative executive may require "the entire story" before making any adjustments to a program. In short, managers will need to have confidence in the results before they use marketing research, so it is essential to assess their expectations prior to the project design.

Selecting the appropriate confidence level is indicative of an experienced and accountable marketing researcher. The questions that lead to that determination also encourage the decision-maker to better frame the research need, contributing to a more successful project. This should be the goal of every marketing researcher.