Working through the challenges of qualitative research.

We regularly pre-test many packaging systems through quantitative on-shelf assessments. On those occasions when new packaging systems perform poorly in our studies, we’re likely to hear a familiar refrain, “They loved it in the qualitative...”
Qualitative research is rarely touted as “predictive”, but it is intended to lead marketers towards effective packaging solutions. However, qualitative findings are often misleading, which has driven some marketers to abandon traditional group sessions for one-on-one personal interviews (IDIs) or Internet-based screening studies.
Our perspective is that there is significant potential value in qualitative packaging research. Tied to the depth/flexibility of questioning, it allows a hands-on learning experience for marketers and designers. However, there’s no question that qualitative research is often misused.
With that thought in mind, we’d like to highlight three common sources of misleading feedback-and offer suggestions to minimize their impact.
  • The Emphasis on Appeal/Aesthetics. First and foremost, qualitative study findings are often a reflection of “what people like,” rather than a function of communication (driving appropriate product expectations, embodying desired brand imagery, etc.). While aesthetics do matter-and research certainly should identify design elements that are alienating or confusing-the most appealing design is not always the most effective. Many qualitative studies quickly disintegrate into beauty contests in which shoppers pick their favorite and then claim that it also happens to be the option that best supports brand imagery and product communication.


  • Presentation of Limited Stimuli. What comes out of qualitative research is largely a function of what goes into the studies. Here, we often find two types of problems. The first, and most obvious, is stimuli that simply don’t “do justice” to the packaging concept. For example, when two-dimensional renderings are used to illustrate new packaging structures or delivery systems, they typically fail to convey the tactile feel of the packaging-and thus fail to drive strong visceral reactions from shoppers. A second (and more subtle) issue is the range of concepts brought into qualitative research. Many marketers are inclined to avoid showing “further-out designs” on the premise that the company would never actually introduce these ideas. However, the value of qualitative research often comes from understanding boundaries, which can come out of “going too far” with some designs-and understanding when/where you have compromised brand equity and/or alienated shoppers. When qualitative stimuli stay too close to current packaging, learning is sacrificed, and the result is often overstated minor differences among options, which won’t have an impact in market.
  • Lack of Shelf and Competitive Context. Perhaps most importantly, qualitative studies can be misleading because they don’t address the key factors that drive in-market success. In-store, the fundamental reality is that packaging must work within a cluttered shelf context-and that packaging typically has only a few short seconds to gain attention, differentiate from competition, lead shoppers to the right product-and ultimately close the sale. In qualitative research, respondents often encounter packaging concepts outside such a competitive context-and they may spend up to two hours discussing alternative concepts. Inevitably, this dichotomy will lead to some misleading findings, as shoppers will often use this extra time to overthink concepts, or place themselves in the role of amateur designers or brand managers.
Best Practices: Presenting and Probing The good news is that these minefields can be minimized. The cardinal rule and guiding principle is to keep shoppers in a shopping mindset. We want their input as people buying products, not as replacements for our marketing and design teams.
To encourage this mindset, it is critical to:
Present packaging concepts within competitive context. When concepts are shown next to competition, it is more realistic to the shopping experience and allows us to gauge competitive differentiation, which is critical to in-store success. It also encourages people to think like shoppers (deciding which product to purchase) and, thus, to focus on the role of packaging in supporting brand imagery and product benefits, as opposed to aesthetics. Specifically, we’d recommend that alternative concepts be presented and discussed sequentially, within a competitive context.
Minimize side-by-side design comparisons. Conversely, when shoppers see alternative design systems for the same brand simultaneously on a side-by-side basis, it immediately takes them out of a shopping mode and encourages a focus on aesthetics. These side-by-side comparisons also tend to overemphasize the differences between options, and thus overstate the potential impact of a relatively minor design change (which may never be noticed on a competitive shelf). While these side-by-side comparisons can add value in clarifying distinctions, we’d recommend that they come at the end of a discussion-and that steps are taken to discourage exercises that encourage shoppers to select their favorite elements from different packages (“Take that cap from design B, and the logo from design F…”)
Show concepts in their totality. Often, a primary objective of qualitative research is to understand the contribution of individual design elements, in order to optimize them. However, it’s also important to remember that shoppers encounter packages in their totality and that the “gestalt” of a design is often more than the sum of its parts. Therefore, we recommend consistently showing shoppers full packages and then probing to understand which design elements are driving reactions. In other words, the process should be one of “deconstructing” packages (to understand key elements), as opposed to “building” packages on an element-by-element basis. If the objective is to isolate and understand a single design element, an effective strategy is to present several complete packages in which only a single element varies.
Show concepts alone (without marketing support). Finally, it’s important to remember that packaging typically has to work on its own, without the benefit of advertising or point-of-sale support. In this day of fragmented media, it is simply unrealistic to assume that shoppers have recently seen your commercial, or even that they have any awareness of a new product prior to seeing the package in store. Thus, when shoppers are shown concept statements before exposure to packaging, it almost inevitably biases the research. Instead, we’d recommend listening to whether shoppers are spontaneously “playing back” elements of the brand positioning from the packaging. A concept statement or advertisement can then be introduced at the end of a discussion, to confirm alignment with the packaging and discuss any inconsistencies.
What Not to Discuss. In terms of discussing packaging concepts with shoppers, the primary focus should be on what not to ask. Simply put, there are some questions that are not consumer issues-and many others that are likely to provide misleading feedback:
Strategic branding issues. Certain issues, such as underlying positioning or branding relationships on packaging (master brand vs. co-brand or sub-brand, etc.) should generally be guided by brand strategy, rather than consumer reaction or preference. In this realm, the role of consumer feedback should be to determine if the packaging is conveying the intended strategy, rather than overturning the strategy itself. Thus, if shoppers are saying that the logo is too small, the reaction should be to understand what is driving this comment rather than reconsider the product’s branding strategy.
Shelf visibility. Across studies, we have consistently found that shoppers can’t accurately gauge which packages they would see (or miss) at shelf. Even when people are shown shelf sets, what they claim to notice or recall differs significantly from what they actually see. Therefore, asking shoppers if a design system is eye-catching is likely to provide very misleading answers. And while presenting new concepts in shelf context is valuable to gauge competitive differentiation-and perhaps to uncover and explore issues of confusion and shoppability-this should not be mistaken for an accurate gauge of visibility.
Pricing issues. Pricing is perhaps the single area where qualitative research, or any form of direct questioning, is most likely to provide misleading feedback. When asked if they would pay more for new packaging, many shoppers will understate their willingness-or place value only on “hard” functional benefits (product protection or ease of use), as opposed to “softer” advantages (a more appealing visual presentation or clearer communication).
However, experience suggests that a more compelling appearance can drive higher price expectations and value perceptions and that price increases can often be “passed along” without impacting sales. Certainly, qualitative research can be used to explore pricing issues, in terms of uncovering whether new packaging concepts are perceived to add-value and in understanding how best to convey new features/benefits on labeling.
However, these insights should lead to appropriate quantitative research. It is important to know where to stop and avoid getting too specific about pricing implications (“Is this worth an extra 10 cents?”).
Perhaps most importantly, it’s important to discourage very specific design recommendations. The value of qualitative research comes in learning how shoppers react to packaging concepts and uncovering why.
When shoppers become “art directors,” the results are often packaging systems that do not perform in-market because they are divorced from marketing strategy.
Getting More from Qualitative How can we leverage the considerable value of qualitative research, while minimizing its potential to mislead? The trick is less about adhering to a specific methodology or sampling approach and more about investing in appropriate stimuli and following a set of best practices to guide how concepts are presented and discussed. However, as with virtually all consumer research, it often comes back to how feedback is interpreted and applied. For this reason, we’d also offer two guiding thoughts regarding the interpretation of findings:
Focus on themes, rather than “winners”. In speaking with shoppers, the focus should be on uncovering shapes, visuals and messages that resonate and differentiate, along with sources of confusion, misinterpretation and alienation. When the research is instead centered on the objective of picking one or two winning designs and/or driven by preference rankings, these elements can be missed, as they are lost within less compelling design systems.
Don’t apply shoppers’ input too literally. It’s also important to avoid being held captive by the literal words of shoppers, particularly in terms of specific design recommendations (“Make it more yellow”). Instead, the trick is to uncover the issues driving these comments and interpret their potential implications. When conclusions are framed in terms of issues and objectives, instead of mandates, the research becomes far more diagnostic-and allows designers the freedom to develop more effective solutions.
Finally, getting more from qualitative packaging research requires recognizing its inherent limitations, and using other methodologies to complement traditional approaches. For example, we are now regularly tracking package viewing patterns in conjunction with traditional qualitative approaches: The viewing patterns document each packaging system’s initial communication hierarchy, while in-depth discussions uncover shoppers’ reactions to each approach.
Similarly, we find ourselves starting at the shelf far more frequently, not to numerically measure visibility or shoppability but, instead, to ground consumers in the shopping experience, present concepts within a realistic context and include a behavioral component to complement attitudinal questioning.
As these innovations suggest, a primary objective is to link qualitative studies more directly to the realities of the shopping experience. By adding value in this manner-and grounding studies in several consistent best practices-qualitative packaging research will be more likely to lead marketers and designers towards in-market success. BP
The author, Scott Young, is the president of Perception Research Services, a company that conducts more than 600 studies annually to help develop, assess and improve packaging systems and help marketers “win at retail”. Contact Scott at or 201.346.1600.