How to Analyze Interview Data and Survey Responses

Market research sometimes requires that a fairly large number of ideas or attributes be sorted and classified according to relationships or attributes. Often, market researchers ask consumers, customers, or clients to organize their ideas. Sometimes it is the market researchers themselves who must classify data. Three ways to organize and analyze qualitative data are described here: affinity diagram, card sort, and constant comparison.

Affinity Diagrams

Affinity diagrams are primarily used to organize information compiled during a brainstorming session. Problems and solutions are often “worked through” by using an affinity diagram. An affinity diagram is one way to organize ideas or attributes. Use of an affinity diagram is also referred to as the KJ Method, named after Kawakita Jiro, who popularized the method in quality improvement circles. Creating an affinity diagram is a six-step process.

• Determine the reason for doing the process
• Identify a logical set of classifications
• List factors related to the classifications
• Place each factor or idea under a classification
• Reduce the classifications by combining and simplifying
• Analyze the diagram—the total group of classifications

Card Sort is a Low-Tech Way to Gain Research Insights

Card sort studies have been used in psychology and cognition research since the military tested soldiers before and during World War II. Today, card sort strategies are often used to test the usability of software architecture. Card sort methods generate information about how respondents associate and group ideas, constructs, or products. As a qualitative process, card sorting helps to support the development of insights.

To participate in a card sort activity, respondents need to organize unsorted cards into groups. They may also be asked to label the categories they create. There are two versions of the card sort activity: closed card sort and open card sort. In an open card sort activity, respondents create their own categories. In a closed card sort, respondents are asked to sort cards into categories that have identified in advance by the market researcher.

Card sorting is a very low-tech method that employs Post-It™ notes or index cards. There are, as you might guess, software packages that support the creation of digital cart sort activities. Card sorting can be conducted with individual respondents, with a small group in which concurrent card sorting is conducted, or as a hybrid activity where respondents individually perform a card sort and then come together as a group to discuss how they approached the task and compare their outcomes.

A card sorting study produces quantitative data in the form of a set of similarity scores. The similarity scores are a measure of the match for various pairs of cards. For example, given a pair of cards, if all the respondents sorted the pair of cards into the same category then the similarity score would be 100 percent. If exactly half of the respondents sorted the two cards into the same category, but the other half sorted the cards into different categories, then the similarity score would be 50 percent.

It is interesting to note that the card sorting technique, which is a qualitative research process, has been used to replace a quantitative technique known as exploratory factor analysis. The citation for this study is as follows: Santos, G. J. (2006), "Card sort technique as a qualitative substitute for quantitative exploratory factor analysis," Corporate Communications: An International Journal, 11 (3), 288–302.

Constant Comparison for Coding Naturalistic Research Data

The constant comparison method is a well-known qualitative research method first described and refined by naturalistic research teams such as Glaser & Strauss and Lincoln & Guba. The constant comparison method is carried out in four stages: (a) comparing data that is applicable to each category, as the categories emerge; (b) integrating the categories and their properties to reduce the data set and data noise; (c) further delimiting the theory based on reduced data set; and (d) writing the theory.

Unlike quantitative research methods, in which a hypothesis is generated before the research even begins, the constant comparison method generates the theory as it progresses. Instead of having a hypothesis to direct the research, themes emerge as the data is coded and analyzed. This is called naturalistic research or grounded theory. Because of the continual building of theory through analysis, the discovery of relationships begins as the initial observations are analyzed. A process of continuous refinement occurs as the coding is integral to the data collection and data analysis.

The narrative content of interviews and open-end survey questions is analyzed for key patterns. The patterns are identified, categorized, and coded in order to uncover themes. A constant comparison process is inductive research. That is, the categories and the meaning of the categories emerge from the data rather than being imposed on the data before the data is even collected or analyzed.