This web page makes a computer program available, PREF, to permit a reader to replicate and to extend the analysis reported in the following article:

Bemmaor, Albert C. and Udo Wagner (2000), "A Multiple-Item Model of Paired Comparisons: Separating Chance From Latent Preference," Journal of Marketing Research, 37 (November), 514-24.

PREF works under Windows 95, Windows NT and Windows XP. You can download all the DATA sets which are mentioned in this study.

You can also download a computer program in executable form and two files, HELPPREF.HTM and REFERENCES.HTM. All the files are zipped. Here is a brief description of the current state of the program:

1. Estimation and simulation of the binomial model;
2. Estimation and simulation of the binomial/Dirichlet model;
3. Estimation and simulation of the multinomial model;
4. Estimation and simulation of the multinomial/Dirichlet model.

Examples of issues that can be addressed with the use of PREF

Here is a set of issues that can be dealt with:

• Issue 1: In conjoint analysis, which product profile (combination of attribute levels) do respondents like most ? Which attributes are most salient? What are the part-worths of the attributes?
• Issue 2: Which of those (say) five advertising copies do consumers like most? By how much? Which advertising copy do they like least ? How much less than the others? In a similar vein, we can ask the following question: How much do consumers like my brand compared to competing brands?
• Issue 3: How well does the new product perform in a "blind" test when compared to competing products?
• Issue 4: How strongly are consumers’ preferences split between alternative advertising copies? Between alternative brands?
• Issue 5: Can the new brand’s sales cannibalize the current brand’s sales?
• Issue 6: Can the new brand draw its sales more than proportionally from the sales of a competing brand?
• Issue 7: How large should a sample size be in order to measure the preference parameters with, say, a ± .05 accuracy? How large should it be in order to measure the preference parameters and the polarization index with the same level of accuracy? Or with different levels of accuracy?
• Issue 8 : How do the « draws» shift when we change the price level of the new brand? What are the «draws» from the store brands versus the national brands? How do the «draws» shift when we change the brand name of the new product? Which brand name seems «best»? Which price level seems «best»?
• Issue 9 : For frequently purchased goods, what are the «draws» as implied by paired comparisons «before trial» versus paired comparisons «after trial»?
• Issue 10 :For a new perfume, what are the «draws» as implied by paired comparisons «right after the spray » versus paired comparaions «half-an-hour after the spray»?
• Issue 11 : What is the effect of a brand name on a paired comparison product test ? Which brand name seems «best»?

Some hints on the means to address the issues stated above are as follows :

• Issue 1: These are standard issues in conjoint analysis. The binomial/Dirichlet model permits the estimation of the preference share of each product profile and the estimation of a polarization index. For example, when there are four profiles, a respondent makes six paired comparisons. The program estimates five preference shares and a polarization index. The program works in the case where a respondent compares all the profiles against each other ; there is no elimination of comparisons due to the domination of an alternative (see, for example, Allenby/Arora/Ginter's study published in Journal of Marketing Research, May 1995). PREF is particularly suitable when (1) the number of attributes is "small" (one or two) and (2) the number of levels par attribute is "small" (one or two). you might work out an example in this case and compare the results to traditional conjoint analysis. The case of a limited number of attributes can often happen in the markets of frequently purchased conusumer goods. one atttribute can be price (with two to three levels) and another can be anti-tartar capacity for toothpaste (three levels) for example. We will omit dominated alternatives in the paired comparisons. The advantage of PREF over the traditional methods used in conjoint analysis is two-fold:

  (i) It assesses the "part-worths" in terms of odds ratios (likelihood to choose Product 1 over Product 2) instead of absolute algebraic values, thereby making the comparisons across products more informative;
  (ii) It captures the heterogeneity of preferences across respondents.
  
  Example:

  
  Let us consider the case of a soup with the following attributes and levels:
  

  Price:	$1.49	$0.90
  Flavor:	onion	chicken	country vegetables
  Salt:	yes	no
  Calories:	100	80

  
  In total, there are: 2 * 3 * 2 * 2 = 24 combinations which would represent 23 * 24/ 2 = 276 combinations. Instead of considering a full factorial design, we will analyze the following six combinations:

  $1.49	Onion	No salt	80
  $1.49	Chicken	No salt	80
  $1.49	Country vegetables	No salt	80
  $0.90	Onion	Salt	100
  $0.90	Chicken	Salt	100
  $0.90	Country vegetables	Salt	100

  
  This makes a total of 15 paired comparisons. We have eliminated the combinations which are dominated in a paired comparisons. Note that we assume that the attributes salt (yes/no) and calories (80 versus 100) are correlated which often happens in practical applications. Rarely all the attributes are independent of one another, in particular when the number of attributes becomes "large".

   

• Issue 2: The parameters m₁, m₂, …measure the mean probabilities that an individual respondent "truly" prefers product 1, product 2, ... The parameter µI+1 is the mean probability that a respondent has "truly" no preference between the I products.
• Issue 3: Again, the same parameters m₁, m₂, … permit to assess the preferences in a "blind" product test;
• Issue 4: The polarization index f (in the binomial/Dirichlet model and the multinomial/Dirichlet model) measures the degree of preference segmentation.When f is close to 0, consumers are homogeneous with respect to their preferences. When f is close to 1, consumers can be split into groups: There are those who are exclusive preferrers of product 1, those who are exclusive preferrers of product 2, …

• Issue 5: The estimate of corresponding to the paired comparison data between the new brand and the current brand (as collected from the users of the current brand) permits to address the issue. If is close to 1, we can expect cannibalization of the sales of the current brand: Some users of the current brand can switch to the new brand. If f is close to 0, we do not expect a higher level of cannibalization than "proportional draw";
• Issue 6. Again, the estimate off corresponding to the paired comparison data between the new product and the competing product (as collected from the users of the competing product) permit to address the issue. If f is close to 1, we can expect that the sales draw from the competing brand will be more than proportional to the market share of the competing brand. If f is close to 0, we can expect the sales draw to be close to proportional to the market share of the competing brand.
• Issues 5 and 6: The main contribution of PREF is the assessment of "draws" (the so-called "sources of volume"), not the assessment of the share of the new product. Assessing "draws" is at least as important as assessing the absolute market share level. what matters is where the "sales" come from. To date, this question has not been addressed in as a straighforward manner as in PREF to our knowledge. Here is a roadmap about how to capture new product substitution with PREF:

  1. Define a set of competing brands (with similar price levels): let us say they are A, B and C.
  2. Collect paired comparison data (see the website and the JMR article);
  3. Use PREF. The output will be the preference shares. PREF can include a fourth "brand" (see the interpretation on the website): you will simply reallocate its share among all the other three brands in proportion to their share;
  4. Include the new brand and collect additional paired comparison data by comparing the new brand to each of the three preceding brands;
  5 Run PREF again; it is quite unlikely that you will obtain a fifth "brand". if you do, please proceed as previously. you will obtain the new set of brand shares. you can assess the draw from each of the existing brand and compare it to a proportional draw. therefore, you will find the brands that lose more than in proportion to their share and those that lose less than in proportion. you can compare the results with expectations. Note that PREF can show that one (or more) of the existing brands do(es) not lose share to the new brand.

This analysis is quite simple to carry out. There has to be at least two brands in the first step. Otherwise, PREF can handle "many" brands. The question to be asked for PREF can be changed to the following one:

Between the two products A and B, which one would you buy?

-2	I would buy B certainly
-1	I would buy B probably
0	I do not know which one i would buy
1	I would buy A probably
2	I would buy A certainly

For frequently packaged goods, you can measure "draws" prior to trial (tasting or use) and after trial (tasting or use). You can include a price to each product tested. In this fashion, you can test alternative formulations of the new product by assessing their corresponding «draws» before making a new product decision. PREF permits to assess the "sources of volume" of line extensions prior to launch : «A line extension … involves a different flavor or ingredient variey, a different form or size, or a different application for the (parent) brand» (Keller 2003, p. 577).

An Example:

With two brands (Classic Coke and Pepsi), the polarization index phi is equal to 0.278. (The closer phi is to 1, the more heterogeneous the respondents are. the closer it is to 0, the more homogenous, the respondents are. phi is equal to 1/(a1 + a2 + a3 + 1) in the case of two brands). you can click on "graph" and obtain a representation of the distribution of the preferences across the respondents. You obtain an inverted-J shaped curve which shows the relative homogeneity of the respondents. When you include New Coke, the polarization index shifts to 0.113. (phi is equal to 1/(a1 + a2 + a3 + a4 + 1) in the case of three brands). Due to the increased similarity between products, the preferences become more homogenous across respondents and consequently choices become potentially more "random".

Note: In an electronic message dated 7/26/2004, Professor Agresti confirmed that the test between New Coke, Classic Coke and Pepsi shown in his 1992 article was a "blind" test. In a test where the brand names appear, would you expect the results to differ? How? Why? Run a similar test as that given in this example.

Using the data available in HELPPREF.HTM, follow the methodology shown above to compute the "draws" of New Coke. Consistent with Agresti's (1992, Table 3) paper, the brands are as follows: A: New Coke; B: Classic Coke; C: Pepsi. The comparisons are given in the following order: A-B, A-C and B-C. Looking at the Table in 3.1, we find that, for example, four respondents preferred Classic Coke to New Coke strongly, 13 preferred Classic Coke to New Coke slightly, nine did not have a preference, 19 preferred New Coke to Classic Coke sligthly and eight preferred New Coke to Classic Coke strongly. It is quite easy to understand the other data. Here is the correspondence between the Bemmaor/Wagner notations and Agresti's in A-B:

Bemmaor/Wagner	Agresti	Label
-2	5	I prefer Classic Coke to New Coke strongly
-1	4	I prefer Classic Coke to New Coke slightly
0	3	I have no preference
1	2	I prefer New Coke to Classic Coke slightly
2	1	I prefer New Coke to Classic Coke strongly

Are the results consistent with the expectations?

Solution given by PREF.

Running PREF with Classic Coke and Pepsi only (The number of products is equal to two and the parameter k is equal to 2), one obtains:

mu1 (Classic Coke) = 0.362
mu2 (Pepsi) = 0.503
mu3 = 0.135
Total = 1

Reallocating the third "brand" in proportion to the shares, one obtains:

mu1R (Classic Coke) = 0.419
mu2R (Pepsi) = 0.581

These results are consistent with previous results found in a blind test between Classic Coke and Pepsi. In a blind test, more people prefer Pepsi to Classic Coke. Including New Coke into the set of brands and running PREF again (the number of products is equal to three), one obtains:

mu1 (New Coke) = 0.407
mu2 (Classic Coke) = 0.274
mu3 (Pepsi) = 0.319
Total = 1

Under the proportional draw assumption, 41.9% of the share of New Coke originates from Classic Coke and 58.1% comes from Pepsi. PREF shows that New Coke "draws" (0.419-0.274)/0.407 = 35.6% of its share from Classic Coke instead of 41.9% and 64.4% of its share from Pepsi instead of 58.1%. Therefore, it "draws" 10.8% (=(64.4 - 58.1)/58.1) more than in proportion to share from Pepsi - as intended. New Coke was sweeter than Classic Coke, and, consequently, more similar to Pepsi. Note that New Coke was preferred to Pepsi and much preferred to Classic Coke in the "blind" test. This analysis does not take account of potential inertia due to past choice behavior. As shown by the experience, this inertia may be key to the failure of a new product.

Note: The qualitative results shown here, i.e., New Coke is more preferred to Classic Coke than it is to Pepsi and Pepsi is preferred to Classic Coke, are consistent with those reported in Agresti (1992, Table 4). As in Agresti, we assume that the repeated ratings are correlated across respondents. But also, we assume that they are independent of each other at the individual level.

One possible means to take inertia into consideration consists of (1) re-running the product tests with the brand names "in the open", (2) shifting the response categories from "I prefer Product 1 to Product 2 strongly" to "I would buy Classic Coke(rather than New Coke) certainly", and (3) running PREF as before. The "draws" and the level of the preference share of New Coke can be surprising when compared with the previous analysis. As we do not have the data available, the question remains open.

This comparison across tests with versus without the brand names can bring valuable insights into the potential interaction of "draws". Again, the main advantage of the version with the brand names is that it aims to capture inertia. Consequently, we can expect the performance of the new product in the test to be less encouraging, in particular in the case of a line extension.

The soft drink data are Professor Agresti's. They were published but there is an error in the reported data (see references.htm). Professor Agresti sent the author the corrected data by (regular) mail. The analysis reported in Agresti (1992) is based upon the correct data set.

Notes:

(i) Although the soft drink data reported in Agresti (1992, Table 3) are in error, the paper is based upon the correct data set. The user of PREF can find the typewriter ribbon data of Fleckenstein and colleagues (1958) in Table 1 of the Agresti paper and he/she can compare the results obtained with PREF to those reported in Table 2 of the paper (adjacent categories logit, cumulative probit and cumulative logit). the disadvantage of this data set as compared with the soft drink data is that intuition on the results is more limited. The Fleckenstein and colleagues data are available by downloading the data set on the website, among other data sets.

(ii) In the heading of Table 1 (Agresti 1991), there is an error: The title refers to typewriter ribbons whereas the Fleckenstein and colleagues' data deal with typewriter carbon papers. You can test for the similarity of the typewriter carbon papers by (1) running PREF on four carbon papers, (2) including a fifth one and (3) running PREF again.

• Issue 7. As PREF permits the simulation of data sets with known parameters, the issue can be addressed. Bemmaor and Wagner (2000) report results on sample size requirements.
• Issues 8, 9, 10 and 11: The issues can be addressed in a fairly «straighforward» manner.

PREF provides a flexible methodology for analyzing paired comparison data. These comparisons can be applied to (1) the comparison of product "profiles" in conjoint analysis, (2) "blind"product tests (between New Coke, Classic Coke and Pepsi as in Agresti's 1992 article in Applied Statistics, No. 2), and (3) survey data with identified brands and products .

Due to potential biases and errors (e.g., confusion, social desirability bias) , PREF does not resort to a respondent’s memory to capture new product substitution: Stimuli are described as accurately as possible, including prices whenever appropriate, to activate all possible cues to a respondent.. A respondent may be asked to taste or to try the product (if available) before providing further comparative evaluations. This testing (tasting or use) can include the existing products. The better informed a respondent is, the more reliable the comparative evaluations are likely to be. Information provided to a respondent at the time of the study is a key ingredient to PREF methodology.

Teaching with PREF

Depending on the level of the course (doctoral seminar versus MBA course versus undergraduate course), the instructor will give the technical details of the model or he/she will not. He/she will explain the "spirit" of the model in all the cases. For your information, we tested PREF in the classroom and had students carry out projects with PREF. They did not ask a single question about the use of PREF.
Reading the JMR paper is not a requirement to understand the output of PREF. Some students can ignore the JMR paper while others can read it to understand where the output comes from. Therefore, self-selection will prevail.

Getting acquainted with PREF

It takes a maximum of five to ten minutes to get acquainted with PREF and to get it to run on a set of hypothetical data. The model which will be the most commonly used one is, by far, the Binomial/Dirichlet model. The other models are either a special case or applicable in special contexts only. The Binomal/Dirichlet is a model with scope: It can be used to address a whole series of issues, from the assessment of « draws » for a new high technology product prior to launch to the measurement of consumer preferences between two yogurts in a «blind» test. The range of applications of PREF is rather unique. (Of course, the user will change the question depending on the purpose of the analysis). It is a multi-purpose model. What makes it attractive also is the inputs, i.e , paired comparisons, which are very-easy-to-collect data. PREF is self-contained to the extent that the data can be collected and the analysis can be carried out in the classroom. Changes can easily be made in the analysis ex post with the collection of additional data and/or the analysis of different subsets of paired comparisons. The students have full control of the whole research process (the design and the analysis). .This website will show neither the names of the adopters nor the names of their universities nor any anonymous testimonial. Each individual will be influenced by his/her own judgement only. PREF relies on the JMR article only.

Keller, Kevin Lane (2003), Strategic Brand Management : Building, Measuring, and Managing Brand Equity, Second Edition, Upper Saddle River, New Jersey : Prentice Hall.

Gibson, Larry (2003), "Letter to the Editor: Why the New Coke Failed," Marketing Research, 15 (2), 52. Available in full text in EBSCO data basis (Business Source Premier).

for a definition of line extensions with examples, see:

Cegarra, Jean-Jack and Dwight Merunka (1993), "Les Extensions de Marque: Concepts et Modèles," Recherche et Applications en Marketing, 8 (1), 53-76.

for an empirical study on the determinants of the potential success of line extensions, see:

Reddy, Srinivas K., Susan L. Holak, and Subodh Bhat (1994), "To Extend or Not to Extend: Success Determinants of Line Extensions," Journal of Marketing Research, 31 (May), 243-262.

for an alternative methodology which is designed to capture the "sources of volume" of a new product, see:

Bockenholt, Ulf and William R. Dillon (1997), Some New Methods for an Old Problem: Modeling Preference Changes and Competitive Market Structure in Pretest Market Data, " Journal of Marketing Research, 34 (February) 130-142.

Fader, Peter S. and Bruce G. Hardie (1996), "Modeling Consumer Choice Among SKU's", Journal of Marketing Research,33 (November), 442-52.

for an example of ex-post measurement of cannibalization, see:

Lomax, Wendy, Kathy Hammond, Robert East, and Maria Clemente (1997), The Measurement of Cannibalization, Journal of Product and Brand Management, 6 (1), 27-39.

The use of the program requires to obtain a license number. For further information, you can contact the Center for Research in Management and Economics, at the following address:

research.center@essec.fr

pref@essec.fr

We do not expect to receive any comment on the technical performance of PREF: PREF works. It was thoroughly tested in the late 90's. The chances of technical failures are 0%.

PS. As an extension, the binomial and binomial/Dirichlet model can be applied to absolute preference scores when there are two products in a test. In this case, letting x₁ be the absolute preference score of item 1 and x₂ be the absolute preference score of item 2, the user can compute a relative preference score d₁₂ = x₁ – x₂ for each individual respondent. However, the model makes the restrictive assumption that the individual scores x1 and x2 are not observed. Only the difference d12 is observed. Further extensions of the model have not yet been implemented.

Acknowledgement: Albert Bemmaor expresses his gratitude to Professor Alan Agresti of the University of Florida in Gainesville for his constant support during the development of this website. The usual disclaimer applies.

Last modified 07/19/04