Designing a conjoint survey
From DDL Wiki
There are no fixed "rules" for how to design a good conjoint survey, but it can be useful to follow a general process. This page provides advice on good practices to follow when designing your own conjoint survey. A key decision is choosing which attributes and levels to include on the survey. "Attributes" are the features of a product that might be important to the consumer, like price, brand, etc., and "levels" are the different versions of the attribute that will be shown in the survey (e.g. if the attribute for a pair of shoes were "price," you might have levels like $25, $50, $75, $100, $125, etc.).
Contents |
The basic process
- Create a list of all attributes that might influence a consumer to purchase a product.
- Begin to narrow down your list - throw out infeasible attributes, review literature on topic, consider needs of the study.
- Consider what levels you might assign for each attribute on the list.
- Go through an iterative process to narrow down your list of attributes and levels. Fielding pilot surveys is an important part of this process, which not only can inform you about which attributes you should or should not include, but also can help you choose language to describe attributes and levels that is more easily understood.
Choosing attributes
To begin choosing attributes, first create a list of all attributes that might influence a consumer to purchase a product. For example, if the survey were on passenger cars, you might include price, fuel economy, 0-60 acceleration time, trunk space, color, brand, number of seats, horsepower, etc. Next, start narrowing down your list. Consider the following:
Previous literature: Conduct a thorough review of previous literature or previous surveys on your topic and try to identify any attributes that have been shown to be important to consumers.
Feasibility: Consider whether any of the attributes would be infeasible to include. For example, testing the "sexiness" of a car would be difficult because it is not a well-defined attribute, and different consumers will have different opinions on what "sexiness" means. This does not mean the attribute must be quantifiable nor continuous in nature. For example, you could include "brand" as an attribute (and many times you should since it is often a big drive of consumer purchase decisions).
Needs of your research: Why are you fielding this conjoint survey in the first place? If it is to examine how important a specific attribute is to consumers, then be sure to include it! Sometimes this might require a compromise. For example, if you really want to know something about the "sexiness" of a car, then try and identify what features of a car make it "sexy" and use those attributes. Is it color, body shape, brand, power? This may require conducting interviews or early simply surveys (non-conjoint) to try and identify these attributes.
Choosing levels
When choosing the levels associated with each attribute, it is first important to consider the nature of the attribute. This will also help in narrowing down your attribute list. Consider the following:
Is the attribute continuous or discrete?: Price is continuous, so you could choose any levels that might represent a reasonable price range. Brand is discrete, so you would need to include every important brand as levels if you chose brand as an attribute.
Will the attribute have too many levels?: If you chose brand, how many brands are there for your product? If the product is soda, then you may have only a handful, but if it is cars, you may end up a hundred. In general, having a larger number of levels will increase the minimum sample size you will need to have enough statistical power to determine a difference between how consumers feel about them, if a difference exists. Sometimes you can compromise. For example, if car brands is an attribute, you might consider using the country of origin of the brand as a proxy for brand. For example, "German" would proxy for all German brands (Volkswagen, Audi, Mercedes, etc.). This reduces your number of levels to only a few countries rather than a hundred independent brands.
What range of levels should I consider?: It is important to choose a range that both fits the needs of you study and is in the realm of belief for your survey respondents. If you are interested in a hypothetical product that may not yet exist in the market, then this becomes a judgement call on the part of the researcher, which may be informed by previous study or knowledge. If you are considering a product that currently exists in the market, then you can use available information to help you choose your levels. In general, try to choose levels that span the currently available range. For example, if you are including price for cars, then make sure to choose levels such that the range from lowest to highest level spans the price range of cars in the segment you are studying. This can be either done by inspection, or aided by examining market data. For example, if you have the price distribution of passenger cars in your market, then you can pull off percentiles from that distribution and use them as levels to represent what is currently available. The figure below is an example for choosing the price levels of cars in China.
Iterate with pilot surveys
Once you have first draft of a narrowed down list of attributes and levels, put them together in a conjoint survey and field a small pilot. The design you use will influence your results, so make sure you consider the design of experiments you are using for the pilot and how it will influence your conjoint analysis. To generate a design, see the tutorial for creating a choice based conjoint design in SAS.
Your results from your pilot will help you decide which of your attributes are important in driving decisions and which might be confusing. If, for example, you find very little to no effect from one attribute, this usually is an indicator that respondents did not consider the attribute when making choices. This can be a result of multiple factors (e.g. the attribute is actually not important to consumer choice, the attribute is confusing and/or misunderstood, the attribute was displayed in a way that was confusing or not clear). You will likely have to adjust your attributes and potentially their levels as well based on your pilot results. Reading respondents' comments can also be very helpful in examining any changes that need to be made.
Finalizing your survey
Once you have made appropriate changes to your survey based on all of the feedback from your pilot surveys, you need to create your final list of attributes and levels. Be sure to document the process of how to came to choose these attributes and levels. Finally, consider again the design of experiments you are using and how information is presented.
