TY - JOUR

T1 - Validity and Reliability of Willingness-to-Pay Estimates

T2 - Evidence from Two Overlapping Discrete-Choice Experiments

AU - Telser, Harry

AU - Becker, Karolin

AU - Zweifel, Peter

PY - 2008/10/1

Y1 - 2008/10/1

N2 - Background: Discrete-choice experiments (DCEs), while becoming increasingly popular, have rarely been tested for validity and reliability. Objective: To address the issues of validity and reliability of willingness-to-accept (WTA) values obtained from DCEs. In particular, to examine whether differences in the attribute set describing a hypothetical product have an influence on preferences and willingness-to-pay (WTP) values of respondents. Methods: Two DCEs were designed, featuring hypothetical insurance contracts for Swiss healthcare. The contract attributes were pre-selzected in expert sessions with representatives of the Swiss healthcare system, and their relevance was checked in a pre-test. Experiment A contained rather radical health system reform options, while experiment B concentrated on more familiar elements such as copayment and the benefit catalogue. Three attributes were present in both experiments: delayed access to innovation (‘innovation’), restricted drug benefit (‘generics’), and the change in the monthly premium (‘premium’). The issue to be addressed was whether WTA values for the overlapping attributes were similar, even though they were embedded in widely differing choice sets. Two representative telephone surveys with 1000 people aged >25 years were conducted independently in the German and French parts of Switzerland during September 2003. Socioeconomic variables collected included age, sex, education, total household income, place of residence, occupation, and household size. Three models were estimated (a simple linear model, a model allowing interaction of the price attribute with socioeconomic characteristics, and a model with a full set of interaction terms). Results: The socioeconomic characteristics of the two samples were very similar. Theoretical validity tends to receive empirical support in both experiments in all cases where economic theory makes predictions concerning differences between socioeconomic groups. However, a systematic inappropriate influence on measured WTA seems to be present in at least one experiment. This is likely to be experiment A, in which respondents were far less familiar with proposed alternatives than in experiment B. Conclusions: Measuring preferences for major, little-known innovations in a reliable way seems to present particular challenges for experimental research. In health economics, stated-preference methods such as discrete-choice experiments (DCEs) have been increasingly used to measure benefits. Applications of DCEs to the valuation of healthcare programs have recently become numerous.[1–3] In a DCE, individuals are given a choice between hypothetical commodities. From the choices that respondents make between the goods (differing in product attributes), the researcher can derive the implicit trade-offs between the product attributes. This allows the computation of respondents’ marginal utility for each attribute. With the inclusion of a cost or price attribute, a money value can be calculated for each characteristic as well as for the entire good or program. The advantage of this approach over other stated-preference methods (e.g. the contingent-valuation [CV] method), lies in the fact that the price attribute is one among several, of which all vary in the course of the experiment. Biases that occur when individuals are asked directly about their willingness to pay (WTP) are less likely to be observed in DCEs.[4] Applications of DCEs in health economics so far mainly comprise studies of WTP for different treatment methods[5–9] or different hospital or physician services.[10,11] DCEs that deal with the health system as a whole (e.g. the present study) are rare.[12] DCEs are usually limited to a small number of attributes.[1,13] In particular, when the product is defined as an entire healthcare system, this raises the question of whether neglected attributes influence the decisions of the respondents, causing bias in the WTP values obtained. Furthermore, it is often unclear what specific effects certain reform proposals will have in practice, which makes the hypothetical character of the experiment more problematic. However, reliable and valid WTP values are of utmost importance if policy recommendations are to be derived from DCE studies.[1] This paper adds to the literature in two ways. First, it seeks to measure and analyze willingness-to-accept (WTA) values for proposed changes to an entire healthcare system; something that has rarely been attempted thus far. Second, it benefits from the unique opportunity to conduct two parallel DCEs with two independent samples for addressing validity and reliability issues, made possible by the inclusion of three overlapping attributes in both DCEs. To the best knowledge of the authors, comparative DCEs have only been conducted by Slothuus-Skoldborg and Gyrd-Hansen,[12] who analyzed WTP for screening methods for different types of cancer, and by Merino-Castellò,[6] who studied the demand for two different drugs. In the field of environmental economics, DeShazo and Fermo[14] have undertaken two DCEs concerning national park attributes in two different countries. The aim of the present study is to examine whether differences in the attribute set describing a hypothetical product have an influence on preferences and WTP values of respondents. This can be tested thanks to three overlapping attributes.

AB - Background: Discrete-choice experiments (DCEs), while becoming increasingly popular, have rarely been tested for validity and reliability. Objective: To address the issues of validity and reliability of willingness-to-accept (WTA) values obtained from DCEs. In particular, to examine whether differences in the attribute set describing a hypothetical product have an influence on preferences and willingness-to-pay (WTP) values of respondents. Methods: Two DCEs were designed, featuring hypothetical insurance contracts for Swiss healthcare. The contract attributes were pre-selzected in expert sessions with representatives of the Swiss healthcare system, and their relevance was checked in a pre-test. Experiment A contained rather radical health system reform options, while experiment B concentrated on more familiar elements such as copayment and the benefit catalogue. Three attributes were present in both experiments: delayed access to innovation (‘innovation’), restricted drug benefit (‘generics’), and the change in the monthly premium (‘premium’). The issue to be addressed was whether WTA values for the overlapping attributes were similar, even though they were embedded in widely differing choice sets. Two representative telephone surveys with 1000 people aged >25 years were conducted independently in the German and French parts of Switzerland during September 2003. Socioeconomic variables collected included age, sex, education, total household income, place of residence, occupation, and household size. Three models were estimated (a simple linear model, a model allowing interaction of the price attribute with socioeconomic characteristics, and a model with a full set of interaction terms). Results: The socioeconomic characteristics of the two samples were very similar. Theoretical validity tends to receive empirical support in both experiments in all cases where economic theory makes predictions concerning differences between socioeconomic groups. However, a systematic inappropriate influence on measured WTA seems to be present in at least one experiment. This is likely to be experiment A, in which respondents were far less familiar with proposed alternatives than in experiment B. Conclusions: Measuring preferences for major, little-known innovations in a reliable way seems to present particular challenges for experimental research. In health economics, stated-preference methods such as discrete-choice experiments (DCEs) have been increasingly used to measure benefits. Applications of DCEs to the valuation of healthcare programs have recently become numerous.[1–3] In a DCE, individuals are given a choice between hypothetical commodities. From the choices that respondents make between the goods (differing in product attributes), the researcher can derive the implicit trade-offs between the product attributes. This allows the computation of respondents’ marginal utility for each attribute. With the inclusion of a cost or price attribute, a money value can be calculated for each characteristic as well as for the entire good or program. The advantage of this approach over other stated-preference methods (e.g. the contingent-valuation [CV] method), lies in the fact that the price attribute is one among several, of which all vary in the course of the experiment. Biases that occur when individuals are asked directly about their willingness to pay (WTP) are less likely to be observed in DCEs.[4] Applications of DCEs in health economics so far mainly comprise studies of WTP for different treatment methods[5–9] or different hospital or physician services.[10,11] DCEs that deal with the health system as a whole (e.g. the present study) are rare.[12] DCEs are usually limited to a small number of attributes.[1,13] In particular, when the product is defined as an entire healthcare system, this raises the question of whether neglected attributes influence the decisions of the respondents, causing bias in the WTP values obtained. Furthermore, it is often unclear what specific effects certain reform proposals will have in practice, which makes the hypothetical character of the experiment more problematic. However, reliable and valid WTP values are of utmost importance if policy recommendations are to be derived from DCE studies.[1] This paper adds to the literature in two ways. First, it seeks to measure and analyze willingness-to-accept (WTA) values for proposed changes to an entire healthcare system; something that has rarely been attempted thus far. Second, it benefits from the unique opportunity to conduct two parallel DCEs with two independent samples for addressing validity and reliability issues, made possible by the inclusion of three overlapping attributes in both DCEs. To the best knowledge of the authors, comparative DCEs have only been conducted by Slothuus-Skoldborg and Gyrd-Hansen,[12] who analyzed WTP for screening methods for different types of cancer, and by Merino-Castellò,[6] who studied the demand for two different drugs. In the field of environmental economics, DeShazo and Fermo[14] have undertaken two DCEs concerning national park attributes in two different countries. The aim of the present study is to examine whether differences in the attribute set describing a hypothetical product have an influence on preferences and WTP values of respondents. This can be tested thanks to three overlapping attributes.

KW - Health sciences

KW - Marginal Utility

KW - Socioeconomics Group

KW - Insurance Contract

KW - Prive Attribute

KW - Medical Innovation

U2 - 10.2165/1312067-200801040-00010

DO - 10.2165/1312067-200801040-00010

M3 - Conference article in journal

C2 - 22272996

VL - 1

SP - 283

EP - 298

JO - The Patient : Patient-Centered Outcomes Research

JF - The Patient : Patient-Centered Outcomes Research

SN - 1178-1658

IS - 4

ER -