The utility-based approach to health measurement, exemplified by EQ-5D and Health Utilities Index (HUI), has been challenged on a theoretical basis, but the statistical implications of such an approach have received little attention. To empirically investigate this issue, psychometric properties and statistical efficiency of the EQ-5D and HUI Mark 3 (HUI3) classifiers were compared when scored using preference weighted (WPS) and equally weighted summary scores using two longitudinal datasets (n _stroke=124; n _{rehabilitation}= 264). Test-retest reliability, construct validity, responsiveness, and relative efficiency (RE) ratios (with bootstrapped 95% confidence intervals) were examined. WPS had slightly lower test-retest reliability, particularly for EQ-5D (intraclass correlation coefficient=0.61 vs 0.72). For known-groups comparisons, WPS had greater inferential power for both EQ-5D and HUI3 (RE>1). No significant differences in sensitivity to change were observed for EQ-5D [0.71 (95% CI: 0.29,1.33)≤RE≤0.96(95% CI: 0.69,1.32)] or HUI3 [0.97 (95% CI: 0.89,1.03)≤RE≤1.23 (95% CI: 0.98,1.72)]. Implications of weighted scoring will depend on whether the weights are greater or less than equal weights where patients fall along the health state classifier continuum. Because utility weights can affect the statistical properties and significance of results, the summary score selected should be appropriate to the purpose of the study and population of interest.