Deciding between the Covariance Analytical Approach and the Change-Score Approach in Two Wave Panel Data
Research output: Journal contributions › Journal articles › Research › peer-review
Standard
In: Multivariate Behavioral Research, Vol. 56, No. 3, 21.07.2021, p. 447-458.
Research output: Journal contributions › Journal articles › Research › peer-review
Harvard
APA
Vancouver
Bibtex
}
RIS
TY - JOUR
T1 - Deciding between the Covariance Analytical Approach and the Change-Score Approach in Two Wave Panel Data
AU - Carmen, Köhler
AU - Hartig, Johannes
AU - Schmid, Christine
N1 - Publisher Copyright: © 2020 Taylor & Francis Group, LLC.
PY - 2021/7/21
Y1 - 2021/7/21
N2 - The manuscript focuses on effects in nonrandomized studies with two outcome measurement occasions and one explanatory variable, and in which groups already differ at the pretest. Such study designs are often encountered in educational and instructional research. Two prominent approaches to estimate effects are (1) covariance analytical approaches and (2) latent change-score models. In current practice, both approaches are applied interchangeably, without a clear rationale for when to use which approach. The aim of this contribution is to outline under which conditions the approaches produce unbiased estimates of the instruction effect. We present a theoretical data generating model in which we decompose the variances of the relevant variables, and examine under which data generating conditions the estimated instruction effect is unbiased. We show that, under specific assumptions, both methods can be used to answer the general question of whether instruction has an effect. Another implication from the results is that practitioners need to consider which underlying data generating assumptions the approaches make, since a violation of those assumptions will lead to biased effects. Based on our results, we give recommendations for preferable research designs.
AB - The manuscript focuses on effects in nonrandomized studies with two outcome measurement occasions and one explanatory variable, and in which groups already differ at the pretest. Such study designs are often encountered in educational and instructional research. Two prominent approaches to estimate effects are (1) covariance analytical approaches and (2) latent change-score models. In current practice, both approaches are applied interchangeably, without a clear rationale for when to use which approach. The aim of this contribution is to outline under which conditions the approaches produce unbiased estimates of the instruction effect. We present a theoretical data generating model in which we decompose the variances of the relevant variables, and examine under which data generating conditions the estimated instruction effect is unbiased. We show that, under specific assumptions, both methods can be used to answer the general question of whether instruction has an effect. Another implication from the results is that practitioners need to consider which underlying data generating assumptions the approaches make, since a violation of those assumptions will lead to biased effects. Based on our results, we give recommendations for preferable research designs.
KW - Educational science
KW - Change-Score Model
KW - conditional model
KW - instruction effect
KW - multilevel SEM
UR - http://www.scopus.com/inward/record.url?scp=85079776359&partnerID=8YFLogxK
U2 - 10.1080/00273171.2020.1726723
DO - 10.1080/00273171.2020.1726723
M3 - Journal articles
C2 - 32075436
VL - 56
SP - 447
EP - 458
JO - Multivariate Behavioral Research
JF - Multivariate Behavioral Research
SN - 0027-3171
IS - 3
ER -