TY - JOUR

T1 - Against the Mainstream

T2 - On the Limitations of Non-Invariance Diagnostics: Response to Fischer et al. and Meuleman et al.

AU - Welzel, Christian

AU - Kruse, Stefan

AU - Brunkert, Lennart

PY - 2023/8

Y1 - 2023/8

N2 - Our original 2021 SMR article “Non-Invariance? An Overstated Problem with Misconceived Causes” disputes the conclusiveness of non-invariance diagnostics in diverse cross-cultural settings. Our critique targets the increasingly fashionable use of Multi-Group Confirmatory Factor Analysis (MGCFA), especially in its mainstream version. We document—both by mathematical proof and an empirical illustration—that non-invariance is an arithmetic artifact of group mean disparity on closed-ended scales. Precisely this arti-factualness renders standard non-invariance markers inconclusive of measurement inequivalence under group-mean diversity. Using the Emancipative Values Index (EVI), OA-Section 3 of our original article demonstrates that such artifactual non-invariance is inconsequential for multi-item constructs’ cross-cultural performance in nomological terms, that is, explanatory power and predictive quality. Given these limitations of standard non-invariance diagnostics, we challenge the unquestioned authority of invariance tests as a tool of measurement validation. Our critique provoked two teams of authors to launch counter-critiques. We are grateful to the two comments because they give us a welcome opportunity to restate our position in greater clarity. Before addressing the comments one by one, we reformulate our key propositions more succinctly.

AB - Our original 2021 SMR article “Non-Invariance? An Overstated Problem with Misconceived Causes” disputes the conclusiveness of non-invariance diagnostics in diverse cross-cultural settings. Our critique targets the increasingly fashionable use of Multi-Group Confirmatory Factor Analysis (MGCFA), especially in its mainstream version. We document—both by mathematical proof and an empirical illustration—that non-invariance is an arithmetic artifact of group mean disparity on closed-ended scales. Precisely this arti-factualness renders standard non-invariance markers inconclusive of measurement inequivalence under group-mean diversity. Using the Emancipative Values Index (EVI), OA-Section 3 of our original article demonstrates that such artifactual non-invariance is inconsequential for multi-item constructs’ cross-cultural performance in nomological terms, that is, explanatory power and predictive quality. Given these limitations of standard non-invariance diagnostics, we challenge the unquestioned authority of invariance tests as a tool of measurement validation. Our critique provoked two teams of authors to launch counter-critiques. We are grateful to the two comments because they give us a welcome opportunity to restate our position in greater clarity. Before addressing the comments one by one, we reformulate our key propositions more succinctly.

KW - compositional Substitutability

KW - emancipative Values

KW - measurement Equivalence

KW - nomological Performance

KW - Non-Invariance

KW - Politics

UR - http://www.scopus.com/inward/record.url?scp=85129148316&partnerID=8YFLogxK

UR - https://www.mendeley.com/catalogue/e8c7419c-3c1a-3afb-bf8d-0816cb11aa8b/

U2 - 10.1177/00491241221091754

DO - 10.1177/00491241221091754

M3 - Comments / Debate / Reports

AN - SCOPUS:85129148316

VL - 52

SP - 1438

EP - 1455

JO - Sociological Methods and Research

JF - Sociological Methods and Research

SN - 0049-1241

IS - 3

ER -