Professor Bart Meuleman (Coordinator of the Centre for Sociological Research at KU Leuven, and President of the European Survey Research Association)

In the special panel on measurement invariance and validity in cross-cultural research, he will give the speech entitled “Measurement equivalence: Between Statistical Dogmatism and Anything Goes”

Abstract:

Over the past decades, tests for measurement equivalence have become increasingly popular in cross-national and cross-cultural research. Among various techniques (see van de Vijver et al. 2019 for an overview), multigroup confirmatory factor analysis (MGCFA) has become the method of choice to assess whether survey instruments can successfully travel linguistic and cultural borders (Davidov et al. 2014). The spread of this technique has contributed importantly to the concern for and understanding of the issue of comparability of measurements.

At the same time, however, the increased popularity of assessing measurement equivalence has brought along several challenges. First of all, many frequently-used instruments are found to violate (stricter levels of) equivalence, leaving researchers with puzzlement about the comparability of their valued survey data, and with uncertainty as to whether they could safely perform cross-group comparisons. Second, the rather technical nature of equivalence testing has stimulated the measurement literature to focus almost exclusively on statistical details, while neglecting the question of theoretical validity. As a reaction to these trends, some voices have called for ‘a paradigm shift’ away from the current practice of MGCFA-based equivalence testing (Welzel & Inglehart 2016; Welzel, Brunkert, Inglehart & Kruse 2019).

In this contribution, I argue that the paradigm shift proposed by Welzel and colleagues throws out the baby with the bathwater; but that it is nevertheless useful to revisit the logic of equivalence testing. The presentation outlines the epistemological foundations of equivalence testing and its operationalization into concrete statistical procedures. By doing so, I show that some criticisms are based on misconceptions about measurement, while others can be addressed within the current framework of measurement equivalence.     

References

Davidov, E., Meuleman, B., Cieciuch, J., Schmidt, P., & Billiet, J. (2014). Measurement equivalence in cross-national research. Annual review of sociology, 40, 55-75.

Welzel, C., & Inglehart, R. F. (2016). Misconceptions of measurement equivalence: Time for a paradigm shift. Comparative Political Studies, 49(8), 1068-1094.

Welzel, C., Brunkert, L., Inglehart, R., & Kruse, S. (2019).  Measurement Equivalence? A Tale of False Obsessions and a Cure. World Values Research, 11(3), 54-84.

Van de Vijver, F. J., Avvisati, F., Davidov, E., Eid, M., Fox, J. P., Le Donné, N., Lek, K., Meuleman, B., Paccagnella, M. & van de Schoot, R. (2019). Invariance analyses in large-scale studies. OECD Education Working Paper 201.