DIF Analysis with Multilevel Data: A Simulation Study Using the Latent Variable Approach

Ying Jin, Hershel Eason

Abstract


The effects of mean ability difference (MAD) and short tests on the performance of various DIF methods have been studied extensively in previous simulation studies. Their effects, however, have not been studied under multilevel data structure. MAD was frequently observed in large-scale cross-country comparison studies where the primary sampling units were more likely to be clusters (e.g., schools). With short tests, regular DIF methods under MAD-present conditions might suffer from inflated type I error rate due to low reliability of test scores, which would adversely impact the matching ability of the covariate (i.e., the total score) in DIF analysis. The current study compared the performance of three DIF methods: logistic regression (LR), hierarchical logistic regression (HLR) taking multilevel structure into account, and hierarchical logistic regression with latent covariate (HLR-LC) taking multilevel structure into account as well as accounting for low reliability and MAD. The results indicated that HLR-LC outperformed both LR and HLR under most simulated conditions, especially under the MAD-present conditions when tests were short. Practical implications of the implementation of HLR-LC were also discussed.


Full Text:

PDF


DOI: https://doi.org/10.5296/jei.v2i2.10045

Refbacks

  • There are currently no refbacks.


Journal of Educational Issues  ISSN 2377-2263

Copyright © Macrothink Institute

To make sure that you can receive messages from us, please add the 'macrothink.org' domains to your e-mail 'safe list'. If you do not receive e-mail in your 'inbox', please check your 'spam' or 'junk' folder.