A growing volume of research has used polynomial regression analysis (PRA) to examine congruence effects in a broad range of organizational phenomena. However, conclusions from congruence studies, even ones using the same theoretical framework, vary substantially. We argue that conflicting findings from congruence research can be attributable to several methodological artifacts, including measurement error, collinearity among predictors, and sampling error. These methodological artifacts can significantly affect the estimation accuracy of PRA and undermine the validity of conclusions from primary studies as well as meta-analytic reviews of congruence research. We introduce two alternative methods that address this concern by modeling congruence within a latent variable framework: latent moderated structural equations (LMS) and reliability-corrected single-indicator LMS (SI-LMS). Using a large-scale simulation study with 6,322 conditions and close to 1.9 million replications, we showed how methodological artifacts affected the performance of PRA, specifically, its (un)biasedness, precision, Type I error rate, and power in estimating linear, quadratic, and interaction effects. We also demonstrated the substantial advantages of LMS and SI-LMS compared with PRA in providing accurate and precise estimates, particularly under undesirable conditions. Based on these findings, we discuss how these new methods can help researchers find more consistent effects and draw more meaningful theoretical conclusions in future research. We offer practical recommendations regarding study design, model selection, and sample size planning. In addition, we provide example syntax to facilitate the application of LMS and SI-LMS in congruence research.
Details
