Multilevel Motivation and Engagement: Assessing Construct Validity
Across Students and Schools
Andrew J. Martin, Lars-Erik Malmberg and Gregory Arief D. Liem
Educational and Psychological Measurement 2010 70: 973
From a multilevel perspective based on data from 21,579 students in 58 high schools, using multilevel confirmatory factor analysis (multilevel CFA), the present study investigates the multilevel (student and school) factor structure of the Motivation and Engagement Scale–High School (MES-HS; Martin, 2007a, 2007b, 2009), a recently developed multidimensional measure of high school student motivation and engagement.
The Motivation and Engagement Scale:
four higher order factors (or clusters) and 11 first-order factors:
(1) Adaptive cognition:
(a) self-efficacy (b) valuing of school (c) mastery orientation
(2)Adaptive behavior:
(d) planning (e) study management (f) persistence
(3) Impeding/maladaptive cognition:
(g) anxiety (h) failure avoidance (i) uncertain control
(4) Maladaptive behavior
(j) self-handicapping (k) disengagement
Multilevel Confirmatory Factor Analysis:
It can estimate stability of models at individual and group levels simultaneously and generate an error-free variance ratio for the intraclass correlation that yields much more reliable individual- and group-level measures.
Mplus Version 4 can be used for parameters estimation.
In multilevel CFA, the researcher posits an a priori structure and tests the ability of a solution based on this structure to fit the data by demonstrating at each level (e.g., student and school) that
(a) the factor solution is well defined, evincing distinct and interpretable factors
(b) parameter estimates are consistent with theory and a priori predictions
(c) the subjective indices of fit are reasonable (McDonald & Marsh, 1990).
Maximum likelihood with robustness to non-normality and non-independence of observations (MLR; L. K. Muthen & Muthen, 2007) was the method of estimation used for the models
Fit indexes
Comparative fit index (CFI): > 0.9 0and 0.95
Standardized root mean square residual (SRMR): < .05 and .08
Root mean square error of approximation (RMSEA) < .05 and .08
Akaike information criterion (AIC)
Comments:
1 Before doing multilevel CFA, the multilevel analysis should be done to determine the level to be done is reasonable. The method is suitable for multilevel data.
2 the number of schools may be replaced by classrooms whose numbers is more than schools in real data.
3 The normal CFA can be used for comparison in this study.
4 If the constructs of the scales are different between student-level and school-level, then how do we to explain the results.