Sandy's readings and review

Topic 4: Modeling

Topic 4: Modeling

by HUANG Sheng Yun -
Number of replies: 3

A Generalized Model with Internal Restrictions on Item Difficulty for Polytomous Items

The standard item response model with internal restrictions on item difficulty (MIRID) usually considers component items and composite items into the model. A component item is to measure the specific component and the item difficulty of a composite item is a combination of several component items. The test developer is focus on measuring the latent trait that composite is designed to measure. Existed model with MIRID is only for dichotomous items, thus the authors proposed the extend model to be used in polytomous items. The present paper proposed two approaches of polytomous items: cumulative logits (i.e. CL-MIRID) and adjacent-category logits (i.e. ACL-MIRID). Also, the authors considers above models into multilevel framework (i.e. latent trait is regress of gender in the study). A series of simulations and an empirical data were conducted and analyzed here. As regards the simulations study, results shown that the 2P-ML-CL-MIRID and the 2P-ML-ACL-MIRID models have well parameters recovery. In terms of empirical data, results illustrated that the 2P-ML-CL-MIRID has the best data fit, followed by the 2P-ML-ACL-MIRID. Overall, cumulative logits approach is better than adjacent-category logits one.

Comments, Questions and Future Study

1) A model itself is not attractive to me, it seems like we just used mathematic symbols to create another complex model. However, the authors used proposed models to fit the guilt data, models and data are meaningful at the same time. Data follows the assumption of models, so that models can be effectively explained, thus we can obtain meaningful information.

2) The authors suggested that the future studies can release some constrains on regression weight of thresholds for component items and error for composite items. This kind of adjustment allows model having more flexibility. On my opinion, it is reasonable and easily to be implemented in the presented study due to the fact that the corresponding data is existed. However, if the corresponding data does not exist on the world, we have to think about the necessary on creating complex models.

3) Maybe we can apply DIF and Person-fit procedures on the proposed models.

In reply to HUANG Sheng Yun

Re: Topic 4: Modeling

by HUANG Sheng Yun -

Formulation and Application of the Generalized Multilevel Facets Model

This paper developed a generalized facets model for multilevel data. As we have known, the facets model was proposed by Linacre in 1989 to deal with more than two traits which are person trait and item character. However, Linacre’s model only takes the third trait of rater severity into consideration. Also, this model belongs to the Rasch family. To enhance flexibility and property of data-model fit, the authors extend the facets model to a more generalized one with discrimination parameter, as called generalized facets model. On the other hand, multilevel data was also taken into consideration. Therefore, structure latent regression was added into the generalized facets model for dealing with problem of measurement error. Two simulations were conducted, the first one is for dichotomous items and the second one is for polytomous items. It was found that most parameters were well recovery. Finally, an empirical data was analyzed by 8 models of nested model of the proposed model. Results illustrated that the proposed model had the best data-model fit.

Main idea of the new method

1) The Generalized Multilevel Facets Model

Equation (14)

Equation (5) and (6)

2) The proposed model belongs to the NLMIXED model, so it can be estimated by SAS software.

3) Bias, RMSE, Variance, Z statistic, and the absolute value of relative bias are the dependent variables for showing performance of parameters recovery.

Comments, Questions and Future Study

1) This paper provided a good paradigm of modeling study, two simulations for dichotomous and polytomous items, respectively. Also, simulations results shown that well recovery for parameters can be obtained by the proposed model. Then, an empirical data was analyzed by this model.

In reply to HUANG Sheng Yun

Re: Topic 4: Modeling

by HUANG Sheng Yun -

Modeling Randomness in Judging Rating Scales with a Random-Effects Rating Scale Model

This paper proposed two models that consider thresholds as random-effect in the rating scale model (RSM) and partial credit model (PCM). The rationale of the RE-RSM and the CRE-PCM allow each item has different threshold difficulty for individual. A series of simulations were conducted by generating separately responses data from the RSM and the RE-RSM, and then analyzing by the aforementioned models. Results illustrated that data generated by the RSM, the well recovery on item difficulties for using the RSM and the RE-RSM to fit the data. However, when the data generated from the RE-RSM, it would result biased estimation when the simple RSM was fit to the data. In the end, an empirical data was analyzed by the RSM and the proposed model.

Main idea of the new method

1) RSM (refer to Fig a)

Each item has the same set of threshold difficulties across all items and individuals. It means that first item, second item and the last item have the same 1st, 2nd, 3rd, and 4th thresholds.

2) PCM (refer to Fig b)

Items with different set of threshold difficulties but for individuals there is the same difficulty set each item. That is, item one’s thresholds are different from item two, and all the other items.

3) RE-RSM (refer to Fig c)

Different item thresholds for different persons which is that person A has a set of item thresholds different from person B, C,…, and the last person. However, variable range of thresholds is the same across all items.

4) RE-PCM (refer to Fig d)

Thresholds of item belongs to random-effect and person related, so different thresholds and different variable range of thresholds across all items for each person.

5) CRE-PCM (refer to Fig e)

Set a constraint into RE-PCM which is let variable arrange of thresholds be the same over all items.

In reply to HUANG Sheng Yun

Re: Topic 4: Modeling

by HUANG Sheng Yun -

The Cognitive-Miser Response Model: Testing for Intuitive and Deliberate Reasoning

The process of cognitive reasoning in literary has been widely studied in the field of cognitive psychology. For some reasoning vignettes, researchers found that people in resolving questions may involve two systems: intuitive process (system 1) and deliberate process (system 2). System 1 depicts that people may give answers immediately and this system is effortless and unconscious; however, system 2 describes that people may response slowly and the system is effortful and deliberative. This paper proposed a psychometric model which considers these two systems and applied it to facilitate the analysis of dual-process, the assessment of individual factors, and conditions that favors one than another.

Main idea of the new method

1) The Cognitive-Miser Response Model

2) The Extended Cognitive-Miser Response Model A

3) Extended Cognitive-Miser Response Model B

Comments, Questions and Future Study

1) This study adopted two dimensions for depicting person abilities while solving reasoning vignettes. It’s quite interesting that the paper considers cognitive process. For classical cognitive psychology, it seems easily to create different paradigms by conducting experiments. However, it’s difficult to find the whole picture without quantity information. The study connected cognitive psychology and proposed psychometric model to provide additional useful information to such issue. However, this kind of item seems not popular for practical testing.