Quantitative Psychology: The 85th Annual Meeting of the Psychometric Society, Virtual (Springer Proceedings in Mathematics & Statistics, 353)

"This proceedings volume highlights the latest research and developments in psychometrics and statistics. It represents selected and peer-reviewed presentations given at the 85th Annual International Meeting of the Psychometric Society (IMPS), held virtually on July 13-17, 2020. The IMPS is one of the largest international meetings on quantitative measurement in education, psychology and the social sciences. It draws approximately 500 participants from around the world, featuring paper and poster presentations, symposiums, workshops, keynotes, and invited presentations. Leading experts and promising young researchers have written the included chapters. The chapters address a wide variety of topics including but not limited to item response theory, adaptive testing, Bayesian estimation, propensity scores, and cognitive diagnostic models. This volume is the 9th in a series of recent works to cover research presented at the IMPS."-- Provided by publisher Preface Contents A Rotation Criterion That Encourages a Hierarchical Factor Structure 1 Introduction 2 Methods 2.1 Proposed Rotation Criterion Function 2.2 Computational Techniques 2.3 Simulation Design 3 Results 4 Discussions References Comparison Between Different Estimation Methods of Factor Models for Longitudinal Ordinal Data 1 Introduction 2 Generalized Linear Latent Variable Models for Longitudinal Ordinal Data 3 Model Estimation 3.1 Pairwise Likelihood Approach 3.2 Dimension-Wise Quadrature Method 4 Simulation Study 5 Real Data Analysis 6 Discussion References An Efficient Scheduling Algorithm for Parallel Planar Rotations of Factors 1 Factor Rotation and Planar Rotations 1.1 General Factor Rotation Problem 1.2 Factor Rotation via Component Planar Rotations 2 Scheduling Parallel Orthogonal Planar Rotations 2.1 1-Factorizations of Complete Graphs 2.2 The Proposed Scheduling Algorithm 2.3 Case with More Than s Processing Threads 2.4 Case with Less than s Processing Threads 2.5 Case with an Odd Number of Factors 2.6 Practical Considerations 3 Simulated and Real Examples 3.1 A Simulated Text Mining Example 3.2 A Real Data Example About Human Activity Recognition by Smartphones 4 Conclusions References Explanatory Response Time Models 1 Introduction 2 Explanatory Response Time Models 2.1 Descriptive-Only Response Time Model 2.2 Person-Explanatory Response Time Model 2.3 Item-Explanatory Response Time Model 3 Real Data Example 3.1 Descriptive-Only Model 3.2 Person-Explanatory Model and Covariates 3.3 Item-Explanatory Model and Covariates 3.4 Model Estimation Results 4 Simulation 4.1 Simulation Design 4.2 Simulation Results 5 Discussion References Response Time Relationships Within Examinees: Implications for Item Response Time Models 1 Introduction 2 Method 2.1 Subjects 2.2 Test 2.3 Variables Analyzed 3 Results 3.1 Between Person and Between Item Correlations 3.2 Within Person Correlations 4 Discussion References Nonlinear Latent Effects in Diagnostic Classification Modeling Incorporating Response Times 1 Introduction 1.1 Nonlinear Joint Cognitive Diagnostic Modeling 1.2 Bayesian Parameter Estimation 2 Method 2.1 Real Data Analysis 2.2 Simulation Study 3 Results 3.1 Real Data Analysis 3.2 Simulation Study 4 Discussion References Sequential Monitoring of Aberrant Test-Taking Behaviors Based on Response Times 1 Introduction 2 Sequential Procedures for Detecting Aberrant Test-Taking Behaviors 2.1 CUSUM-Based Approach 2.2 SGLRT-Based Approach 2.3 Moving Sample Strategy 3 Simulation Study 3.1 Data Generation and Simulation Design 3.2 Evaluation 3.3 Results 4 Application to Process Data 4.1 Data 4.2 Results 5 Discussion References Estimating Approximate Number Sense (ANS) Acuity 1 Approximate Number Sense Acuity: Psychophysical Scaling Models as Item Response Models 2 An Application to Data: Magnitudes 2.1 Least Squares Estimation of ANS 2.2 Latent Trait Estimation of ANS 2.3 Results 3 Conclusion References Differences in Symbolic and Non-symbolic Measures of Approximate Number Sense 1 Introduction 2 Method 3 Measurements 4 Results 4.1 Symbolic vs Non-symbolic ANS 4.2 Relationship of ANS to Other Measures of Math Ability 5 Discussion References Formulas of Multilevel Reliabilities for Tests with Ordered Categorical Responses 1 Introduction 2 Reliabilities 2.1 CTT Approach 2.2 CFA Approach 3 Multilevel Reliabilities 3.1 Multilevel Confirmatory Factor Analysis (MCFA) 3.2 Multilevel Reliabilities 4 Multilevel Reliability for Categories Responses 5 Conclusions References Polytomous IRT Models Versus IRTree Models for Scoring Non-cognitive Latent Traits 1 Introduction 2 Traditional Polytomous IRT Models 2.1 The Graded Response Model 2.2 The Generalized Partial Credit Model 3 Non Traditional Polytomous IRT Models 3.1 IRTree Models 4 Application 4.1 Data 4.2 Analyses 4.3 Results 5 Conclusions and Discussion References On the Coefficient Alpha in High-Dimensions 1 Introduction 2 Definitions and Assumptions 3 Theorem 4 Preliminary Results 5 Proof of Theorem 1 6 Illustration with Simulated Data 7 Discussion References IRT Analysis of Dimensional Structure and Item Wording Effects 1 Introduction 1.1 Item Wording: Directionality and Orientation 1.2 Multidimensional Structure 1.3 Graded Response IRT Model 1.4 Summary 2 Method 2.1 Instrument 2.2 Participants 2.3 Analysis 3 Results 3.1 Descriptive Statistics 3.2 Model Fit 3.3 Item Parameter Estimates 3.4 Trait Score Estimates 4 Discussion References Item Level Measurement of Extreme Response Style 1 Introduction 1.1 Anchoring Vignettes 1.2 Item Response Model for Anchoring Vignette Data 2 Methods 3 Results 3.1 IRT Analysis 3.2 Model Comparison Analysis 4 Discussion and Conclusions References On the Marginal Effect Under Partitioned Populations: Definition and Interpretation 1 Introduction 2 Global Marginal Effect 2.1 Definition of the Global Marginal Effect 2.2 Interpretation of the Global Marginal Effect 3 Application 3.1 Results 4 Conclusions and Discussion A.1 Invariant Property of the Global Marginal Effect References Range-Preserving Confidence Intervals and Significance Tests for Scalability Coefficients in Mokken Scale Analysis 1 Introduction 2 Sampling Distribution of Scalability Coefficients 2.1 Wald-Based Methods 2.2 Range-Preserving Methods 2.3 Approximating the Sampling Distribution 3 Simulation Study 4 Implementation in Software 5 Discussion References Equating Nonequivalent Groups Using Propensity Scores: Model Misspecification and Sensitivity Analysis 1 Introduction 2 Kernel Equating 3 Propensity Scores 4 Simulation Study 4.1 Equating Estimators 4.2 Simulation Design 4.3 Evaluation Measures 4.4 Results 5 Discussion References Possible Factors Which May Impact Kernel Equating of Mixed-Format Tests 1 Introduction 2 IRT Kernel Equating 3 Simulation Study 4 Results 5 Discussion References Population Invariance of Equating for Subgroups Differing in Achievement Level 1 Introduction 2 Methodology 2.1 Data and Subgroups 2.2 Equating Analysis 2.3 Comparisons of Equating Results 3 Results 3.1 Comparisons Between Results Based on the Ranked and Random Samples for Equipercentile Equating 4 Conclusions and Discussion References Comparison of Outlier Detection Methods in NEAT Design 1 Introduction 2 Data 3 Methods 3.1 Outlier Manipulation 3.2 Outlier Detection Methods 3.3 Simulation Procedures 3.4 Evaluation Criteria 4 Results 4.1 Baseline Conditions Without Outliers 4.2 Simulated Conditions with Outliers 5 Conclusion and Discussion References An Illustration on the Quantile-Based Calculation of the Standard Error of Equating in Kernel Equating 1 Introduction 2 Equating and the Standard Error of Equating 2.1 Kernel Equating 2.2 SEE in Kernel Equating 3 Quantile-Based Estimation of SEE 3.1 Quantile-Based SEE in KE 4 Illustration 4.1 Data 4.2 Analyses 4.3 Results 5 Discussion References Improving the Measurement Efficiency in Test Construction Related to Cognitive Diagnosis Models 1 Introduction 1.1 CDM 1.2 CDI and ADI 1.3 MCDI and MADI 2 Method 2.1 rCDI and rADI 2.2 Simulation Design 3 Results 4 Discussion References Exploring Temporal Functional Dependencies Between Latent Skills in Cognitive Diagnostic Models 1 Methods 1.1 Dataset Description 1.2 Data Preprocessing/Design 1.3 Parameter Estimation and Model Selection 1.4 Correlation Analysis 2 Results 3 Discussion References Sample Size for Latent Dirichlet Allocation of Constructed-Response Items 1 Introduction 2 Simulations for Latent Dirichlet Allocation 2.1 Latent Dirichlet Allocation (LDA) 2.2 Prior Specification 2.3 Data Generation 2.4 Model Estimation 2.5 Label Switching 2.6 Evaluation 3 Simulation Study 3.1 Design 3.2 Results 4 Discussion References The Asymptotic Power of the Lagrange Multiplier Tests for Misspecified IRT Models 1 Introduction 2 The Lagrange Multiplier Test 2.1 Approximation Procedures for the Asymptotic Power 3 The Generalized Lagrange Multiplier Test 3.1 Estimation Procedure for the Noncentrality Parameter 4 Simulation Study 4.1 Simulation Design 4.2 Results 5 Conclusion References Residual Analysis in Rasch Counts Models 1 Introduction 2 Rasch Counts Model 2.1 Alternative Counts Distribution 2.2 Alternative RC Models 3 Estimation Methods 4 Residual Analysis 5 Application 6 Final Comments References A Bayesian Solution to Non-convergence of Crossed Random Effects Models 1 Introduction 2 Crossed-Random Effects Model (CREMs) 3 Bayesian Approach 4 Simulation 5 Results 5.1 Convergence Rate 5.2 Parameter Recovery 6 Conclusion References Priors in Bayesian Estimation Under the Two-Parameter Logistic Model 1 Introduction 1.1 The 2PL Model and Priors 2 Methods 3 Results 4 Discussion Appendix A: OpenBUGS Code Appendix B: Summary of Priors and Specifications References Increasing Measurement Precision of PISA Through Multistage Adaptive Testing 1 Introduction 2 Experimental Study: Invariance of Item Parameters by Unit Order 2.1 Methods 2.2 Results 3 Simulation Study: Parameter Recovery and Measurement Accuracy 3.1 Methods 3.2 Results 4 Discussion References Simulation Studies of Item Bias Estimation Accuracy 1 Introduction 2 Related Work 3 Data 4 Method 4.1 Bootstrapped Data Set Pairs 4.2 Bootstrap-AB Estimator 4.3 Bagged Parameter Estimates and Bagged-AB Estimator 5 Results 5.1 ECPE Data Set 5.2 TIMSS Data Set 5.3 Social Psychology Data Set 6 Conclusion Appendix A: R Code-Snippets for Bootstrap-AB and Bagged-AB Estimators References Multiple Answer Multiple Choice Items: A Problematic Item Type? 1 Introduction 1.1 Suggestions for MAMC Scoring 1.2 Local Dependence and Gaming Behavior in MAMC Items 1.3 User Perception of MAMC Items 1.4 Examinee Responses to Novel Item Types 2 Method 2.1 Participants 2.2 Instrument 2.3 Analysis Plan and Justification 3 Results 3.1 Latent Class Analysis Results 3.2 Jaccard's Distance Results 4 Conclusions and Next Steps References Modified Method of Drawing Classical ICCs Comparable to IRT-Based ICCs 1 Introduction 1.1 Classical ICCs 1.2 Purpose of Study 2 Proposed Method 2.1 Lateral Coordinates of Group Points 2.2 Score-Based Grouping and Scale-Based Grouping 2.3 Mean Scales of Groups for Scale-Based Grouping 3 Evaluation 3.1 Performance Index 3.2 Simulation 1: Examples of Modified Classical ICCs 3.3 Simulation 2: Evaluation of Proposed Method 3.4 Practical Example 4 Conclusion Appendix References Ontological and Methodological Barriers to the Incorporation of Event Data in Psychometric Models 1 Ontological Difficulties: What Is Meant by ``Process Data?'' 1.1 Terminological Overabundance as Evidence of an Emerging Understanding 1.2 Towards a General Approach 2 Moving Forward: Necessary Conditions for Building Shared Knowledge in Process Data Research 3 Conclusion References Psychometrics for Forensic Fingerprint Comparisons 1 Introduction 2 Data 3 Item Response Trees 3.1 Model for Fingerprint Comparisons 4 Results 4.1 Participant Parameters 4.2 Generating an `Answer Key' from Item Parameters 5 Discussion References After Thematic Analysis: Introducing the Fuzzy Thematic Network Analysis in Psychological Research 1 Introduction 1.1 Structure of a Thematic Network (TN) 1.2 After Thematic Network 2 Fuzzy Thematic Network Analysis (FTNA) 3 Practical Example 4 Conclusion References Psychometric Models for a New State Science Assessment Aligned to the Next Generation Science Standards 1 Introduction 2 Science Pilot Overview 3 Dimensionality Analysis 3.1 Description of Four Datasets and Three IRT Models 3.2 Unidimensional IRT Model (Model 1) 3.3 Three-Dimensional IRT Model (Model 2) 3.4 Testlet Model (Model 3) 3.5 IRT Model-Fit Comparisons 3.6 Item Fit Statistics 3.7 Local Dependency Among Items Within a Task 4 Discussion References Diagnostic Classification Using a Polytomous Measure of Korean Organizational Commitment 1 Introduction 1.1 Recent Polytomous DCMs 2 MORDM 3 Analysis of Data 3.1 Specification of the MORDM 3.2 Fitting the Model 3.3 RMSEA Item-Fit 3.4 Posterior Prediction Model Checking 4 Results 4.1 Item Parameters 4.2 Fit Indices 5 Discussion References An Empirical Study of Developing Automated Scoring Engine Using Supervised Latent Dirichlet Allocation 1 Introduction 1.1 Constructed Response Items and Its Scoring 1.2 Topic Models and Supervised Latent Dirichlet Allocation 2 Methods 2.1 Supervised Latent Dirichlet Allocation 2.2 Data 2.3 Evaluation Criteria 3 Results 3.1 Topic Numbers Selection 3.2 Classification Results 3.3 Unigram Model 4 Discussion and Conclusion References Where the Choice of Model Leads Us: An Empirical Comparison of Dyadic Data Analysis Frameworks 1 Introduction 2 Overview of Dyadic Data Analysis Models 2.1 Actor-Partner Interdependence Model 2.2 Common Fate Model 3 Empirical Analysis 3.1 Research Questions 3.2 Data and Variables 3.3 Analysis Models and Results 4 Discussion References Generalized Additive Modeling for Learning Trajectories in E-Learning Environments 1 Introduction 1.1 Technology-Enhanced E-Learning 1.2 Generalized Additive Mixed Model 1.3 Purpose of Study 2 Application to Real-Life Data 2.1 Statistics Online Data 2.2 Models and Methods 2.3 Results 3 Conclusion and Implication Appendix References Students Ratings Their Open Classroom Discussion 1 Introduction 2 Referent Shift Items 3 Methods 3.1 Selected Data and Measures for Illustrations 4 Problem 1: Wrong Inference Model 5 Problem 2: Students Rating Agreement Variability 6 Conclusion and Discussion References A Generalizability Study of Teach, a Classroom Observation Tool 1 Introduction 1.1 Teach, a Classroom Observation Tool 1.2 Generalizability Theory Framework 2 Methods 2.1 Participants 2.2 Instrument 2.3 Psychometric Model and Data Analysis 3 Results 3.1 Descriptive Statistics 3.2 Generalizability Study 4 Discussion Appendix: Code to Estimate G Models in R Using Teach Data References Index