Technology Enhanced Assessment: 21st International Conference, TEA 2018, Amsterdam, The Netherlands, December 10–11, 2018, Revised Selected Papers (Communications ... Computer and Information Science Book 1014)

This book constitutes the proceedings of the 21st International Conference on Technology Enhanced Assessment, TEA 2018, held in Amsterdam, The Netherlands, in December 2018. The 14 papers presented were carefully selected from 34 submissions. They are centered around topics like e-learning, computer-assisted instruction, interactive learning environments, collaborative learning, computing education, student assessment. -- Provided by publisher Preface 6 Organization 7 Contents 9 Dynamic Generation of Assessment Items Using Wikidata 11 Abstract 11 1 Introduction 11 2 Related Work 12 3 Technical Background 13 4 Comparison of Four Sample Cases 14 4.1 Case 1: Archaeological Sites 14 4.2 Case 2: Chemical Compounds 16 4.3 Case 3: Geography Data 18 4.4 Case 4: German Governors 20 4.5 Summary 22 5 Conclusions and Future Work 23 Acknowledgements 23 References 24 Students’ Attitudes Towards Personal Data Sharing in the Context of e-Assessment: Informed Consent or Privacy Paradox? 26 Abstract 26 1 Introduction 26 1.1 General Problem 26 1.2 Research Questions 28 2 Method 29 2.1 Design 29 2.2 Participants 29 2.3 Material 29 2.4 Procedure 30 2.5 Data Analysis 30 3 Results 30 4 Conclusions and Discussion 33 5 Limitations and Future Work 34 Acknowledgement 35 References 35 Students’ and Teachers’ Perceptions of the Usability and Usefulness of the First Viewbrics-Prototype: A Methodology and Online Tool to Formatively Assess Complex Generic Skills with Video-Enhanced Rubrics (VER) in Dutch Secondary Education 37 Abstract 37 1 Introduction 37 2 The ‘Viewbrics’ Project 39 3 The Design and Functionality of the First Prototype of the Viewbrics-Online Tool 40 4 Stakeholder Evaluation Method 44 4.1 Participants 44 4.2 Materials 44 4.3 Design and Procedure 45 5 Results 46 5.1 Questionnaires 46 5.2 Card Sorting Exercise 46 6 Conclusions and Discussion 49 Acknowledgement 50 References 50 Assessing Learning in MOOCs Through Interactions Between Learners 52 Abstract 52 1 Theoretical Framework 52 1.1 Knowledge Building in Online Learning Environments 53 1.2 Learning Attribution in Online Learning Environments 54 1.3 Modelling Personal and Shared Knowledge Construction Through Discourse in a Shared Online Learning Environment 55 2 Methodology 56 2.1 Context 56 2.2 Data Analysis 56 3 Results 59 3.1 Interaction in Group Forums: Visible Patterns 59 4 Conclusion 61 References 62 Corrective Feedback and Its Implications on Students' Confidence-Based Assessment 65 1 Introduction 65 2 Background 67 3 Objectives of This Study 68 4 Experimental Study 69 4.1 Participants and the Computer-Based Assessment Environment 69 4.2 Study Design 69 4.3 Data Description 70 5 Data Analyses and Results 71 5.1 Higher Education Student Confidence Judgment in Response's Correctness 71 5.2 Sessions of Variable Lengths 71 5.3 Comparison of Feedback Seeking in Variable Lengths Sessions 72 5.4 Feedback Seeking Behavior in Distinct Confidence-Outcome Categories 72 5.5 Feedback Reading Time in Distinct Confidence-Outcome Categories 74 5.6 Impact of Feedback Seeking Behavior on Confidence-Outcome Category in Next Attempt 75 6 Discussion 80 References 82 Automated Feedback for Workplace Learning in Higher Education 83 Abstract 83 1 Introduction 83 2 Theoretical Background 84 2.1 Workplace Learning in Higher Education 84 2.2 Formative Assessment in WPL 84 2.3 Principles of Good (Automated) Feedback 86 2.4 Supporting WPL with Technology 86 2.5 Theory Gap and Research Question 89 3 Methodology 89 4 Results 91 4.1 Framework for Automated Feedback in TEWL 91 4.2 Automated Feedback: Conceptual Design and Design Principles 93 4.3 Proof of Concept 94 5 Conclusion and Discussion 96 5.1 Conclusion 96 5.2 Discussion and Future Work 97 Acknowledgements 97 References 97 Formative Assessment of Inquiry Skills for Responsible Research and Innovation Using 3D Virtual Reality Glasses and Face Recognition 101 Abstract 101 1 Introduction 102 2 Methodology 103 3 Findings 104 4 Discussion and Conclusion 108 5 Display Items 109 Acknowledgments 110 References 110 Quizbot: Exploring Formative Feedback with Conversational Interfaces 112 1 Introduction 112 2 Setting up the Frame 115 2.1 Formative Feedback, Assessment and Learning 115 2.2 Quizzes, Feedback and Confidence 116 2.3 Chatbots in Education 118 3 Designing Formative Feedback for the SQL Quizbot 119 3.1 Quiz and Feedback Design 119 3.2 Implementation 123 4 Evaluation 123 4.1 Experiment Design and Main Findings 123 4.2 Results and Discussion 125 5 Conclusions and Future Work 127 References 127 Best of Two Worlds: Using Two Assessment Tools in One Course 131 Abstract 131 1 Introduction 131 2 The E-Assessment System AlephQ 132 2.1 AlephQ as a User Friendly Building Block Based System 132 2.2 Usage and Benefits of AlephQ Within the Antwerp University 133 2.3 AlephQ: Practical Examples 134 2.4 AlephQ: Limitations 137 3 The E-Assessment System JACK 137 3.1 Usage Scenarios 138 3.2 Sample Exercises 139 3.3 Benefits for Teachers 140 3.4 Benefits for Students 142 4 The Use of the Complementary Packages AlephQ and JACK 142 4.1 The Use of AlephQ and JACK to Activate Large Groups of Students 142 4.2 The Accountancy Course 142 4.3 Use of AlephQ 143 4.4 Deploying JACK 145 4.5 Advantages to the Students 145 4.6 Advantages to the Teaching Staff 145 5 Conclusion 145 Annex: Longitudinal Statistical Analysis of Results Taking the Qualitative Evolution of the Accountancy Course into Consideration 146 References 149 Digital Exams in Engineering Education 150 Abstract 150 1 Problem Description 150 2 A Study of Digital Exams for Engineering at the Delft University of Technology 152 3 Research Questions 153 4 Method 153 4.1 Cycle 1 154 4.2 Cycle 1 Results 155 4.3 Recommendations Based on First Cycle 161 4.4 Cycles 2 and 3 163 5 Conclusion and Discussion 167 Appendix 1: Example 1—Underpinning Scenario 168 Appendix 2: Example 2—Underpinning Scenario 170 Appendix 3: Example 3—Scaffolding Scenario 171 References 173 A Cost–Benefit Analysis for Developing Item Banks in Higher Education 175 Abstract 175 1 Background 175 2 Cost and Benefits 176 2.1 Cost: Development 177 2.2 Benefits: Increasing Validity and Reliability 177 2.3 Benefits: Decreasing Misclassification in Achievement Tests 178 2.4 Benefit: Re-use 180 2.5 Example 180 3 Practical Value 182 3.1 Cost and Workload 182 3.2 Flawed Test Items and Their Impact on Item Quality 183 3.3 Discussion of the Presented Model 184 4 Conclusion and Limitations 185 References 186 Advantages of Using Automatic Formative Assessment for Learning Mathematics 190 Abstract 190 1 Introduction 190 2 Theoretical Framework 191 2.1 Automatic Formative Assessment 191 2.2 Reluctant Learners and Reluctant Users 194 3 Research Questions 194 4 Experimentation and Research Method 195 4.1 The Experimentation “Educating City” 195 4.2 Instruments 195 4.3 The Automatically Assessed Learning Materials 196 4.4 Data Collection 199 5 Results 200 6 Main Conclusions and Implications for Research and Practice 204 References 206 Getting to Grips with Exam Fraud: A Qualitative Study Towards Developing an Evidence Based Educational Data Forensics Protocol 209 Abstract 209 1 Introduction 209 1.1 Problem Description 209 1.2 Educational Data Forensics Protocol 211 1.3 Research Question and Model 212 1.4 Scientific and Practical Relevance 212 2 Method 212 2.1 Step 1 - Literature Search 213 2.2 Step 2 - Developing an EDF-Protocol Prototype 213 2.3 Step 3 - Validating the EDF-Protocol Standards 214 2.4 Adjustment of the Prototype and Final EDF Protocol 215 2.5 Implementation of the EDF Protocol 215 3 Results 215 3.1 Literature Search 215 3.2 Developing an EDF Protocol Prototype 218 3.3 Validating the EDF Protocol Standards 219 3.4 Adjustment of the Prototype and Final EDF Protocol 221 3.5 Implementation of the EDF Protocol 224 4 Conclusion 224 5 Contribution to Practice 225 5.1 Establishing a Tailor-Made Protocol 225 5.2 The Practical Value of the Protocol 226 5.3 Proper Use of Data Forensics 226 5.4 Final Words 226 References 227 Assessment of Collaboration and Feedback on Gesture Performance 229 Abstract 229 1 Introduction 229 2 Related Work 230 2.1 Collaboration on Tangible Interfaces 230 2.2 Gesture Taxonomies and Gesture Recognition 231 2.3 Feedback on Gesture Performance 232 3 Assessment Framework and Interactive Feedback Strategy 233 4 Gesture Taxonomy 234 4.1 Manual Gesture Coding 234 5 Gesture Application on Tangible User Interfaces 236 5.1 Calibration Procedure 237 5.2 Interpretation of Log Files 238 6 Conclusion and Future Prospects 239 Acknowledgments 240 References 240 Author Index 243 Front Matter ....Pages i-x Dynamic Generation of Assessment Items Using Wikidata (Michael Striewe)....Pages 1-15 Students’ Attitudes Towards Personal Data Sharing in the Context of e-Assessment: Informed Consent or Privacy Paradox? (Ekaterina Muravyeva, José Janssen, Kim Dirkx, Marcus Specht)....Pages 16-26 Students’ and Teachers’ Perceptions of the Usability and Usefulness of the First Viewbrics-Prototype: A Methodology and Online Tool to Formatively Assess Complex Generic Skills with Video-Enhanced Rubrics (VER) in Dutch Secondary Education (Ellen Rusman, Rob Nadolski, Kevin Ackermans)....Pages 27-41 Assessing Learning in MOOCs Through Interactions Between Learners (Francis Brouns, Olga Firssova)....Pages 42-54 Corrective Feedback and Its Implications on Students’ Confidence-Based Assessment (Rabia Maqsood, Paolo Ceravolo)....Pages 55-72 Automated Feedback for Workplace Learning in Higher Education (Esther van der Stappen, Liesbeth Baartman)....Pages 73-90 Formative Assessment of Inquiry Skills for Responsible Research and Innovation Using 3D Virtual Reality Glasses and Face Recognition (Alexandra Okada, Ana Karine Loula Torres Rocha, Simone Keller Fuchter, Sangar Zucchi, David Wortley)....Pages 91-101 Quizbot: Exploring Formative Feedback with Conversational Interfaces (Bharathi Vijayakumar, Sviatlana Höhn, Christoph Schommer)....Pages 102-120 Best of Two Worlds: Using Two Assessment Tools in One Course (Raoul Deuss, Christine Lippens, Michael Striewe)....Pages 121-139 Digital Exams in Engineering Education (Meta Keijzer-de Ruijter, Silvester Draaijer)....Pages 140-164 A Cost–Benefit Analysis for Developing Item Banks in Higher Education (Silvester Draaijer)....Pages 165-179 Advantages of Using Automatic Formative Assessment for Learning Mathematics (Alice Barana, Marina Marchisio, Matteo Sacchet)....Pages 180-198 Getting to Grips with Exam Fraud: A Qualitative Study Towards Developing an Evidence Based Educational Data Forensics Protocol (Christiaan J. van Ommering, Sebastiaan de Klerk, Bernard P. Veldkamp)....Pages 199-218 Assessment of Collaboration and Feedback on Gesture Performance (Dimitra Anastasiou, Eric Ras, Mehmetcan Fal)....Pages 219-232 Back Matter ....Pages 233-233 This book constitutes the proceedings of the 20th International Conference on Technology Enhanced Assessment, TEA 2017, held in Barcelona, Spain, in October 2017. The 17 papers presented were carefully selected from 42 submissions. They are centered around topics like e-learning, computer-assisted instruction, interactive learning environments, collaborative learning, computing education, student assessment. Chapter "Student perception of scalable peer-feedback design in Massive Open Online Courses" is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/). For further details see license information in the chapter.