Testing and Assessment in Translation and Interpreting Studies: A call for dialogue between research and practice (American Translators Association Scholarly Monograph Series, Volume 14)

Testing and Assessment in Translation and Interpreting Studies examines issues of measurement that are essential to translation and interpreting. Conceptualizing testing both as a process and a product, the collection of papers explores these issues across languages and settings (including university classrooms, research projects, the private sector, and professional associations). The authors have approached their chapters from different perspectives using a variety of methods, some focusing on very specific variables, and others providing a much broader overview of the issues at hand. Chapters range from a discussion of the measurement of text cohesion in translation; the measurement of interactional competence in interpreting; the use of a particular scale to measure interpreters’ renditions to the application of a specific approach to grading or general program assessment (such as interpreter or translator certification at the national level or program admissions processes). These studies point to the need for greater integration of research and practice in the specific area of testing and assessment and are a welcome addition to the field. John Benjamins Publishing Company Testing and Assessment in Translation and Interpreting Studies 2 Editorial page 3 Title page 4 LCC data 5 Table of contents 6 Introduction. Testing and assessment in translation and interpreting studies 8 References 16 Part I. The development of assessment instruments 18 Using a rubric to assess translation ability 20 Introduction 20 1. Initial considerations 21 1.1 Questions preceding test development 22 1.2 Nature of the test 22 1.3 Validity 23 1.4 Reliability 24 1.5 Test authenticity 27 1.6 Task authenticity 28 2. Defining the test construct 29 3. Review of relevant literature 31 4. An expanded framework for an expanded construct 37 5. Using a rubric 45 6. Levels of performance of a rubric 50 Conclusion 51 Limitations and implications 52 References 52 Moving beyond words in assessing mediated interaction 56 Introduction 56 Interpreter performance-based assessment in healthcare 57 Interpreting as discourse 60 Measuring interactional competence in healthcare interpreters 67 Conclusion 73 References 74 Part II. The development of assessment instruments 78 The perks of norm-referenced translation evaluation 80 1. Introduction 80 2. Research hypothesis 83 3. Method 84 3.1 Participants 84 3.2 Materials and procedure 85 3.3 Data processing 86 4. Results and discussion 87 5. Conclusion: Implications and limitations of the study 94 5.1 Implications 94 5.2 Limitations 95 References 95 Appendix 1. 96 Appendix 2. 98 Appendix 3. 98 Appendix 4. 100 Revisiting Carroll's scales 102 1. Introduction 102 1.1 Purpose and research questions 103 2. Background 104 2.1 Carroll’s scales 104 2.2 Applying grading scales to interpreting 106 3. Data and method 108 3.1 Adaptation of the scales 108 3.2 Eliciting material 111 3.3 The grading procedure 114 3.4 Measuring significant difference and inter-rater reliability 115 4. Results 116 4.1 Inter-rater reliability 116 4.2 Intelligibility 116 4.3 Informativeness 119 4.4 Spontaneous comments from graders 120 5. Discussion 121 5.1 Limitations 121 5.2 Discussion of the results 122 5.3 Conclusion 124 References 124 Appendix 1. 126 Appendix 2. 127 Meaning-oriented assessment of translations 130 Introduction 130 Background 131 NAATI assessment criteria for translation tests 132 Underlying theories of meaning-oriented assessment criteria 134 Meaning-oriented assessment criteria 142 Applications in a pedagogical context 152 Limitations and further study 158 References 160 Appendix 1 163 Assessing cohesion 166 Introduction 166 Using corpora in translation assessment 167 Methods 171 Discussion 178 Conclusions 186 Note 188 References 188 Assessing software localization 192 Introduction 192 Localization: A process of adaptation 193 What is software and how is it developed? 194 What is software quality? How is it evaluated and measured? 195 Localized software quality: A problem of perspective 196 Localized software quality and quality assessment: The vendor’s (or practitioner’s) perspective 197 Software localization processes 197 Linguistic testing 200 Automatic linguistic testing versus manual linguistic testing 205 Manual linguistic testing: Translation quality in localization 205 Accuracy, equivalence, and consistency in manual linguistic testing: What constitutes an error? 208 Cosmetic testing 217 Functional testing 219 Localization testing from the vendor’s (or practitioner’s) perspective: Implications 220 Localized software quality: The client reviewer’s (end-user’s) perspective 222 Conclusion 225 References 227 Part III. Professional certification 230 The predictive validity of admission tests for conference interpreting courses in Europe 232 Introduction 232 Admissions testing 233 Basic concepts of aptitude testing 235 Development of new aptitude tests 237 Validation of existing admissions tests 240 Admissions tests and final exams at Wilhelm University 241 Discussion: Issues raised 247 Conclusion 248 References 250 Getting it right from the start 254 Study procedure 268 Study 1: Survey instrument 270 Results and discussion 274 Study 2: Program admission test 280 Results and discussion 290 Conclusion 296 Acknowledgments 297 References 298 Standards as critical success factors in assessment 304 Introduction 304 Social interpreting in a context of increasing rationality 305 The certification of social interpreters in Flanders, Belgium 313 The certification exam procedure 314 The professional standard for social interpreters 322 Professional competency profile for graders 328 Conclusion 332 References 333 Appendix 1. 336 Assessing ASL-English interpreters 338 Introduction 338 The Canadian context 339 The Australian context 352 The United States context 353 Comparing testing models in Australia, Canada and the United States 354 The new AVLIC testing model 355 The new four-step testing process 358 Considerations arising from the new testing model 371 Summary and conclusion 375 References 375 Appendix 1 378 Introduction 378 I DISCOURSE STRATEGIES 379 II FORM 379 Appendix 2 380 Introduction 380 I MESSAGE PROCESSING 381 II INTERPRETING SUB-TASKS 381 III MISCUE PATTERNS 382 IV ADDITIONAL OBSERVATIONS 382 References 382 Author index 384 Subject index 388 The American Translators Association Scholarly Monograph Series 394 9027231907,9789027231901 Introduction: Testing And Assessment In Translation And Interpreting Studies: A Call For Dialogue Between Research And Practice / Claudia V. Angelelli And Holly E. Jacobson -- Part I. The Development Of Assessment Instruments: Theoretical Applications. Using A Rubric To Assess Translation Ability: Defining The Construct / Claudia V. Angelelli -- Moving Beyond Words In Assessing Mediated Interaction: Measuring Interactional Competence In Healthcare Settings / Holly E. Jacobson -- Part Ii. The Development Of Assessment Instruments: Empirical Approaches. The Perks Of Norm-referenced Translation Evaluation / June Eyckmans, Philippe Anckaert And Winibert Segers -- Revisiting Carroll's Scales / Elisabet Tiselius -- Meaning-oriented Assessment Of Translations: Sfl And Its Application To Formative Assessment / Mira Kim -- Assessing Cohesion: Developing Assessment Tools On The Basis Of Comparable Corpora / Brian James Baer And Tatyana Bystrova-mcintyre -- Assessing Software Localization: For A Valid Approach / Keiran Dunne -- Part Iii. Professional Certification: Lessons From Case Studies. The Predictive Validity Of Admission Tests For Conference Interpreting Courses In Europe: A Case Study / Šárka Timarová And Harry Ungoed-thomas -- Getting It Right From The Start: Program Admission Testing Of Signed Language Interpreters / Karen Bontempo And Jemina Napier -- Standards As Critical Success Factors In Assessment: Certifying Social Interpreters In Flanders, Belgium / Hildegard Vermeiren, Jan Van Gucht And Leentje De Bontridder -- Assessing Asl-english Interpreters: The Canadian Model Of National Certification / Debra Russell And Karen Malcolm. Edited By Claudia V. Angelelli, Holly E. Jacobson. Includes Bibliographical References And Indexes.