Oecd Skills Studies The Survey Of Adult Skills Reader’s Companion, Third Edition

This edition of the Reader's Companion accompanies Skills Matter: Additional Results from the Survey of Adult Skills that reports the results from the 39 countries and regions that participated in the 3 rounds of data collection in the first cycle of PIAAC, with a particular focus on the 6 countries that participated in the third round of the study (Ecuador, Hungary, Kazakhstan, Mexico, Peru and the United States). It describes the design and methodology of the survey and its relationship to other international assessments of young students and adults. The Survey of Adult Skills, a product of the OECD Programme for the International Assessment of Adult Competencies (PIAAC), was designed to provide insights into the availability of some key skills in society and how they are used at work and at home. The first survey of its kind, it directly measures proficiency in several information-processing skills - namely literacy, numeracy and problem solving in technology-rich environments Foreword Acknowledgements Table of Contents Introduction What is measured? Aspects of the design and implementation of the survey How results are reported How the survey relates to other work on measuring and assessing skills and human capital Notes References Chapter 1. What the Survey of Adult Skills (PIAAC) measures Some major features of the assessment An assessment of key information-processing competencies A use-oriented conception of competency Proficiency as a continuum The importance of contextual information Box 1.1 Competencies or skills? An overview of literacy, numeracy and problem solving in technology-rich environments Table 1.1 Summary of assessment domains in the Survey of Adult Skills (PIAAC) Literacy Definition Content Cognitive strategies Contexts Distribution of test items by task characteristics Table 1.2 Distribution of literacy items by medium Table 1.3 Distribution of literacy items by context Table 1.4 Distribution of literacy items by cognitive strategy Literacy sample items Reading components Examples of reading component items Numeracy Definition Content Representations of mathematical information Cognitive strategies Contexts Distribution of test items by task characteristics Table 1.5 Distribution of numeracy items by response type Table 1.6 Distribution of numeracy items by context Table 1.7 Distribution of numeracy items by mathematical content Numeracy sample items Problem solving in technology-rich environments Definition Content Cognitive strategies Contexts Distribution of test items by task characteristics Table 1.8 Distribution of problem-solving tasks by cognitive dimensions Table 1.9 Distribution of problem-solving tasks by technology dimensions Table 1.10 Distribution of problem-solving tasks by context Table 1.11 Distribution of problem-solving tasks by intrinsic complexity (number of steps) Table 1.12 Distribution of problem-solving tasks by intrinsic complexity (number of constraints) Table 1.13 Distribution of problem-solving tasks by explicitness of problem statement Problem-solving sample item Notes References Chapter 2. The background questionnaire of the Survey of Adult Skills (PIAAC) The characteristics and background of respondents Table 2.1 Data collected concerning the characteristics and background of respondents Educational attainment and participation in learning activities Table 2.2 Information collected regarding educational experience and current participation in learning activities Labour force status, work history and job characteristics Table 2.3 Information collected regarding labour force status, work history and job characteristics Social participation and health Table 2.4 Information collected on social participation and health The use of skills Engagement as a component of proficiency The role of literacy and numeracy practices and computer use in maintaining and developing skills Comparative information on a broader range of key skills Demand for skills Skills use: Task clusters Table 2.5 Information collected regarding tasks and activities in work and everyday life Table 2.6 Skills use indicators Box 2.1 Using Item Response Theory to derive skills use indicators in the Survey of Adults Skills (PIAAC) Table 2.7 Information collected on aspects of qualifications and skills match/mismatch Box 2.2 Deriving the survey’s measures of skills mismatch in literacy, numeracy or problem solving Notes References Chapter 3. The methodology of the Survey of Adult Skills (PIAAC) and the quality of data Table 3.1 List of participating countries and economies and dates of key phases Table 3.2 Areas of activity covered by the PIAAC Standards and Guidelines Box 3.1 How the Survey of Adult Skills (PIAAC) was managed Assessment design Pathways through the cognitive assessments in the Survey of Adult Skills (PIAAC): Computer-based assessment Figure 3.1 Pathways through the cognitive assessments in the Survey of Adult Skills (PIAAC): Computer-based assessment Table 3.3 Participation in the cognitive-assessment modules Sampling The target population and sampling frame Table 3.4 Sampling frames for countries/economies with registry samples Table 3.5 Sampling frames for countries using master samples Coverage of the target population Sample size Table 3.6 Sampling frames for countries/economies using area samples Table 3.7 Exclusions from target population Table 3.8 Sample size Sample design Translation and adaptation of instruments Survey administration Box 3.2 PIAAC Data collections in the United States Response rates and non-response bias analysis Table 3.9 Achieved response rates and population coverage Table 3.10 PIAAC NRBA outcome summary for countries/economies with response rates less than 70% Literacy-related non-response Table 3.11 Literacy-related non-response to the assessment: Proportion of respondents Scoring Table 3.12 Scoring of paper-based instruments: Within- and between-country agreement Overall assessment of data quality Notes References Chapter 4. Reporting the results of the Survey of Adult Skills (PIAAC) The proficiency scales Table 4.1 Probability of successfully completing items of varying difficulty for a person scoring 300 on the literacy scale Proficiency levels Table 4.2 Literacy item map Table 4.3 Numeracy item map Table 4.4 Problem solving in technology-rich environments item map Literacy and numeracy Table 4.5 Proficiency levels: Literacy and numeracy Table 4.6 Probability of successfully completing items at different difficulty levels, by proficiency score: Literacy Table 4.7 Probability of successfully completing items at different difficulty levels, by proficiency score: Numeracy Problem solving in technology-rich environments Table 4.8 Technology, task and cognitive features of problems at each of the three main levels of proficiency Table 4.9 Proficiency levels: Problem solving in technology-rich environments Table 4.10 Probability of successfully completing items at different difficulty levels by proficiency score: Problem solving in technology-rich environments A note about the reporting of problem solving in technology-rich environments Test languages and reporting Table 4.11 Test languages by country Notes References Chapter 5. Relationship of the Survey of Adult Skills (PIAAC) to other international skills surveys Countries and economies participating in the Survey of Adult Skills (PIAAC) and IALS and/or ALL Table 5.1 Countries and economies participating in IALS, ALL and PIAAC: Dates of data collection Constructs and instruments: The Survey of Adult Skills, ALL and IALS Table 5.2 Skills assessed in the Survey of Adult Skills (PIAAC), ALL and IALS Literacy Numeracy Problem solving in technology-rich environments Mode of delivery Comparability of background questions Survey methods and operational standards and procedures The target population Table 5.3 Population coverage: IALS, ALL and the Survey of Adult Skills (PIAAC) Sample design Survey operations Survey response Table 5.4 Response rates: IALS, ALL and the Survey of Adult Skills (PIAAC) Educational Attainment in IALS Summary of the relationship between the Survey of Adult Skills (PIAAC), IALS and ALL The relationship between the Survey of Adult Skills (PIAAC), LAMP and STEP LAMP Table 5.5 Skills assessed in the Survey of Adult Skills (PIAAC), STEP, LAMP, ALL and IALS STEP Notes References Chapter 6. Relationship between the Survey of Adult Skills (PIAAC) and the OECD Programme for International Student Assessment (PISA) PISA cohorts in the target population of the Survey of Adult Skills (PIAAC) Table 6.1 Age of PISA cohorts in 2011-12, 2014-15 and 2017-18 Differences in the target populations Skills assessed Table 6.2 Comparison of the Survey of Adult Skills (PIAAC) and PISA: Skills assessed Psychometric links The relationships between constructs in the domains of literacy, numeracy and problem solving Literacy Table 6.3 Comparison of the Survey of Adult Skills (PIAAC) and PISA: Literacy Numeracy Table 6.4 Comparison of the Survey of Adult Skills (PIAAC) and PISA: Numeracy Problem solving Table 6.5 Comparison of the Survey of Adult Skills (PIAAC) and PISA: Problem solving Conclusion Notes References Chapter 7. The Survey of Adult Skills (PIAAC) and “key competencies”97 The definition of key competencies What is competency? What is a key competency or skill? Table 7.1 Competency groups and examples of specific competencies in competency frameworks The Survey of Adult Skills (PIAAC) and key competencies Table 7.2 Key competencies and skills covered in the Survey of Adult Skills (PIAAC) Notes References Chapter 8. The Survey of Adult Skills (PIAAC) and the measurement of human capital Defining “human capital” Table 8.1 Components of human capital Coverage of the dimensions of human capital in the Survey of Adult Skills (PIAAC) Table 8.2 Coverage of the dimensions of human capital directly assessed in the Survey of Adult Skills (PIAAC) Educational attainment as a measure of human capital Table 8.3 Coverage of the dimensions of human capital by educational qualifications Comparing measures of human capital Table 8.4 Comparison of direct measures from the Survey of Adult Skills (PIAAC) and qualifications on four criteria Empirical evidence Enhancing the measurement of human capital Notes References Annex A. Relationship between the level of descriptors used in the Survey of Adult Skills (PIAAC) and other skills surveys A single literacy scale The definition of proficiency levels Table A.1 Location of items on the literacy scale using RP67 and RP80 Table A.2 Location of items on the numeracy scale using RP67 and RP80 Table A.3 Descriptors of literacy proficiency levels Table A.4 Descriptors of literacy proficiency levels References Annex B. Content of background questionnaires in the Survey of Adult Skills (PIAAC) and other skills surveys Table B.1 Summary of the background variables common to the Survey of Adult Skills (PIAAC), IALS and ALL Annex C. Project participants in Round 1 of the Survey of Adult Skills International consortium Expert groups Board of participating countries National project managers OECD secretariat Annex D. Project participants in Round 2 of the Survey of Adult Skills International consortium Expert groups Board of participating countries National project managers OECD secretariat Annex E. Project participants in Round 3 of the Survey of Adult Skills International consortium Expert groups Board of participating countries National project managers OECD secretariat