
Testing ESL Pragmatics : Development and Validation of a Web-Based Assessment Battery.
Title:
Testing ESL Pragmatics : Development and Validation of a Web-Based Assessment Battery.
Author:
Roever, Carsten.
ISBN:
9783653047806
Personal Author:
Physical Description:
1 online resource (172 pages)
Series:
Language Testing and Evaluation ; v.2
Language Testing and Evaluation
Contents:
COVER -- ABSTRACT -- TABLE OF CONTENTS -- List of Tables -- List of Figures -- INTRODUCTION -- Goals of this project -- Organization of this book -- INTERLANGUAGE PRAGMATICS -- Definition and content issues -- Pragmalinguistics and sociopragmatics -- Grammatical and pragmatic competence in models ofcommunicative competence -- Learning of L2 pragmatics: General considerations -- Factors in the development of interlanguage pragmatics -- Pragmatic transfer -- Learning setting -- An aptitude for pragmatics? -- Components of pragmatic knowledge: Speech acts -- Apology -- Apologies in L2 research -- Requests -- Requests in L2 research -- Refusals -- Refusals in L2 research -- Components of pragmatic knowledge: Routines -- Routines in L2 research -- Components of pragmatic knowledge: Implicature -- Implicature in L2 research -- Research instruments in interlanguage pragmatics research -- Production questionnaires: Discourse Completion Tests -- Multiple Choice Questionnaires -- Conclusion -- ASSESSMENT OF SECOND LANGUAGEPRAGMATICS: PREVIOUS STUDIES ANDVALIDATION ISSUES -- Approaches to assessment of interlanguage pragmatics to date -- Validity -- Validation -- Judgmentaillogical analyses and theoretical rationales -- Quantitative analyses: Correlational analyses -- Quantitative analyses: Analyses of group differences and longitudinal change -- Analyses of processes -- Effects of intervention -- Consequential validity -- Conclusion -- THIS STUDY: PURPOSE, CONSTRUCT,RESEARCH QUESTIONS -- Research Questions -- General statistical information about the test -- Judgmentai/Logicai Analyses -- Correlational analyses -- Group differences -- Analyses of processes -- Consequential validity -- DESIGN AND PILOT STUDY -- Instrument design -- Technical aspects -- Content aspects -- Implicature -- Routines -- Speech Acts -- Visuals vs. vocabulary explanations.
Pilot Study -- Condusion -- METHODS -- Participants -- Materials -- The test -- Capturing of responses -- Procedure -- Standard group -- NS comparison group -- NS and NNS verbal protocol groups -- Oral Group -- Scoring and score computation -- Scoring of multiple-choice items -- Scoring and rating of brief-response items -- Scoring of multitrait-multimethod responses and oral responses -- Missing responses and computation of scores -- RESULTS -- General test characteristics -- What are the statistical characteristics of the test and its sections? -- What were the item characteristics? -- Were the time limits appropriate for all groups? -- What is the reliability and standard error of the test and its sections? -- What is the fit of the items to an IRT model? -- What is the relationship between rater severity, item difficulty, and testtaker ability on the speech act section? -- Judgmental / logical analyses -- Do domain experts, native speakers and test takers agree that the testmeasures pragmalinguistic knowledge? -- Correlational analyses -- What is the relationship between the items and between the sections of the test? -- How does modality (multiple choice vs. brief response vs. spoken response) affect the test outcome? -- How does test taker's familiarity with web browser affect scores? -- Is there a relationship between response times, viewing instructions,viewing vocabulary, and test taker ability? -- Comparisons of groups -- Do native speakers of English score higher than non-native speakers? -- Do learners improve their knowledge of idiosyncratic implicature through exposure? -- Do learners improve their knowledge of formulaic implicature less through exposure than their knowledge of idiosyncratic implicature? -- Is formulaic implicature more difficult than idiosyncratic implicature?.
Are situational routines much more difficult for learners with no exposure than for learners with exposure? -- Does short-term exposure (up to 3 months) lead to a measurable advantage in knowledge of routines over no exposure? -- Does knowledge of speech acts increase with proficiency? -- Analyses of processes -- Re-reading -- Interpretation and re-phrasing: Implicatures and speech acts -- Response planning (speech act items) -- Weighing of response options and justification of responses: Routines -- Consequential validity -- What is the practicality of the test? -- What inferences and decisions can the test scores support? -- What inferences and decisions can the test scores support? -- DISCUSSION -- Construct validity -- Test-internal evidence: A unifled construct? -- Divergent validity evidence: Effects of exposure and proficiency -- Threats to construct validity -- The role of sociopragmatic knowledge -- Method effects -- Reading level -- Empirical foundation of items -- Test issues -- Item difficulty and item misfit -- Mysterious Misfit: The case of item 302 -- Implications for ILP test development -- The speech act section and its rejoinders: turning a rock into a semi-soft place -- Defensible inferences, consequences, and practicality -- CONCLUSION AND OUTLOOK -- Empirical foundation of items -- New item types -- Multiple-choice items for speech acts -- Audio and video enhancement -- Testing additional aspects of pragmatic competence -- New test types -- Adaptivity -- Other target languages -- Testing in the service of learning -- Research in interlanguage pragmatics -- Reaping reciprocal benefits: Testing and SLA -- APPENDIX 1: TAXONOMY OF SITUATIONAL ROUTINES FOR L2 -- APPENDIX 2: STANDARD TEST -- APPENDIX 3: MULTIPLE-CHOICE ITEMS FOR THE SPEECH ACT SECTION IN THE MTMM VERSION.
Abstract:
Although second language learners' pragmatic competence (their ability to use language in context) is an essential part of their general communicative competence, it has not been a part of second language tests. This book helps fill this gap by describing the development and validation of a web-based test of ESL pragmalinguistics. The instrument assesses learners' knowledge of routine formulae, speech acts, and implicature in 36 multiple-choice and brief-response items. The test's quantitative and qualitative validation with 300 learners showed high reliability and provided strong evidence of the instrument's construct validity. Its web-based format makes it easy to administer and score. The book ends with a discussion of future research directions in assessing second language pragmatics.
Local Note:
Electronic reproduction. Ann Arbor, Michigan : ProQuest Ebook Central, 2017. Available via World Wide Web. Access may be limited to ProQuest Ebook Central affiliated libraries.
Genre:
Electronic Access:
Click to View