Evaluation of cross-language information retrieval systems : Second Workshop of the Cross-Language Evaluation Forum, CLEF 2001, Darmstadt, Germany, September 3-4, 2001 : revised papers

The Second Evaluation Campaign Of The Cross Language Evaluation Forum (clef) For European Languages Was Held From January To September 2001. This Campaign Proved A Great Success, And Showed An Increase In Participation Of Around 70% Com Pared With Clef 2000. It Culminated In A Two Day Workshop In Darmstadt, Germany, 3–4 September, In Conjunction With The 5th European Conference On Digital Libraries (ecdl 2001). On The First Day Of The Workshop, The Results Of The Clef 2001 Evalua Tion Campaign Were Reported And Discussed In Paper And Poster Sessions. The Second Day Focused On The Current Needs Of Cross Language Systems And How Evaluation Cam Paigns In The Future Can Best Be Designed To Stimulate Progress. The Workshop Was Attended By Nearly 50 Researchers And System Developers From Both Academia And In Dustry. It Provided An Important Opportunity For Researchers Working In The Same Area To Get Together And Exchange Ideas And Experiences. Copies Of All The Presentations Are Available On The Clef Web Site At Http://www. Clef Campaign. Org. This Volume Con Tains Thoroughly Revised And Expanded Versions Of The Papers Presented At The Work Shop And Provides An Exhaustive Record Of The Clef 2001 Campaign. Clef 2001 Was Conducted As An Activity Of The Delos Network Of Excellence For Digital Libraries, Funded By The Ec Information Society Technologies Program To Further Research In Digital Library Technologies. The Activity Was Organized In Collabo Ration With The Us National Institute Of Standards And Technology (nist). System Evaluation Experiments At Clef 2001 -- Clef 2001 — Overview Of Results -- Mainly Cross-language -- Report On Clef-2001 Experiments: Effective Combined Query-translation Approach -- Multilingual Information Retrieval Using English And Chinese Queries -- Exeter At Clef 2001: Experiments With Machine Translation For Bilingual Retrieval -- Tno At Clef-2001: Comparing Translation Resources -- Itc-irst At Clef 2001: Monolingual And Bilingual Tracks -- Experiments With The Eurospider Retrieval System For Clef 2001 -- Using Co-occurrence, Augmented Restrictions, And C-e Wordnet For Chinese-english Cross-language Information Retrieval At Clef 2001 -- Utaclir @ Clef 2001 — Effects Of Compound Splitting And N-gram Techniques -- Using Statistical Translation Models For Bilingual Ir -- Cross-lingual Pseudo-relevance Feedback Using A Comparable Corpus -- Investigation On Disambiguation In Clir: Aligned Corpus And Bi-directional Translation-based Strategies --^ Vector-based Semantic Analysis Using Random Indexing For Cross-lingual Query Expansion -- Query Expansion Techniques For The Clef Bilingual Track -- Intelligent Information Access Systems (sinai) At Clef 2001: Calculating Translation Probabilities With Semcor -- Jhu/apl Experiments At Clef: Translation Resources And Score Normalization -- Dictionary-based Thai Clir: An Experimental Survey Of Thai Clir -- English-dutch Clir Using Query Translation Techniques -- Thomson Legal And Regulatory At Clef 2001: Monolingual And Bilingual Experiments -- Working With Russian Queries For The Girt, Bilingual, And Multilingual Clef Tasks -- Ir-n: A Passage Retrieval System At Clef-2001 -- Monolingual Experiments -- Spanish Monolingual Track: The Impact Of Stemming On Retrieval -- Shallow Morphological Analysis In Monolingual Information Retrieval For Dutch, German, And Italian -- Stemming Evaluated In 6 Languages By Hummingbird Searchserver™ At Clef 2001 --^ Minimalistic Test Runs Of The Eidetica Indexer -- Across The Bridge: Clef 2001 — Non-english Monolingual Retrieval. The French Task -- Mpro-ir In Clef 2001 -- Some Terms Are More Interchangeable Than Others -- Interactive Track -- The Clef 2001 Interactive Track -- Noun Phrase Translations For Cross-language Document Selection -- Iclef At Sheffield -- Iclef 2001 At Maryland: Comparing Term-for-term Gloss And Mt -- Evaluation Issues And Results -- The Philosophy Of Information Retrieval Evaluation -- Clir System Evaluation At The Second Ntcir Workshop -- Multilingual Topic Generation Within The Clef 2001 Experiments -- Clef Methodology And Metrics. Carol Peters ... [et Al.], Eds. Includes Bibliographical References And Index. Also Available Via The World Wide Web. The Second Evaluation Campaign Of The Cross Language Evaluation Forum (clef) For European Languages Was Held From January To September 2001. This Campaign Proved A Great Success, And Showed An Increase In Participation Of Around 70% Com Pared With Clef 2000. It Culminated In A Two Day Workshop In Darmstadt, Germany, 3–4 September, In Conjunction With The 5th European Conference On Digital Libraries (ecdl 2001). On The First Day Of The Workshop, The Results Of The Clef 2001 Evalua Tion Campaign Were Reported And Discussed In Paper And Poster Sessions. The Second Day Focused On The Current Needs Of Cross Language Systems And How Evaluation Cam Paigns In The Future Can Best Be Designed To Stimulate Progress. The Workshop Was Attended By Nearly 50 Researchers And System Developers From Both Academia And In Dustry. It Provided An Important Opportunity For Researchers Working In The Same Area To Get Together And Exchange Ideas And Experiences. Copies Of All The Presentations Are Available On The Clef Web Site At Http://www. Clef Campaign. Org. This Volume Con Tains Thoroughly Revised And Expanded Versions Of The Papers Presented At The Work Shop And Provides An Exhaustive Record Of The Clef 2001 Campaign. Clef 2001 Was Conducted As An Activity Of The Delos Network Of Excellence For Digital Libraries, Funded By The Ec Information Society Technologies Program To Further Research In Digital Library Technologies. The Activity Was Organized In Collabo Ration With The Us National Institute Of Standards And Technology (nist). System Evaluation Experiments At Clef 2001 -- Clef 2001 — Overview Of Results -- Mainly Cross-language -- Report On Clef-2001 Experiments: Effective Combined Query-translation Approach -- Multilingual Information Retrieval Using English And Chinese Queries -- Exeter At Clef 2001: Experiments With Machine Translation For Bilingual Retrieval -- Tno At Clef-2001: Comparing Translation Resources -- Itc-irst At Clef 2001: Monolingual And Bilingual Tracks -- Experiments With The Eurospider Retrieval System For Clef 2001 -- Using Co-occurrence, Augmented Restrictions, And C-e Wordnet For Chinese-english Cross-language Information Retrieval At Clef 2001 -- Utaclir @ Clef 2001 — Effects Of Compound Splitting And N-gram Techniques -- Using Statistical Translation Models For Bilingual Ir -- Cross-lingual Pseudo-relevance Feedback Using A Comparable Corpus -- Investigation On Disambiguation In Clir: Aligned Corpus And Bi-directional Translation-based Strategies --^ Vector-based Semantic Analysis Using Random Indexing For Cross-lingual Query Expansion -- Query Expansion Techniques For The Clef Bilingual Track -- Intelligent Information Access Systems (sinai) At Clef 2001: Calculating Translation Probabilities With Semcor -- Jhu/apl Experiments At Clef: Translation Resources And Score Normalization -- Dictionary-based Thai Clir: An Experimental Survey Of Thai Clir -- English-dutch Clir Using Query Translation Techniques -- Thomson Legal And Regulatory At Clef 2001: Monolingual And Bilingual Experiments -- Working With Russian Queries For The Girt, Bilingual, And Multilingual Clef Tasks -- Ir-n: A Passage Retrieval System At Clef-2001 -- Monolingual Experiments -- Spanish Monolingual Track: The Impact Of Stemming On Retrieval -- Shallow Morphological Analysis In Monolingual Information Retrieval For Dutch, German, And Italian -- Stemming Evaluated In 6 Languages By Hummingbird SearchserverTM At Clef 2001 --^ Minimalistic Test Runs Of The Eidetica Indexer -- Across The Bridge: Clef 2001 — Non-english Monolingual Retrieval. The French Task -- Mpro-ir In Clef 2001 -- Some Terms Are More Interchangeable Than Others -- Interactive Track -- The Clef 2001 Interactive Track -- Noun Phrase Translations For Cross-language Document Selection -- Iclef At Sheffield -- Iclef 2001 At Maryland: Comparing Term-for-term Gloss And Mt -- Evaluation Issues And Results -- The Philosophy Of Information Retrieval Evaluation -- Clir System Evaluation At The Second Ntcir Workshop -- Multilingual Topic Generation Within The Clef 2001 Experiments -- Clef Methodology And Metrics. Carol Peters ... [et Al.], Eds. Includes Bibliographical References And Index. Also Available Via The World Wide Web. Introduction....Pages 1-5 CLEF 2001 — Overview of Results....Pages 9-26 Report on CLEF-2001 Experiments: Effective Combined Query-Translation Approach....Pages 27-43 Multilingual Information Retrieval Using English and Chinese Queries....Pages 44-58 Exeter at CLEF 2001: Experiments with Machine Translation for Bilingual Retrieval....Pages 59-77 TNO at CLEF-2001: Comparing Translation Resources....Pages 78-93 ITC-irst at CLEF 2001: Monolingual and Bilingual Tracks....Pages 94-101 Experiments with the Eurospider Retrieval System for CLEF 2001....Pages 102-110 Using Co-occurrence, Augmented Restrictions, and C-E WordNet for Chinese-English Cross-Language Information Retrieval at CLEF 2001....Pages 111-117 Utaclir @ CLEF 2001 — Effects of Compound Splitting and N-Gram Techniques....Pages 118-136 Using Statistical Translation Models for Bilingual IR....Pages 137-150 Cross-Lingual Pseudo-Relevance Feedback Using a Comparable Corpus....Pages 151-157 Investigation on Disambiguation in CLIR: Aligned Corpus and Bi-directional Translation-Based Strategies....Pages 158-168 Vector-Based Semantic Analysis Using Random Indexing for Cross-Lingual Query Expansion....Pages 169-176 Query Expansion Techniques for the CLEF Bilingual Track....Pages 177-184 Intelligent Information Access Systems (SINAI) at CLEF 2001: Calculating Translation Probabilities with SemCor....Pages 185-192 JHU/APL Experiments at CLEF: Translation Resources and Score Normalization....Pages 193-208 Dictionary-Based Thai CLIR: An Experimental Survey of Thai CLIR....Pages 209-218 English-Dutch CLIR Using Query Translation Techniques....Pages 219-225 Thomson Legal and Regulatory at CLEF 2001: Monolingual and Bilingual Experiments....Pages 226-234 Working with Russian Queries for the GIRT, Bilingual, and Multilingual CLEF Tasks....Pages 235-243 IR-n: A Passage Retrieval System at CLEF-2001....Pages 244-252 Spanish Monolingual Track: The Impact of Stemming on Retrieval....Pages 253-261 Shallow Morphological Analysis in Monolingual Information Retrieval for Dutch, German, and Italian....Pages 262-277 Stemming Evaluated in 6 Languages by Hummingbird SearchServerTM at CLEF 2001....Pages 278-287 Minimalistic Test Runs of the Eidetica Indexer....Pages 288-290 Across the Bridge: CLEF 2001 — Non-english Monolingual Retrieval. The French Task....Pages 291-299 Mpro-IR in CLEF 2001....Pages 300-302 Some Terms Are More Interchangeable than Others....Pages 303-307 The CLEF 2001 Interactive Track....Pages 308-319 Noun Phrase Translations for Cross-Language Document Selection....Pages 320-331 iCLEF at Sheffield....Pages 332-335 iCLEF 2001 at Maryland: Comparing Term-for-Term Gloss and MT....Pages 336-352 The Philosophy of Information Retrieval Evaluation....Pages 355-370 CLIR System Evaluation at the Second NTCIR Workshop....Pages 371-388 Multilingual Topic Generation within the CLEF 2001 Experiments....Pages 389-393 CLEF Methodology and Metrics....Pages 394-404 The second evaluation campaign of the Cross Language Evaluation Forum (CLEF) for European languages was held from January to September 2001. This campaign proved a great success, and showed an increase in participation of around 70% com pared with CLEF 2000. It culminated in a two day workshop in Darmstadt, Germany, 3{u2013}4 September, in conjunction with the 5th European Conference on Digital Libraries (ECDL 2001). On the first day of the workshop, the results of the CLEF 2001 evalua tion campaign were reported and discussed in paper and poster sessions. The second day focused on the current needs of cross language systems and how evaluation cam paigns in the future can best be designed to stimulate progress. The workshop was attended by nearly 50 researchers and system developers from both academia and in dustry. It provided an important opportunity for researchers working in the same area to get together and exchange ideas and experiences. Copies of all the presentations are available on the CLEF web site at http://www. clef campaign. org. This volume con tains thoroughly revised and expanded versions of the papers presented at the work shop and provides an exhaustive record of the CLEF 2001 campaign. CLEF 2001 was conducted as an activity of the DELOS Network of Excellence for Digital Libraries, funded by the EC Information Society Technologies program to further research in digital library technologies. The activity was organized in collabo ration with the US National Institute of Standards and Technology (NIST) This book constitutes the thoroughly refereed post-proceedings of the Second Workshop of the Cross-Language Evaluation Forum, CLEF 2001, held in Darmstadt, Germany in September 2001. The 35 revised full papers presented together with two introductory survey articles and a comprehensive appendix were carefully improved during the round of reviewing and selections. The papers are organized in topical sections on systems evaluation experiments, mainly cross-language, monolingual experiments, interactive issues, and evaluation issues and results CLEF 2001 has built on the success of the first CLEF campaign in 2000 [3], which had already seen a substantial increase in the number of participating groups compared to the predecessors, the TREC cross-language (CLIR) tracks [7].