CLEF

Projektbeschreibung

Das Cross-Language Evaluation Forum (CLEF) etabliert in Kooperation mit TREC (Text Retrieval Conference) eine Evaluationsumgebung für mehr­sprachiges Infor­ma­tion Retrieval, die mehrere europäische Sprachen umfasst. Beteiligt sind zahl­reiche europäische Partner (http://www.clef-campaign.org). Die Partner schufen ein mehr­­spra­chiges Korpus, das voll­ständige Jahrgänge nationaler Zeitungen umfasst. Jedes Jahr formulieren die Institute Testaufgaben in ihrer Sprache, die mit dieser Daten­menge beant­wortet werden können.

Die Informationswissenschaft der Universität Hil­des­heim nahm von 2002 bis 2010 mit selbst entwickelten Systemen an den Retrievaltests im Rahmen von CLEF teil. Das System wurde 2003 anhand des viersprachiges Korpus evaluiert und schnitt als bestes „newcomer“ System ab. 2005 und 2006 erreichte Hildesheim mit dem ersten mehr­sprachigen Web-Korpus das beste Ergebnis für sprachübergreifendes Ergebnis.

Das IWIST hat für die Forschungs-Community die Koordination von Tracks und damit die Gestaltung neuer Evaluierungsaufgaben übernommen. So wurde 2005 und 2006 der Robust Track entwickelt, 2007 und 2008 der GeoCELF Track koordiniert und seit 2009 der LogCLEF Track initiiert.

Publikationen

Mandl, T.; Di Nunzio, G.; Schulz, J. M. (2010): LogCLEF 2010: the CLEF 2010 Multilingual Logfile Analysis Track Overview. In: Working Notes 11th Workshop of the Cross-Language Evaluation Forum, CLEF 2010. URL clef2010.org/resources/proceedings/LogCLEF-2010-Overview.pdf

Becks, D.; Mandl, T.; Womser-Hacker, C. (2010): Phrases or Terms? The Impact of Different Query Types In: Working Notes 11th Work¬shop of the Cross-Language Evaluation Forum, CLEF 2010. URL clef2010.org/resources/proceedings/clef2010labs_submission_99.pdf

Agirre, E.; Di Nunzio, G.; Ferro, N.; Mandl, T.; Peters, C. (2010): CLEF 2009 Ad Hoc Track Overview: Robust-WSD Task. In: Multilingual Information Access Evaluation I: Text Retrieval Experiments: Proceedings 10th Workshop of the Cross-Language Evaluation Forum, CLEF 2009, Corfu, Greece. Revised Selected Papers. Berlin et al.: Springer [Lecture Notes in Computer Science]  

Mandl, T.; Agosti, M.; Di Nunzio, G.; Yeh, A., Mani, I.; Doran, C.; Schulz, J. M. (2010): LogCLEF 2009: the CLEF 2009 Cross-Language Logfile Analysis Track Overview. In: Multilingual Information Access Evaluation I: Text Retrieval Experiments: Proceedings 10th Workshop of the Cross-Language Evaluation Forum, CLEF 2009, Corfu, Greece. Revised Selected Papers. Berlin et al.: Springer [Lecture Notes in Computer Science]

Lamm, K.; Koelle, R.; Mandl, T. (2010): Search Path Visualization and Session Success Evaluation with Log Files from The European Library (TEL). In: Multilingual Information Access Evaluation I: Text Retrie¬val Experiments: Proceedings 10th Workshop of the Cross-Language Evaluation Forum, CLEF 2009, Corfu, Greece. Revised Selected Papers. Berlin et al.: Springer [Lecture Notes in Computer Science]

Becks, D.; Womser-Hacker, C.; Mandl, T.; Kölle, R. (2010): Patent Retrieval Experiments in the Context of the CLEF IP Track 2009. In: Multilingual Information Access Evaluation I: Text Retrieval Experiments: Pro¬ceedings 10th Workshop of the Cross-Language Evaluation Forum, CLEF 2009, Corfu, Greece. Revised Selected Papers. Berlin et al.: Springer [Lecture Notes in Computer Science]

Agirre, E.; Di Nunzio, G.; Ferro, N.; Mandl, T.; Peters, C. (2009): CLEF 2008: Ad Hoc Track Overview. In: . Evaluating Systems for Multilingual and Multimodal Access: 9th Work¬shop of the Cross-Language Evaluation Forum, CLEF 2008, Aarhus, Den¬mark, Revised Selected Papers. Berlin et al.: Springer [Lecture Notes in Computer Science 5706]

Mandl, T.; Carvalho, P.; Gey, F.; Di Nunzio, G.; Larson, R.; Santos, D., Womser-Hacker, C. (2009): GeoCLEF 2008: the CLEF 2008 Cross-Language Geographic Information Retrieval Track Overview. In: Evaluating Systems for Multilingual and Multimodal Access: 9th Workshop of the Cross-Language Evaluation Forum, CLEF 2008, Aarhus, Den¬mark, Revised Selected Papers. Berlin et al.: Springer [Lecture Notes in Computer Science 5706]

Di Nunzio, G.; Ferro, N.; Mandl, T.; Peters, C.(2008): CLEF 2007: Ad Hoc Track Overview. In: 8th Workshop of the Cross-Language Evaluation Forum, CLEF 2007, Budapest, Hungary, Revised Selected Papers. Berlin et al.: Springer [Lecture Notes in Computer Science 5152]  

Mandl, T.; Gey, F.; Di Nunzio, G.; Ferro, N.; Larson, R.; Sanderson, M.; Santos, D.; Womser-Hacker, C.; Xing, X. (2008): GeoCLEF 2007: the CLEF 2007 Cross-Language Geographic Information Retrieval Track Overview. In: 8th Workshop of the Cross-Language Evaluation Forum, CLEF 2007, Budapest, Hungary, Revised Selected Papers. Berlin et al.: Springer [Lecture Notes in Computer Science 5152]

Kölle, R.; Heuwing, B.; Mandl, T.; Womser-Hacker, C. (2008): Mono- and Crosslingual Retrieval Experiments with Spatial Restrictions at GeoCLEF 2007. In: 8th Workshop of the Cross-Language Evaluation Forum, CLEF 2007, Budapest, Hungary, Revised Selected Papers. Berlin et al.: Springer [Lecture Notes in Computer Science 5152]

Heuwing, B.; Mandl, T. (2008): Robust Retrieval Experiments at the University of Hildesheim. In: 8th Workshop of the Cross-Language Evaluation Forum, CLEF 2007, Budapest, Hungary, Revised Selected Papers. Berlin et al.: Springer [Lecture Notes in Computer Science 5152]

Gey, F.; Larson, R.; Sanderson, M.; Bishoff, K.; Mandl, T.; Womser-Hacker, C.; Santos, D.; Rocha, P.; Di Nunzio, G.; Ferro, N. (2007): Challenges to Evaluation of Multilingual Geographic Information Retrieval in GeoCLEF. In: Proceedings EVIA 2007: The First International Workshop on Evaluating Information Access (NTCIR-6 Pre-Meeting Workshop) May 15, 2007 National Institute of Informatics (NII), Tokyo, Japan. URL research.nii.ac.jp/ntcir/workshop/OnlineProceedings6/EVIA/16.pdf

Di Nunzio, G.; Ferro, N.; Mandl, T.; Peters, C.(2007): CLEF 2006: Ad Hoc Track Overview. In: 7th Workshop of the Cross-Language Evaluation Forum, CLEF 2005, Alicante, Spain, Revised Selected Papers. Berlin et al.: Springer [Lecture Notes in Computer Science 4730]

Gey, F.; Larson, R.; Sanderson, M.; Bishoff, K.; Mandl, T.; Womser-Hacker, C.; Santos, D.; Rocha, P.; Di Nunzio, G.; Ferro, N. (2007): GeoCLEF 2006: the CLEF 2006 Cross-Language Geographic Information Retrieval Track Overview. In: 7th Workshop of the Cross-Language Evaluation Forum, CLEF 2005, Alicante, Spain, Revised Selected Papers. Berlin et al.: Springer [Lecture Notes in Computer Science 4730]

Heuwing, B.; Mandl, T.; Strötgen, R. (2007): Multilingual Web Retrieval Experiments with Field Specific Indexing Strategies for WebCLEF 2006 at the University of Hildesheim. In: 7th Workshop of the Cross-Language Evaluation Forum, CLEF 2005, Alicante, Spain, Revised Selected Papers. Berlin et al.: Springer [Lecture Notes in Computer Science 4730]

Mandl, T.; Hackl, R.; Womser-Hacker, C. (2007): Robust Ad-hoc Retrieval Experiments with French and English at the University of Hildesheim. In: 7th Workshop of the Cross-Language Evaluation Forum, CLEF 2005, Alicante, Spain, Revised Selected Papers. Berlin et al.: Springer [Lecture Notes in Computer Science 4730]

Bischoff, K.; Mandl, T.; Womser-Hacker, C. (2007): Blind Relevance Feedback and Named Entity based Query Expansion for Geographic Retrieval at GeoCLEF 2006. In: 7th Workshop of the Cross-Language Evaluation Forum, CLEF 2005, Alicante, Spain, Revised Selected Papers. Berlin et al.: Springer [Lecture Notes in Computer Science 4730].

Jensen, N.; Hackl, R.; Mandl, T.; Strötgen, R. (2006): Web Retrieval Experiments with the EuroGOV Corpus at the University of Hildesheim. In: Accessing Multilingual Information Repositories: 6th Workshop of the Cross-Language Evaluation Forum, CLEF 2005, Vienna, Austria, Revised Selected Papers. Berlin et al.: Springer [Lecture Notes in Computer Science 4022]

Strötgen, R.; Mandl, T.; Schneider, R. (2006): A Fast Forward Approach to Cross-lingual Question Answering for English and German. In: Accessing Multilingual Information Repositories: 6th Workshop of the Cross-Language Evaluation Forum, CLEF 2005, Vienna, Austria, Revised Selected Papers. Berlin et al.: Springer [Lecture Notes in Computer Science 4022]

Hackl, R.; Mandl, T.; Womser-Hacker, C. (2006): Ad-hoc Mono- and Multilingual Retrieval Experiments at the University of Hildesheim. In: Accessing Multilingual Information Repositories: 6th Workshop of the Cross-Language Evaluation Forum, CLEF 2005, Vienna, Austria, Revised Selected Papers. Berlin et al.: Springer [Lecture Notes in Computer Science 4022]

Hackl, R.; Mandl, T. (2006): Bilingual Retrieval Experiments with Social Science Documents. In: Accessing Multilingual Information Repositories: 6th Workshop of the Cross-Language Evaluation Forum, CLEF 2005, Vienna, Austria, Revised Selected Papers. Berlin et al.: Springer [Lecture Notes in Computer Science 4022]

Strötgen, R.; Mandl, T.; Schneider, R. (2006): Entwicklung und Evaluierung eines Question Answering Systems im Rahmen des Cross Language Evaluation Forum (CLEF). In: Mandl, T.; Womser-Hacker, C. (Hrsg.): Effektive Information Retrieval Verfahren in der Praxis: Proceedings Vierter Hildesheimer Evaluierungs- und Retrievalworkshop (HIER 2005) Hildesheim, 20.7.2005. Konstanz: Universitätsverlag [Schriften zur Informationswissenschaft 45]

Bischoff, K.; Mandl, T.; Womser-Hacker, C. (2006): GeoCLEF 2006: Cross-linguales geographisches Information Retrieval. In: Schaaf, M.; Althoff, K.-D. (Hrsg): LWA 2006 - Lernen - Wissensentdeckung - Adaptivität Proceedings Workshopwoche GI. 9.-11.Oktober 2006, Universität Hildesheim. Workshop Information Retrieval 2006 [Hildesheimer Informatik-Berichte] S. 89-93. URL web1.bib.uni-hildesheim.de/edocs/2007/522070280/doc/522070280.pdf

Jensen, N.; Hackl, R.; Mandl, T.; Strötgen, R. (2005): Web Retrieval Experiments with the EuroGOV Corpus at the University of Hildesheim. In: Working Notes of the 6th Workshop of the Cross-Language Evaluation Forum, CLEF 2005. Sept. 2005, Wien.

Strötgen, R.; Mandl, T.; Schneider, R. (2005): A Fast Forward Approach to Cross-lingual Question Answering for English and German. In: Working Notes of the 6th Workshop of the Cross-Language Evaluation Forum, CLEF 2005. Sept. 2005, Wien.

Hackl, R.; Mandl, T.; Womser-Hacker, C. (2005): Ad-hoc Mono- and Multilingual Retrieval Experiments at the University of Hildesheim. In: Working Notes of the 6th Workshop of the Cross-Language Evaluation Forum, CLEF 2005. Sept. 2005, Wien.

Hackl, R.; Mandl, T. (2005): Bilingual Retrieval Experiments with Social Science Documents. In: Working Notes of the 6th Workshop of the Cross-Language Evaluation Forum, CLEF 2005. Sept. 2005, Wien.

Mandl, T.; Womser-Hacker, C. (2005): How do Named Entities Contribute to Retrieval Effectiveness? In: Evaluation of Cross-Language Information Retrieval Systems. Proceedings of the CLEF 2004 Workshop. Berlin et al.: Springer [Lecture Notes in Computer Science]

Hackl, R.; Mandl, T.; Womser-Hacker, C. (2005): Mono- and Cross-lingual Retrieval Experiments at the University of Hildesheim. In: Multilingual Information Access for Text, Speech and Images: Results of the Fifth CLEF Evaluation Campaign. Berlin et al.: Springer [Lecture Notes in Computer Science 3491] S. 165-169.

Mandl, T.; Womser-Hacker, C. (2005): How do Named Entities Contribute to Retrieval Effectiveness? In: Multilingual Information Access for Text, Speech and Images: Results of the Fifth CLEF Evaluation Campaign. Berlin et al.: Springer [Lecture Notes in Computer Science 3491] S. 833 - 842.

Mandl, T.; Womser-Hacker, C. (2004): Proper Names in the Multilingual CLEF Topic Set. In: Evaluation of Cross-Language Information Retrieval Systems. Proceedings of the CLEF 2003 Workshop. Berlin et al.: Springer [Lecture Notes in Computer Science 3237] S. 21-28.

Hackl, R.; Kölle, R.; Mandl, T.; Ploedt, A.; Scheufen, J.-H.; Womser-Hacker, C. (2004): Multilingual Retrieval Experiments with MIMOR at the University of Hildesheim. In: Evaluation of Cross-Language Information Retrieval Systems. Proceedings of the CLEF 2003 Workshop. Berlin et al.: Springer [Lecture Notes in Computer Science 3237] S. 166-173.

Mandl, T.; Womser-Hacker, C. (2003): Linguistic and Statistical Analysis of the CLEF Topics. In: Evaluation of Cross-Language Information Retrieval Systems. Proceedings of the CLEF 2002 Workshop. Berlin et al.: Springer [Lecture Notes in Computer Science]

Hackl, R.; Kölle, R.; Mandl, T.; Womser-Hacker, C. (2003): Domain Specific Retrieval Experiments at the University of Hildesheim with the MIMOR System. In: Evaluation of Cross-Language Information Retrieval Systems. Proceedings of the CLEF 2002 Workshop. Berlin et al.: Springer [Lecture Notes in Computer Science]