|
Overview of the SPMRL 2013 Shared Task: Cross-Framework Evaluation
of Parsing Morphologically Rich Languages| old_uid | 13187 |
|---|
| title | Overview of the SPMRL 2013 Shared Task: Cross-Framework Evaluation
of Parsing Morphologically Rich Languages |
|---|
| start_date | 2013/12/13 |
|---|
| schedule | 11h-12h30 |
|---|
| online | no |
|---|
| location_info | salle 106 |
|---|
| summary | In this presentation, we will present the outcoumes on the first shared task on statistical parsing of morphologically rich languages. The task features data sets from nine languages (Arabic, Basque, French, German, Hebrew, Hungarian, Korean, Polish and Swedish), each available both in constituency and dependency annotation. We report on the preparation of the data sets, on the proposed parsing scenarios, and on the evaluation metrics for parsing MRLs given different representation types. We present and analyze parsing results obtained by the task participants, and then provide an analysis and comparison of the parsers across languages and frameworks, reported for gold input as well as more realistic parsing scenarios. The shared task saw submissions from seven teams, and results produced by more than 14 different systems. The parsing results were obtained in different input scenarios (gold, predicted, and raw) and evaluated using different protocols (cross-framework, cross-scenario, and cross-language). In particular, this is the first time a multilingual evaluation campaign reports on the execution of parsers in realistic, morphologically ambiguous, settings. |
|---|
| responsibles | Candito |
|---|
| |
|