The Wikicorpus is a trilingual corpus (Catalan, Spanish, English) that contains large portions of the Wikipedia (based on a 2006 dump) and has been automatically enriched with linguistic information. In its present version, it contains over 750 million words.
The corpora have been annotated with lemma and part of speech information using the open source library FreeLing. Also, they have been sense annotated with the state of the art Word Sense Disambiguation algorithm UKB. As UKB assigns WordNet senses, and WordNet has been aligned across languages via the InterLingual Index, this sort of annotation opens the way to massive explorations in lexical semantics that were not possible before.
Moreover, we also provide an open source Java-based parser for Wikipedia pages developed for the construction of the corpus.
From this page you can download:
For more information (in French), see:
NOTE: Some known issues are reported in the README. If you fix them, find other bugs in the corpus, or are interested in improving and further developing the Wikicorpus or the Java Parser, please get in touch with Gemma Boleda.
This work has been partially funded by the Spanish Government through projects KNOW (TIN2006-15049-C03-01, TIN2006-15049-C03-03), KNOW2 (TIN2009-14715-C04-01, TIN2009-14715-C04-04), and through contract JCI-2007-57-1479, as well as the European Union through the EU PASCAL2 Network of Excellence (FP7-ICT-216886). We also thank Horacio Rodríguez for help with the Java-based Wikipedia Library.