User:Pfctdayelise/Using Wikipedia as a resource for computational linguistics

From Wikibooks, the open-content textbooks collection

The aim of this book is to outline some areas of research in computational linguistics or natural language processing where Wikipedia, and by extension the other Wikimedia projects, have the potential to be valuable resources. It is not intended to serve as an introduction to either of these fields and does not assume any knowledge of Wikipedia.

[edit] Description of Wikimedia projects

Witkionary, Wikinews, Commons, Wikibooks, Wikisource, Wikiquote. Languages. Meta & Commons direct translations of help (etc) pages. Who contributes? growth.

Possibly dv.wp is the smallest project which has a translated logo. Less than 200 articles, probably less than 20 legitimate registered users, Dhivehi language, <300000 speakers, official lang of the Maldives.

[edit] Database dumps

[edit] Post(pre?)-processing tools

[edit] Description of the English Wikipedia

• License
• Accessible - dumps
• Coverage - biased to pop-culture and geek topics (best coverage), wikiprojects
• Format - MOS - but not reliable
• FAs, cleanup tags
• RDRs

[edit] Interwiki links

[edit] Categories

[edit] Disambiguation pages

[edit] Possible tasks

[edit] Word sense disambiguation

[edit] Word and phrase translation

[edit] Web mining, data mining

[edit] Machine translation

[edit] Geospatial term disambiguation and named entity recognition

[edit] Image analysis (?)

[edit] Synonymy, abbreviations (RDRs)

• folksonomy
• WordNet
• w:Semantic Web
• m:Semantic MediaWiki

• http://wm.sieheauch.de/?p=48 papers
• Wiki Research Bibliography
• Michael Strube
• "Web corpus mining by instance of Wikipedia"
• Video mining