ruTenTen: Corpus of the Russian Web
The Russian Web Corpus (ruTenTen) is a Russian corpus made up of texts collected from the Internet. The corpus belongs to the TenTen corpus family which is a set of the web corpora built using the same method with a target size 10+ billion words. Sketch Engine currently provides access to TenTen corpora in more than 30 languages. The corpora are built using technology specialized in collecting only linguistically valuable web content.
This Russian Web corpus is cleaned, deduplicated and cleaned of Ukranian and Belarusian texts. It is processed with the RFTagger and TreeTagger tool.
It contains POS tags, lemmas, lempos and word sketches enabling to explore the grammatical and collational behavior of the Russian language.
Detailed information about TenTen corpora is on the separate page Common TenTen corpora attributes.
Part-of-speech tagset
The summary of POS tags used in these Russian corpora is here.
rutenten corpus in detail
The chart shows the distribution of the parts of speech in the Russian Web corpus 2011.
Basic information
Frequency | |
Tokens | 18,280,486,876 |
Words | 14,553,856,113 |
Sentences | 1,016,579,568 |
Web pages | 36,946,344 |
Tools to work with the Russian Web Corpus
A complete set of Sketch Engine tools is available to work with the Russian corpus to generate:
- word sketch – Russian collocations categorized by grammatical relations
- thesaurus – synonyms and similar words for every word
- keywords – terminology extraction of one-word and multi-word units
- word lists – lists of Russian nouns, verbs, adjectives etc. organized by frequency
- n-grams – frequency list of multi-word units
- concordance – examples in context
- text type analysis – statistics of metadata in the corpus
Changelog
version 8 (November 2014)
- further cleaning of Ukrainian and Belarusian texts
- dynamic case, number, gender
- gender lemma attribute
version 7 (April-May 2014)
- reprocessed with the RFTagger and TreeTagger tool – 14.5 billion words
- removed documents containing Ukrainian characters [ІіЇїЄє] or Belarusian characters [Ўў],
- removed documents from sites yielding high relative frequency of word порно (porn)
- added lempos
version 1 (April 2012)
- initial version – 15.8 billion words
Bibliography
TenTen corpora
Jakubíček, M., Kilgarriff, A., Kovář, V., Rychlý, P., & Suchomel, V. (2013, July). The TenTen corpus family. In 7th International Corpus Linguistics Conference CL (pp. 125-127).
Suchomel, V., & Pomikálek, J. (2012). Efficient web crawling for large text corpora. In Proceedings of the seventh Web as Corpus Workshop (WAC7) (pp. 39-43).
Use Sketch Engine in minutes
Generate collocations, frequency lists, examples in contexts, n-grams or extract terms is easy. Use our Quick Start Guide to learn it in minutes.