trTenTen: Corpus of the Turkish Web
The Turkish Web Corpus (trTenTen) is a Turkish corpus made up of texts collected from the Internet. The corpus belongs to the TenTen corpus family which is a set of web corpora built using the same method with a target size 10+ billion words. Sketch Engine currently provides access to TenTen corpora in more than 40 languages.
The most recent version of the trTenTen corpus consists of 4.9 billion words. The texts were downloaded in 2020. The sample texts of the biggest web domains which account for 58% of all corpus texts were checked semi-manually and content with poor quality text and spam was removed.
Detailed information about TenTen corpora is on the separate page Common TenTen corpora attributes.
Part-of-speech tagset and stemming
The Turkish Web corpora are part-of-speech tagged with the following Turkish TRMorph Penn Treebank tagset summary indicating the part of speech and grammatical category. The corpus texts also contain stemming when each word form from the corpus is assigned to its word stem. In this case, stems replace lemma attributes.
Genre annotation and topic classification
A part of the English Web 2020 corpus contains genre annotation and topic classification. These can be displayed as corpus structures in Concordance or in the Text type Analysis tool. Genres refer to writing styles and are divided into four groups (blog, discussion, legal, news) whereas topic classification is inspired by categories used by https://curlie.org/ (formerly dmoz.org) and includes the following topics: arts, business, games, health, home, recreation, reference, science, sport, society, and technology.
- genres cover 26.7% of the corpus, i.e. 1.6 billion tokens
- topic classification covers 6.3% of the corpus, i.e. 385 million tokens
Hover over the chart to display a number of tokens of the particular topic.
Basic information about Turkish Web 2020 (trTenTen20)
These web corpora were crawled and processed repeatedly during the years:
| Frequency | |
| Tokens | 6,145,927,191 |
| Words | 4,980,168,485 |
| Sentences | 368,345,301 |
| Web pages | 20,795,994 |
Overview of Turkish TenTen corpora
These web corpora were crawled and processed repeatedly during the years:
- Turkish Web corpus 2020 (trTenTen20) – 4.9 billion words, genre annotation and topic classification
- Turkish Web corpus 2012 (trTenTen12) – 3.3 billion words
Tools to work with these Turkish corpora
A complete set of Sketch Engine tools is available to work with these Turkish corpora from the web to generate:
- word sketch – Turkish collocations categorized by grammatical relations
- thesaurus – synonyms and similar words for every word
- keywords – terminology extraction of one-word units
- word lists – lists of Turkish nouns, verbs, adjectives etc. organized by frequency
- n-grams – frequency list of multi-word units
- concordance – examples in context
- text type analysis – statistics of metadata in the corpus
Changelog
Turkish Web 2020 (trTenTen20)
initial version (March 2023)
- TrMoprh tools – part-of-speech tagging and stemming
- genre and topic classification
Turkish Web 2012 (trTenTen12)
universal word sketches (February 2018)
- created universal word sketches
tagged version (April 2017)
- part-of-speech tagging
initial version (March 2012)
- initial version 3.3 billion words
- no tagging, no sketches
Bibliography
Vít Baisa and Vít Suchomel (2012). Large Corpora for Turkic Languages and Unsupervised Morphological Analysis. In Proceedings of the Eight International Conference on Language Resources and Evaluation (LREC’12), Turkey, May 2012, pp. 28–32.
TRMorph tool (part-of-speech tagging, stemming)
(2014) A Set of Open Source Tools for Turkish Natural Language Processing In: Proceedings of the Ninth International Conference on Language Resources and Evaluation (LREC’14) Ed. by N. Calzolari et al. 1079–1086 [pdf] [bib]
(2010) A Freely Available Morphological Analyzer for Turkish In: Proceedings of the 7th International Conference on Language Resources and Evaluation (LREC 2010) 820–827
TenTen corpora
Jakubíček, M., Kilgarriff, A., Kovář, V., Rychlý, P., & Suchomel, V. (2013, July). The TenTen corpus family. In 7th International Corpus Linguistics Conference CL (pp. 125-127).
Suchomel, V., & Pomikálek, J. (2012). Efficient web crawling for large text corpora. In Proceedings of the seventh Web as Corpus Workshop (WAC7) (pp. 39-43).
Word Sketches
Ambati, B. R., Reddy, S., & Kilgarriff, A. (2012). Word Sketches for Turkish. In LREC (pp. 2945-2950).
Search the Turkish corpus
Sketch Engine offers a range of tools to work with this Turkish corpus.
Use Sketch Engine in minutes
Generate collocations, frequency lists, examples in contexts, n-grams or extract terms. Use our Quick Start Guide to learn it in minutes.
