plTenTen: Corpus of the Polish Web
The Polish Web Corpus (plTenTen) is a Polish corpus made up of texts collected from the Internet. The corpus belongs to the TenTen corpus family which is a set of the web corpora built using the same method with a target size 10+ billion words. Sketch Engine currently provides access to TenTen corpora in more than 30 languages.
Data was crawled by the SpiderLing web spider in December 2019 and January 2020.
Detailed information about TenTen corpora is on the separate page Common TenTen corpora attributes.
Part-of-speech tagset
The plTenTen corpus was annotated by the RFTagger tool using the NKJP tagset, a POS tagset description used for the National Corpus of Polish (Narodowy Korpus Języka Polskiego).
Overview of Polish TenTen corpora
A list of Polish TenTen corpora available in Sketch Engine:
- Polish Web corpus 2019 (plTenTen19_rft2) – 4 billion words – better lemmatization, improved terminology extraction, bad content removal, topic and genre classification
- Polish Web corpus 2019 (plTenTen19_rft1) – 4.2 billion words
- Polish Web corpus 2012 (plTenTen12) – 7.1 billion words
Basic frequency statistics of the Polish Web 2019 corpus
| Tokens | 4,863,900,915 |
| Words | 3,994,024,317 |
| Sentences | 267,069,825 |
| Web pages | 12,386,517 |
Genre annotation and topic classification
A part of the Polish Web 2019 corpus contains genre annotation and topic classification. These can be displayed as corpus structures in Concordance or in the Text type Analysis tool. Genres refer to writing styles and are divided into four groups (blog, discussion, fiction, legal, news, reference/encyclopedia) whereas topic classification is inspired by categories used by https://curlie.org/ (formerly dmoz.org).
- genres cover 17.1% of the corpus, i.e. 830 million tokens
- topic classification covers 8.5% of the corpus, i.e. 400 million tokens
Please refer to our article on topic and genre classification for more information: https://www.sketchengine.eu/blog/topics-and-genres-in-corpora/
Hover over the chart to display a number of tokens of the particular topic.
Tools to work with the Polish corpus
A complete set of Sketch Engine tools is available to work with this Polish Web corpus to generate:
- word sketch – Polish collocations categorized by grammatical relations
- thesaurus – synonyms and similar words for every word
- keywords – terminology extraction of one-word and multi-word units
- word lists – lists of Polish nouns, verbs, adjectives etc. organized by frequency
- n-grams – frequency list of multi-word units
- concordance – examples in context
- text type analysis – statistics of metadata in the corpus
Changelog
pltenten19
version 2 (November 2024)
- new lemmatization
- improved term grammar
- topic and genre classification
- bad content removal
pltenten19
version 1 (September 2021)
-
- 5.2 billion tokens
- tagged by the RFTagger tool with the NKJP tagset
- samples from the biggest web domains were manually checked and content with poor linguistic quality was removed
pltenten12
version 3 (March 2017)
-
- corpus tagged by the RFTagger tool with the NKJP tagset
- created a lempos attribute
version 2 (1 July 2013)
-
- corpus tagged by the WCRFT tagger
version 1 (23 July 2012)
-
- initial version – 7.7 billion words, untagged
a sample for Cesar (25 October 2012)
-
- 640 million words sample
- tagged by WCRFT (source: Wayback Machine) with the NKJP tagset
Bibliography
TenTen corpora
Jakubíček, M., Kilgarriff, A., Kovář, V., Rychlý, P., & Suchomel, V. (2013, July). The TenTen corpus family. In 7th International Corpus Linguistics Conference CL (pp. 125-127).
Suchomel, V., & Pomikálek, J. (2012). Efficient web crawling for large text corpora. In Proceedings of the seventh Web as Corpus Workshop (WAC7) (pp. 39-43).
Word Sketches
Radziszewski, A., Kilgarriff, A., & Lew, R. (2011). Polish word sketches.
Genre annotation
SUCHOMEL, Vít. Genre Annotation of Web Corpora: Scheme and Issues. In Kohei Arai, Supriya Kapoor, Rahul Bhatia. Proceedings of the Future Technologies Conference (FTC) 2020, Volume 1. Vancouver, Canada: Springer Nature Switzerland AG, 2021. s. 738-754. ISBN 978-3-030-63127-7. doi:10.1007/978-3-030-63128-4_55.
Search the Polish corpus
Sketch Engine offers a range of tools to work with this Polish corpus.
Use Sketch Engine in minutes
Generating collocations, frequency lists, examples in contexts, n-grams or extracting terms is easy with Sketch Engine. Use our Quick Start Guide to learn it in minutes.
