ltTenTen: Corpus of the Lithuanian Web
The Lithuanian Web Corpus (ltTenTen) is a Lithuanian corpus made up of texts collected from the Internet. The corpus belongs to the TenTen corpus family which is a set of the web corpora built using the same method with a target size 10+ billion words. Sketch Engine currently provides access to TenTen corpora in more than 40 languages.
The whole Lithuanian corpus 2021 is comprised of 1.7 billion words. The corpus data was crawled by the SpiderLing web spider from August to October 2021. The Wikipedia part is from December 2020 and August–September 2021. The sample texts of the biggest web domains which account for 64% of all corpus texts were checked semi-manually and content with poor quality text and spam was removed.
Detailed information about TenTen corpora is on the separate page Common TenTen corpora attributes.
Part-of-speech tagset
The Lithuanian Web corpora are annotated using the following part-of-speech tagset indicating the part of speech and grammatical category. The corpus texts also contain lemmatization when each word form from the corpus is assigned to its base form (lemma).
Overview of Lithuanian TenTen corpora
This is a list of Lithuanian Web corpora available in Sketch Engine:
- Lithuanian Web corpus 2021 (ltTenTen21) including PoS tagging and lemmatization – 1.7 billion words, genre annotation and topic classification
- Lithuanian Web corpus 2014 (ltTenTen14) – 770 million words
Basic frequency statistics of the Lithuanian Web 2021 corpus
| Number of tokens | 2.2+ billion |
| Number of words | 1.7+ billion |
| Number of sentences | 135+ million |
| Number of web pages | 7+ million |
Genre annotation and topic classification
A part of the Lithuanian Web 2021 corpus contains genre annotation and topic classification. These can be displayed as corpus structures in Concordance or in the Text type Analysis tool. Genres refer to writing styles and are divided into four groups (blog, discussion, fiction, legal, news, reference/encyclopedia) whereas topic classification is inspired by categories used by https://curlie.org/ (formerly dmoz.org).
- genres cover 15.48% of the corpus, i.e. 353 million tokens
- topic classification covers 9.12% of the corpus, i.e. 208 million tokens
Please refer to our article on topic and genre classification for more information: https://www.sketchengine.eu/blog/topics-and-genres-in-corpora/
Hover over the chart to display a number of tokens of the particular topic.
Tools to work with the Lithuanian Web corpora
A complete set of Sketch Engine tools is available to work with the Lithuanian corpora to generate:
- word sketch – Lithuanian collocations categorized by grammatical relations
- thesaurus – synonyms and similar words for every word
- keywords – terminology extraction of one-word and multi-word units
- word lists – lists of Lithuanian words organized by frequency
- n-grams – frequency list of multi-word units
- concordance – examples in context
- text type analysis – statistics of metadata in the corpus
Changelog
lttenten21 – version 1 (2024-12-12)
- topic and genre classification
- POS tagging
- word sketch and term grammar
- spam removal
version 0 (April 2014)
- crawled by SpiderLing in April 2014
- 0.982 billion tokens
Bibliography
TenTen corpora
SUCHOMEL, Vít. Better Web Corpora For Corpus Linguistics And NLP. 2020. Available also from: https://is.muni.cz/th/u4rmz/. Doctoral thesis. Masaryk University, Faculty of Informatics, Brno. Supervised by Pavel RYCHLÝ.
Jakubíček, M., Kilgarriff, A., Kovář, V., Rychlý, P., & Suchomel, V. (2013, July). The TenTen corpus family. In 7th International Corpus Linguistics Conference CL (pp. 125-127).
Suchomel, V., & Pomikálek, J. (2012). Efficient web crawling for large text corpora. In Proceedings of the seventh Web as Corpus Workshop (WAC7) (pp. 39-43).
Genre annotation
SUCHOMEL, Vít. Genre Annotation of Web Corpora: Scheme and Issues. In Kohei Arai, Supriya Kapoor, Rahul Bhatia. Proceedings of the Future Technologies Conference (FTC) 2020, Volume 1. Vancouver, Canada: Springer Nature Switzerland AG, 2021. s. 738-754. ISBN 978-3-030-63127-7. doi:10.1007/978-3-030-63128-4_55.
Search the Lithuanian corpus
Sketch Engine offers a range of tools to work with this Lithuanian corpus.
Use Sketch Engine in minutes
Generate collocations, frequency lists, examples in contexts, n-grams or extract terms with Sketch Engine. Use our Quick Start Guide to learn it in minutes.
