dvTenTen: Corpus of the Maldivian Web
The Maldivian Web Corpus (dvTenTen) is a Maldivian corpus made up of texts collected from the Internet. The corpus belongs to the TenTen corpus family. Sketch Engine currently provides access to TenTen corpora in more than 50 languages. The corpora are built using technology specialized in collecting only linguistically valuable web content. The Maldivian language is also known as Dhivehi or Divehi.
For detailed information about TenTen corpora, see Common TenTen corpora attributes.
The most recent version of the Maldivian corpus dvTenTen consists of 20 million words. The texts were downloaded in August 2022. The sample texts of the largest web domains which account for 88% of all corpus texts were checked manually and content with poor quality text and spam was removed.
Part-of-speech tagset and lemmatization
This Maldivian corpus is neither part-of-speech tagged nor lemmatized.
Maldivian Web 2022 corpus sizes
| Frequency | |
| Tokens | 24+ million |
| Words | 20+ million |
| Sentences | 2+ million |
| Web pages | 80+ thousand |
Search the Maldivian corpus dvTenTen
Sketch Engine offers a range of tools to work with this Maldivian corpus.
Genre annotation and topic classification
A part of the Maldivian Web 2022 corpus contains genre annotation and topic classification. These can be displayed as corpus structures in Concordance or in the Text type Analysis tool. Genres refer to writing styles and are divided into four groups (blog, discussion, fiction, legal, news, reference/encyclopedia) whereas topic classification is inspired by categories used by https://curlie.org/ (formerly dmoz.org) and includes the following topics: arts, beauty & fashion, cars & bikes, culture & entertainment, economy finance & business, games, health, history, hobbies, home family & children, nature & environment, pets & animal, politics & government, religion, science, sex, sports, technology & IT, and travel & tourism.
- genres cover 66% of the corpus, i.e. 16.3 million tokens
- topic classification covers 11.9% of the corpus, i.e. 3 million tokens
Please refer to our article on topic and genre classification for more information: https://www.sketchengine.eu/blog/topics-and-genres-in-corpora/
Hover over the chart to display a number of tokens of the particular topic.
Overview of Maldivian TenTen corpora
At present, there is only single TenTen Dhivehi corpus available in Sketch Engine:
- Maldivian Web corpus 2022 (dvTenTen22) – 20 million words, genre annotation and topic classification
Tools to work with the Maldivian corpus from the web
A partial set of Sketch Engine tools is available to work with this Maldivian corpus to generate:
- word sketch – Maldivian collocations categorized by grammatical relations
- thesaurus – synonyms and similar words for every word
- keywords – terminology extraction of one-word units
- word lists – lists of Maldivian words
- n-grams – frequency list of multi-word units
- concordance – examples in context
- text type analysis – statistics of metadata in the corpus
Changelog
Maldivian Web 2022 (dvTenTen22)
version v1_tok3 (January 2025)
- topic and genre classification
- bad content removal
- tokenization
- universal Sketch Grammar
Bibliography
TenTen corpora
SUCHOMEL, Vít. Better Web Corpora For Corpus Linguistics And NLP. 2020. Available also from: https://is.muni.cz/th/u4rmz/. Doctoral thesis. Masaryk University, Faculty of Informatics, Brno. Supervised by Pavel RYCHLÝ.
Jakubíček, M., Kilgarriff, A., Kovář, V., Rychlý, P., & Suchomel, V. (2013, July). The TenTen corpus family. In 7th International Corpus Linguistics Conference CL (pp. 125-127).
Suchomel, V., & Pomikálek, J. (2012). Efficient web crawling for large text corpora. In Proceedings of the seventh Web as Corpus Workshop (WAC7) (pp. 39-43).
Genre annotation
SUCHOMEL, Vít. Genre Annotation of Web Corpora: Scheme and Issues. In Kohei Arai, Supriya Kapoor, Rahul Bhatia. Proceedings of the Future Technologies Conference (FTC) 2020, Volume 1. Vancouver, Canada: Springer Nature Switzerland AG, 2021. s. 738-754. ISBN 978-3-030-63127-7. doi:10.1007/978-3-030-63128-4_55.
Use Sketch Engine in minutes
Generate collocations, frequency lists, examples in contexts, n-grams or extract terms. Use our Quick Start Guide to learn it in minutes.
