msTenTen: Corpus of the Malay Web

The Malay Web Corpus (msTenTen) is a Malay corpus made up of texts collected from the Internet. This Malay corpus includes varieties of the Malay language used in Brunei, Malaysia, and Singapore. The Indonesian language is available in the Indonesian Web corpus. The corpus belongs to the TenTen corpus family. Sketch Engine currently provides access to TenTen corpora in more than 40 languages. The corpora are built using technology specialized in collecting only linguistically valuable web content.

For detailed information about TenTen corpora, see Common TenTen corpora attributes.

The msTenTen Malay 2024 corpus consists of 805 million words. The texts were downloaded in June–July 2024, October–December 2023 and in June–August 2020. The sample texts of the biggest web domains which account for 42% of all corpus texts were checked manually and content with poor quality text and spam was removed.

Part-of-speech tagset and lemmatization

The msTenTen Malay corpus is part-of-speech tagged with the following Indonesian TreeTagger tagset indicating the part of speech and grammatical category. The corpus texts also contain 

Overview of Malay TenTen corpora

This is a list of Malay Web corpora available in Sketch Engine:

  • Malay Web corpus 2024 (msTenTen24) – 805 million words, genre annotation and topic classification, part-of-speech tagging and lemmatization
  • Malay Web corpus 2020 (msTenTen20) – 296 million words

Basic information about the msTenTen24 corpus

Frequency
Tokens 960+ million
Words 805+ million
Sentences 56+ million
Web pages 2+ million

Genre annotation and topic classification

A part of the Malay Web 2024 corpus contains genre annotation and topic classification. These can be displayed as corpus structures in Concordance or in the Text type Analysis tool. Genres refer to writing styles and are divided into four groups (blog, discussion, fiction, legal, news, reference/encyclopedia) whereas topic classification is inspired by categories used by https://curlie.org/ (formerly dmoz.org).

  • genres cover 23% of the corpus, i.e. 225 million tokens
  • topic classification covers 1.3% of the corpus, i.e. 13 million tokens

Please refer to our article on topic and genre classification for more information: https://www.sketchengine.eu/blog/topics-and-genres-in-corpora/

Hover over the chart to display a number of tokens of the particular topic.

Search the Malay corpus msTenTen

Sketch Engine offers a range of tools to work with this Malay corpus.

open in Sketch Engine
about Sketch Engine

Tools to work with the Malay corpus from the web

A complete set of Sketch Engine tools is available to work with this Malay corpus to generate:

  • word sketch – Malay collocations categorized by grammatical relations
  • thesaurus – synonyms and similar words for every word
  • keywords – terminology extraction of one-word units
  • word lists – lists of words organized by frequency
  • n-grams – frequency list of multi-word units
  • concordance – examples in context
  • text type analysis – statistics of metadata in the corpus

mstenten24_tt1 (2025-02-21)

  • bad content removal
  • part-of-speech tagging and lemmatization
  • partial topic and genre classification

mstenten20

  • initial size of 1+ billion tokens
  • manual checking of the samples from the biggest web domains, content with poor linguistic quality was removed
  • final size of 352 million words

TenTen corpora

SUCHOMEL, Vít. Better Web Corpora For Corpus Linguistics And NLP. 2020. Available also from: https://is.muni.cz/th/u4rmz/. Doctoral thesis. Masaryk University, Faculty of Informatics, Brno. Supervised by Pavel RYCHLÝ.

Jakubíček, M., Kilgarriff, A., Kovář, V., Rychlý, P., & Suchomel, V. (2013, July). The TenTen corpus family. In 7th International Corpus Linguistics Conference CL (pp. 125-127).

Suchomel, V., & Pomikálek, J. (2012). Efficient web crawling for large text corpora. In Proceedings of the seventh Web as Corpus Workshop (WAC7) (pp. 39-43).

Other Malay corpora

A list of Malay corpora available in Sketch Engine.

available Malay corpora

Use Sketch Engine in minutes

Generate collocations, frequency lists, examples in contexts, n-grams or extract terms. Use our Quick Start Guide to learn it in minutes.

Quick Start Guide