idWaC: Indonesian web corpus
The Indonesian web corpus (idWaC) is an Indonesian corpus made up of texts collected from the Internet. The corpus was prepared according to standards described in the document A Corpus Factory for Many Languages (Kilgarriff et al. at LREC 2010).
Data was crawled by the Heritrix web spider in 2010 with the final size of 100 million words.
Part-of-speech tagset
The idWaC is POS tagged with the following part-of-speech tagset.
Tools to work with the Indonesian Web corpus
A complete set of Sketch Engine tools is available to work with this Indonesian Web corpus to generate:
- word sketch – Indonesian collocations categorized by grammatical relations
- thesaurus – synonyms and similar words for every word
- keywords – terminology extraction of one-word units
- word lists – lists of Indonesian nouns, verbs, adjectives etc. organized by frequency
- n-grams – frequency list of multi-word units
- concordance – examples in context
- text type analysis – statistics of metadata in the corpus
Changelog
v2.0 (17th Jan 2012)
- The corpus is tagged using a new POS tagger (90.73% accuracy), lemmatizer and morph analyser downloaded from http://sivareddy.in/downloads
- simple sketch grammar for Telugu and generated word sketches and distributional thesaurus for Telugu
Bibliography
Corpus factory method
Kilgarriff, A., Reddy, S., Pomikálek, J., & Avinesh, P. V. S. (2010, May). A corpus factory for many languages. In LREC.
Search the Indonesian Web corpus
Sketch Engine offers a range of tools to work with the IndonesianWaC.
Use Sketch Engine in minutes
Generating collocations, frequency lists, examples in contexts, n-grams or extracting terms is easy with Sketch Engine. Use our Quick Start Guide to learn it in minutes.
