zhTenTen: Corpus of the Chinese Web
The Chinese Web Corpus (zhTenTen) is a Chinese corpus made up of texts collected from the Internet. The corpus belongs to the TenTen corpus family which is a set of the web corpora built using the same method with a target size 10+ billion words. Sketch Engine currently provides access to TenTen corpora in more than 30 languages. The corpora are built using technology specialized in collecting only linguistically valuable web content.
Data of the last version of the Chinese web corpus was crawled by the SpiderLing web spider in August and November 2017 and comprised of more than 15.9 billion words. The corpus was processed with Stanford NLP Core Tools. The corpus was divided into 2 language variants:
- Chinese Simplified corpus with Simplified Chinese characters
- Chinese Traditional corpus with Traditional Chinese characters
Detailed information about TenTen corpora is on the separate page Common TenTen corpora attributes.
Part-of-speech tagset
The zhTenTen corpus was POS annotated by the Stanford Log-linear Part-Of-Speech Tagger using Chinese Penn TreeBank, see the tagset legend.
Overview of Chinese TenTen corpora
A list of Chinese TenTen corpora available in Sketch Engine:
- Chinese Web corpus 2017 Simplified (zhTenTen17) – 13.5 billion words
- Chinese Web corpus 2017 Traditional (zhTenTen17) – 2.4 billion words
- Chinese Web corpus 2011 (zhTenTen11) – 1.7 billion words
zhTenTen 2017 corpus in detail
Basic information (Chinese Web 2017 Simplified)
| Frequency* | |
| Tokens | 16,593,146,196 |
| Words | 13,531,331,169 |
| Sentences | 667,201,792 |
| Web pages | 40,233,300 |
Basic information (Chinese Web 2017 Traditional)
| Frequency* | |
| Tokens | 2,977,351,219 |
| Words | 2,400,405,372 |
| Sentences | 130,120,839 |
| Web pages | 7,538,026 |
Tools to work with the Chinese corpus
A complete set of tools is available to work with this Chinese corpus from the web to generate:
- word sketch – Chinese collocations categorized by grammatical relations
- thesaurus – synonyms and similar words for every word
- keywords – terminology extraction of one-word and multi-word units
- word lists – lists of Chinese nouns, verbs, adjectives etc. organized by frequency
- n-grams – frequency list of multi-word units
- concordance – examples in context
- text type analysis – statistics of metadata in the corpus
Changelog
December 2018 – January 2019
- new data downloaded by SpiderLing in Aug & Nov 2017
- size 19.6 billion tokens
- classifcation of language variants (Simplified and Traditional)
- foreign languages filtered
February 2017
- new word sketch grammar prepared by Ondřej Svoboda
October 2016
- created preloaded subcorpora
2nd December 2011
- initial version – 1.7 billion words
Bibliography
TenTen corpora
Jakubíček, M., Kilgarriff, A., Kovář, V., Rychlý, P., & Suchomel, V. (2013, July). The TenTen corpus family. In 7th International Corpus Linguistics Conference CL (pp. 125-127).
Suchomel, V., & Pomikálek, J. (2012). Efficient web crawling for large text corpora. In Proceedings of the seventh Web as Corpus Workshop (WAC7) (pp. 39-43).
Other relating bibliographic references
Kilgarriff, A., Keng, N., & Smith, S. (2015). Learning Chinese with the Sketch Engine. In Corpus Linguistics in Chinese Contexts (pp. 63-73). Palgrave Macmillan UK.
Shih, M. H., & Hsieh, S. K. (2014). Sketching the Dependency Relations of Words in Chinese. ROCLING XXVI (2014), 139.
Search the Chinese corpus
Sketch Engine offers a range of tools to work with this Chinese corpus.
Use Sketch Engine in minutes
Generate collocations, frequency lists, examples in contexts, n-grams or extract terms. Use our Quick Start Guide to learn it in minutes.
