deTenTen: Corpus of the German Web

The German Web Corpus (deTenTen) is a corpus made up of texts collected from the internet. The corpus belongs to the TenTen corpus family which is a set of the web corpora built using the same method with a target size 10+ billion words. Sketch Engine currently provides access to TenTen corpora in more than 40 languages.

The newest German Web 2020 corpus consists of 17.5 billion words of texts collected from the web in June and July 2020. The Wikipedia part is from September to November 2020. The corpus contains lemmatization including gender lemmas and part-of-speech tagging. The sample texts of the biggest web domains which account for 50% of all corpus texts were checked semi-manually and content with poor quality text and spam was removed.

Detailed information about TenTen corpora is on the separate page Common TenTen corpora attributes.

Part-of-speech tagset

This German corpus was tagged by RFTagger using this POS tagset.

Genre annotation and topic classification

A part of the German Web 2020 corpus contains genre annotation and topic classification. These can be displayed as corpus structures in Concordance or in the Text type Analysis tool. Genres refer to writing styles and are divided into three groups (blog, discussion, legal, news) whereas topic classification is inspired by categories used by https://curlie.org/ (formerly dmoz.org) and includes the following topics: arts, business, games, health, home, recreation, reference, science, sport, society, and technology.

  • genres cover 18.9% of the corpus, i.e. 3.97 billion tokens
  • topic classification covers 22.9% of the corpus, i.e. 4.8 billion tokens

Hover over the chart to display a number of tokens of the particular topic.

Overview of German TenTen corpora

This is a list of German Web corpora available in Sketch Engine:

  • German Web corpus 2020 (deTenTen18) – 17.5 billion words, genre annotation and topic classification
  • German Web corpus 2018 (deTenTen18) – 5.3 billion words
  • German Web corpus 2013 (deTenTen13) – 16.5 billion words
  • German Web corpus 2010 (deTenTen10) – 2.3 billion words

Basic frequency statistics of the German Web 2020 corpus

Tokens20,999,598,683
Words17,512,733,172
Sentences1,145,230,688
Web pages47,255,278

Tool work with the German corpus from the web

A complete set of Sketch Engine tools is available to work with this German corpus to generate:

  • word sketch – German collocations categorized by grammatical relations
  • thesaurus – synonyms and similar words for every word
  • keywordsterminology extraction of one-word and multi-word units
  • word lists – lists of German nouns, verbs, adjectives etc. organized by frequency
  • n-grams – frequency list of multi-word units
  • concordance – examples in context
  • text type analysis – statistics of metadata in the corpus

German Web corpus (deTenTen20)

  • August 2022 – 17.5 billion words
  • genre and topic classification
  • new term grammar 2.0

German Web corpus (deTenTen13)

  • data crawled in December 2018 and January 2019 – 6.3 billion tokens

German Web corpus (deTenTen13)

  • data crawled in 2013 – 16.5 billion tokens

German Web corpus (deTenTen10)

  • version 2.0 (28 April 2011)
    • fixed problems with part-of-speech tagging which caused a major data loss in the previous version
    • 2.8 billion tokens
  • version 1.0 (30 November 2010)
    • initial version – 1.2 billion tokens

TenTen corpora

SUCHOMEL, Vít. Better Web Corpora For Corpus Linguistics And NLP. 2020. Available also from: https://is.muni.cz/th/u4rmz/. Doctoral thesis. Masaryk University, Faculty of Informatics, Brno. Supervised by Pavel RYCHLÝ.

Jakubíček, M., Kilgarriff, A., Kovář, V., Rychlý, P., & Suchomel, V. (2013, July). The TenTen corpus family. In 7th International Corpus Linguistics Conference CL (pp. 125-127).

Suchomel, V., & Pomikálek, J. (2012). Efficient web crawling for large text corpora. In Proceedings of the seventh Web as Corpus Workshop (WAC7) (pp. 39-43).

Genre annotation

SUCHOMEL, Vít. Genre Annotation of Web Corpora: Scheme and Issues. In Kohei Arai, Supriya Kapoor, Rahul Bhatia. Proceedings of the Future Technologies Conference (FTC) 2020, Volume 1. Vancouver, Canada: Springer Nature Switzerland AG, 2021. s. 738-754. ISBN 978-3-030-63127-7. doi:10.1007/978-3-030-63128-4_55.

Search the German corpus

Sketch Engine offers a range of tools to work with this German corpus from the web.

Other text corpora

Sketch Engine offers dozens of German language corpora.

Use Sketch Engine in minutes

Generate collocations, frequency lists, examples in contexts, n-grams or extract terms. Use our Quick Start Guide to learn it in minutes.