ukTenTen: Ukrainian corpus from the Web

The Ukrainian Web Corpus (ukTenTen) is a Ukrainian corpus made up of texts collected from the Internet. The corpus belongs to the TenTen corpus family which is a set of web corpora built using the same method with a target size 10+ billion words. Sketch Engine currently provides access to TenTen corpora in more than 40 languages.

Data for the Ukrainian Web 2020 corpus consists of texts from May 2014, July–August 2020, and October–December 2023. The Wikipedia part is from December 2020 and 2022. The final size of the corpus contains 7.5+ billion words.

Detailed information about TenTen corpora is on the separate page Common TenTen corpora attributes.

Basic information about the Ukrainian Web 2022 corpus

Frequency
Tokens 9,550,046,777
Words 7,594,784,148
Sentences 556,723,376
Web pages 29,361,538

Part-of-speech tagset and lemmatization

The Ukrainian Web 2020 corpus is lemmatized by CSTLemma and part-of-speech tagged by RFTagger using two different tagsets:

  1. MULTEXT-East Ukrainian PoS tagset (which is more detailed),
  2. Universal Dependencies PoS tagset (showing only elementary parts of speech tags)
Lemmatizer was trained on the UK-Brown dictionary available on GitHub (that was kindly provided by authors for use in Sketch Engine).
Tagger was trained on the Universal Dependencies data with a support dictionary harvested from the UK-Brown dictionary.

Overview of Ukrainian TenTen corpora

These web corpora were crawled and processed repeatedly during the years:

  • Ukrainian Web corpus 2022 (ukTenTen22) – 7.5 billion words, genre & topic classification
  • Ukrainian Web corpus 2020 (ukTenTen20) – 2.5 billion words, part-of-speech tagging and lemmatization
  • Ukrainian Web corpus 2014 (ukTenTen14) – 2 billion word

Tools to work with the Ukrainian corpus

A complete set of Sketch Engine tools is available to work with this Ukrainian Web corpus to generate:

  • word sketch – Ukrainian collocations categorized by grammatical relations
  • thesaurus – synonyms and similar words for every word
  • keywords – terminology extraction of one-word and multi-word units
  • word lists – lists of Ukrainian words organized by frequency
  • n-grams – frequency list of multi-word units
  • concordance – examples in context
  • text type analysis – statistics of metadata in the corpus

ukTenTen22

version 1.0 (September 2022)

  • 9.55 billion tokens
  • genre & topic classification
  • fix incorrect sentence boundaries in data from 2014
  • tagged (RFTagger) and lemmatized (CSTLemma) by Ukrainian RFTagger pipeline version 2 with improved post-processing
  • multi-word term extraction available

ukTenTen20

version 1.0 (April 2022)

  • the initial version 3.68 billion tokens downloaded from the Web by SpiderlLing in 2020 and the Wikipedia part is from 2020 processed by Wiki2corpus tool
  • cleaned (semi-manual process), deduplicated
  • tagged (RFTagger) and lemmatized (CSTLemma) by Ukrainian RFTagger pipeline version 1

ukTenTen14

version 1.0 (spring 2014)

  • the initial version, 2.73 billion tokens obtained from the web in April 2014
  • no tagging and lemmatization available

TenTen corpora

SUCHOMEL, Vít. Better Web Corpora For Corpus Linguistics And NLP. 2020. Available also from: https://is.muni.cz/th/u4rmz/. Doctoral thesis. Masaryk University, Faculty of Informatics, Brno. Supervised by Pavel RYCHLÝ.

Jakubíček, M., Kilgarriff, A., Kovář, V., Rychlý, P., & Suchomel, V. (2013, July). The TenTen corpus family. In 7th International Corpus Linguistics Conference CL (pp. 125-127).

Suchomel, V., & Pomikálek, J. (2012). Efficient web crawling for large text corpora. In Proceedings of the seventh Web as Corpus Workshop (WAC7) (pp. 39-43).

Search the Ukrainian corpus

Sketch Engine offers a range of tools to work with this Ukrainian corpus from the web.

open in Sketch Engine
about Sketch Engine

Other text corpora

Sketch Engine offers 800+ language corpora.

available corpora

Use Sketch Engine in minutes

Generate collocations, frequency lists, examples in contexts, n-grams or extract terms with Sketch Engine. Use our Quick Start Guide to learn it in minutes.

Quick Start Guide