AI-GPE Lab中文
Datasets[EssentialAI](https://www.essential.ai)

Essential-Web v1.0: 24T tokens of organized web data

Abstract

A 24-trillion-token dataset in which every document is annotated with a twelve-category taxonomy covering topic, format, content complexity, and quality.

Catalogue metadata

Record ID
AISLI-R-87B3CF7FA25E006C4C52
Type
Datasets
Source catalogue
registry.opendata.aws
Topic
科技与开放工具
Use conditions
Essential-Web-v1.0 contributions are made available under the [ODC attribution license](https://opendatacommons.org/licenses/by/odc_by_1.0_public_text.txt); however, users should also abide by the [Common Crawl - Terms of Use](https://commoncrawl.org/terms-of-use). We do not alter the license of any of the underlying data.