HuggingFaceFW/fineweb
FineWeb is a dataset of over 15 trillion tokens of cleaned and deduplicated English web data from CommonCrawl. It is optimized for LLM performance and processed using the datatrove library. The dataset aims to provide high-quality data for training large language models and outperforms other commonly used web datasets.We’re on a journey to advance and democratize artificial intelligence through open source and open science.
Részletes Bevezetés
FineWeb is a large-scale dataset designed to provide high-quality web data for training large language models. It includes over 15 trillion tokens of cleaned and deduplicated English web data from CommonCrawl. The dataset is processed using the datatrove library and is optimized for LLM performance. It outperforms other commonly used web datasets in benchmark tasks.
TöbbAdatkészlet
SoulChat2.0: Psychological Counselor's Digital Twin Framework
SoulChat2.0 is a framework for constructing the digital twin of psychological counselors, designed to support the development of AI applications in mental health. It includes a data generation module and a modeling module, enabling the creation of personalized counseling models based on limited real-world counseling cases.
ISSP: International Social Science Survey Program
The ISSP is a cross-national collaboration program conducting annual surveys on diverse topics relevant to social sciences. Established in 1984, it includes members from various cultures around the globe. Over one million respondents have participated in ISSP surveys, and all collected data and documentation are available free of charge.
APA PsycInfo: Behavioral and Social Sciences Database
APA PsycInfo is the premier abstracting and indexing database covering the behavioral and social sciences. It provides over 5,000,000 peer-reviewed records, 144 million cited references, and spans 600 years of content. The database is updated twice-weekly and includes research in 30 languages from 50 countries.