Hugging Face's logo Hugging Face
  • Models
  • Datasets
  • Spaces
  • Buckets
  • Docs
  • Enterprise
  • Pricing
    • Website
      • Tasks
      • HuggingChat
      • Collections
      • Languages
      • Organizations
    • Community
      • Blog
      • Posts
      • Daily Papers
      • Hardware
      • Learn
      • Discord
      • Forum
      • GitHub
    • Solutions
      • Team & Enterprise
      • Hugging Face PRO
      • Enterprise Support
      • Inference Providers
      • Inference Endpoints
      • Storage Buckets

  • Log In
  • Sign Up
trl-lib 's Collections
Papers implemented in TRL
SFT datasets
Prompt datasets (for GRPO, RLOO, ...)
Preference datasets (for DPO, reward modeling)
Examples
Unpaired preference datasets (KTO)
Models from the docs
Tools
StackLLaMA
Comparing DPO with IPO and KTO

Unpaired preference datasets (KTO)

updated 6 days ago

Single completions labeled good or bad, for KTOTrainer. Formats: https://huggingface.co/docs/trl/dataset_formats

Upvote
1

  • trl-lib/kto-mix-14k

    Viewer • Updated Mar 25, 2024 • 15k • 241 • 9

    Note Conversational


  • trl-lib/ultrafeedback-gpt-3.5-turbo-helpfulness

    Viewer • Updated Jan 8, 2025 • 16.6k • 150 • 5

    Note Conversational

Upvote
1
  • Collection guide
  • Browse collections
Company
TOS Privacy About Careers
Website
Models Datasets Spaces Pricing Docs