---
language: en
license: mit
tags:
- bert
- text-classification
- sequence-classification
datasets:
- agentlans/cosmopedia-classification
metrics:
- f1
- loss
model-index:
- name: GIST-small-cosmopedia-v1
  results:
  - task:
      type: text-classification
      name: Text Classification
    metrics:
    - type: f1
      value: 0.6313
      name: Evaluation F1
    - type: loss
      value: 1.0927
      name: Evaluation Loss
---
# GIST-small-cosmopedia-v1

A fine-tuned version of the **bert** architecture (`BertForSequenceClassification`) optimized for the `text-classification` task.

- **Model type:** bert
- **Problem Type:** single_label_classification
- **Number of Labels:** 113
- **Vocabulary Size:** 30522
- **License:** MIT

## Use

To get started with this model in Python using the Hugging Face Transformers library, run the following code:

```python
from transformers import AutoTokenizer, AutoModelForSequenceClassification
import torch

model_id = "agentlans/GIST-small-cosmopedia-v1"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForSequenceClassification.from_pretrained(model_id)

text = "Replace this with your input text."
inputs = tokenizer(text, return_tensors="pt")

with torch.no_grad():
    logits = model(**inputs).logits

predicted_class_id = logits.argmax().item()
predicted_class_name = model.config.id2label[predicted_class_id]

print(f"Predicted Class ID: {predicted_class_id}")
print(f"Predicted Class Name: {predicted_class_name}")
```

## Intended Uses & Limitations

### Intended Use
This model is designed for sequence classification tasks. Below are the specific class labels mapped to their corresponding IDs:

| Label ID | Label Name |
|---|---|
| 0 | Addiction and Mental Illness |
| 1 | American Football |
| 2 | Arts and Crafts |
| 3 | Astrology |
| 4 | Astronomy and Astrophysics |
| 5 | Audio Equipment and Home Theater Systems |
| 6 | Automotive Parts and Accessories |
| 7 | Baseball |
| 8 | Biochemistry and Molecular Biology |
| 9 | Business and Entrepreneurship |
| 10 | Business and Management |
| 11 | Cannabis and CBD Products |
| 12 | Career Development and Job Opportunities |
| 13 | Christian Theology and Spirituality |
| 14 | Christianity and Theology |
| 15 | Cleaning and Maintenance |
| 16 | Computer Antivirus Software and Security |
| 17 | Computer Hardware and Graphics Cards |
| 18 | Computer Programming and Web Development |
| 19 | Computer Science |
| 20 | Computer Security & Privacy |
| 21 | Cooking and Baking |
| 22 | Cooking and Recipes |
| 23 | Cosmetic Surgery and Body Modifications |
| 24 | Cricket |
| 25 | Cryptocurrency and Blockchain Technology |
| 26 | Culinary Arts and Beverages |
| 27 | Data Privacy and Protection |
| 28 | Dentistry |
| 29 | Digital Imaging and Photography |
| 30 | Digital Marketing and Business |
| 31 | Economics and Finance |
| 32 | Education |
| 33 | Education and Youth Development |
| 34 | Electric Vehicles and Battery Technology |
| 35 | Energy and Environmental Policy |
| 36 | Energy and Natural Resources |
| 37 | Entomology and Apiculture |
| 38 | Events and Community Happenings |
| 39 | Fashion & Apparel |
| 40 | Fiction and Fantasy Writing |
| 41 | Finance and Investment |
| 42 | Fire Incidents |
| 43 | Football/Soccer |
| 44 | Genetics and Mental Health |
| 45 | Geography and Weather |
| 46 | Gun Control and Violence |
| 47 | HIV Treatment and Care |
| 48 | Hair Care |
| 49 | Hair Care and Styling |
| 50 | Health and Lifestyle |
| 51 | Healthcare & Medical Services |
| 52 | Healthcare and Operations Management |
| 53 | Home Improvement and Maintenance |
| 54 | Human Resources / Organizational Management |
| 55 | Human Resources and Education |
| 56 | Ice Hockey |
| 57 | Infant Feeding and Child Development |
| 58 | Insurance |
| 59 | International Relations and Conflict |
| 60 | International Relations and Current Events |
| 61 | International Relations and Politics |
| 62 | Jewelry Design and Manufacturing |
| 63 | Leadership and Education |
| 64 | Legal Services and Issues |
| 65 | Legal Studies / Law |
| 66 | Legal Studies and Public Policy |
| 67 | Lighting Design and Technology |
| 68 | Literature and Creative Writing |
| 69 | Loans and Mortgages |
| 70 | Marketing and Business Strategies |
| 71 | Medicine |
| 72 | Mental Health Counseling |
| 73 | Mental Health and Therapy |
| 74 | Molecular Biology and Genetics |
| 75 | Moving Services and Logistics |
| 76 | Music |
| 77 | Nutrition and Health |
| 78 | Online Chat Platforms and Data Privacy |
| 79 | Online Dating & Relationships |
| 80 | Online Platforms & Web Technologies |
| 81 | Performing Arts |
| 82 | Personal Development and Empowerment |
| 83 | Personal Finance and Investments |
| 84 | Pets and Pet Care |
| 85 | Pharmaceutical manufacturing and technology |
| 86 | Physical Fitness and Health |
| 87 | Political Science |
| 88 | Politics and Government |
| 89 | Product Marketing and Design |
| 90 | Professional Basketball/NBA |
| 91 | Professional Wrestling and Sports Entertainment |
| 92 | Psychology |
| 93 | Public Administration and Policy |
| 94 | Public Safety and Emergency Response |
| 95 | Public Transit and Transportation |
| 96 | Real Estate & Investment |
| 97 | Recreational Fishing |
| 98 | Skincare and Beauty Products |
| 99 | Sports and Education |
| 100 | Taxation and Finance |
| 101 | Technology and Computer Science |
| 102 | Technology and Consumer Electronics |
| 103 | Tennis |
| 104 | Transportation and City Planning |
| 105 | Travel |
| 106 | Video Games |
| 107 | Visual Arts and Art Appreciation |
| 108 | Waste Management and Recycling |
| 109 | Watchmaking and Horology |
| 110 | Weddings |
| 111 | Wine & Winemaking |
| 112 | Writing and Storytelling |

## Training Details

### Hyperparameters
The following hyperparameters were used during fine-tuning:
- **Learning Rate:** 5e-05
- **Train Batch Size:** 8
- **Eval Batch Size:** 8
- **Optimizer:** OptimizerNames.ADAMW_TORCH_FUSED
- **Number of Epochs:** 3.0
- **Mixed Precision:** BF16

<details>
<summary><b>Show Advanced Training Configuration</b></summary>

#### Optimization & Regularization
- **Gradient Accumulation Steps:** 1
- **Learning Rate Scheduler:** SchedulerType.LINEAR
- **Warmup Steps:** 0
- **Warmup Ratio:** None
- **Weight Decay:** 0.0
- **Max Gradient Norm:** 1.0

#### Hardware & Reproducibility
- **Number of GPUs:** 1
- **Seed:** 42

</details>

## Training Results & Evaluation

During fine-tuning, the model achieved the following results on the evaluation set:

| Metric | Value |
|---|---|
| **Train Loss** | 1.0233 |
| **Validation Loss** | 1.0927 |
| **Validation F1 Score** | 0.6313 |
| **Total FLOPs** | 9.8351e+15 |

For performance on the test set, [click here](report.md).

### Speed Performance
- **Training Runtime:** 1304.8729 seconds
- **Train Samples per Second:** 456.771
- **Evaluation Runtime:** 26.6644 seconds
- **Eval Samples per Second:** 1862.781


<details>
<summary><b>Show Detailed Training Logs</b></summary>

### Training Logs History

| Step | Epoch | Learning Rate | Training Loss | Validation Loss | Validation F1 |
|---|---|---|---|---|---|
| 500 | 0.02 | 4.9665e-05 | 3.8688 | N/A | N/A |
| 1000 | 0.04 | 4.9330e-05 | 2.8619 | N/A | N/A |
| 1500 | 0.06 | 4.8994e-05 | 2.4201 | N/A | N/A |
| 2000 | 0.081 | 4.8658e-05 | 2.161 | N/A | N/A |
| 2500 | 0.101 | 4.8323e-05 | 1.9853 | N/A | N/A |
| 3000 | 0.121 | 4.7987e-05 | 1.8134 | N/A | N/A |
| 3500 | 0.141 | 4.7652e-05 | 1.708 | N/A | N/A |
| 4000 | 0.161 | 4.7316e-05 | 1.6879 | N/A | N/A |
| 4500 | 0.181 | 4.6981e-05 | 1.648 | N/A | N/A |
| 5000 | 0.201 | 4.6645e-05 | 1.6214 | N/A | N/A |
| 5500 | 0.221 | 4.6310e-05 | 1.5573 | N/A | N/A |
| 6000 | 0.242 | 4.5974e-05 | 1.4505 | N/A | N/A |
| 6500 | 0.262 | 4.5639e-05 | 1.4213 | N/A | N/A |
| 7000 | 0.282 | 4.5303e-05 | 1.4116 | N/A | N/A |
| 7500 | 0.302 | 4.4967e-05 | 1.4007 | N/A | N/A |
| 8000 | 0.322 | 4.4632e-05 | 1.361 | N/A | N/A |
| 8500 | 0.342 | 4.4296e-05 | 1.3746 | N/A | N/A |
| 9000 | 0.362 | 4.3961e-05 | 1.2952 | N/A | N/A |
| 9500 | 0.383 | 4.3625e-05 | 1.2967 | N/A | N/A |
| 10000 | 0.403 | 4.3290e-05 | 1.3186 | N/A | N/A |
| 10500 | 0.423 | 4.2954e-05 | 1.316 | N/A | N/A |
| 11000 | 0.443 | 4.2619e-05 | 1.2962 | N/A | N/A |
| 11500 | 0.463 | 4.2283e-05 | 1.2782 | N/A | N/A |
| 12000 | 0.483 | 4.1948e-05 | 1.2217 | N/A | N/A |
| 12500 | 0.503 | 4.1612e-05 | 1.2416 | N/A | N/A |
| 13000 | 0.523 | 4.1276e-05 | 1.2418 | N/A | N/A |
| 13500 | 0.544 | 4.0941e-05 | 1.2475 | N/A | N/A |
| 14000 | 0.564 | 4.0605e-05 | 1.2107 | N/A | N/A |
| 14500 | 0.584 | 4.0270e-05 | 1.2335 | N/A | N/A |
| 15000 | 0.604 | 3.9934e-05 | 1.2252 | N/A | N/A |
| 15500 | 0.624 | 3.9599e-05 | 1.2382 | N/A | N/A |
| 16000 | 0.644 | 3.9263e-05 | 1.1816 | N/A | N/A |
| 16500 | 0.664 | 3.8928e-05 | 1.2079 | N/A | N/A |
| 17000 | 0.685 | 3.8592e-05 | 1.2031 | N/A | N/A |
| 17500 | 0.705 | 3.8256e-05 | 1.2082 | N/A | N/A |
| 18000 | 0.725 | 3.7921e-05 | 1.2155 | N/A | N/A |
| 18500 | 0.745 | 3.7585e-05 | 1.1903 | N/A | N/A |
| 19000 | 0.765 | 3.7250e-05 | 1.1614 | N/A | N/A |
| 19500 | 0.785 | 3.6914e-05 | 1.1556 | N/A | N/A |
| 20000 | 0.805 | 3.6579e-05 | 1.1657 | N/A | N/A |
| 20500 | 0.825 | 3.6243e-05 | 1.1904 | N/A | N/A |
| 21000 | 0.846 | 3.5908e-05 | 1.1701 | N/A | N/A |
| 21500 | 0.866 | 3.5572e-05 | 1.1741 | N/A | N/A |
| 22000 | 0.886 | 3.5237e-05 | 1.14 | N/A | N/A |
| 22500 | 0.906 | 3.4901e-05 | 1.1826 | N/A | N/A |
| 23000 | 0.926 | 3.4565e-05 | 1.1334 | N/A | N/A |
| 23500 | 0.946 | 3.4230e-05 | 1.1562 | N/A | N/A |
| 24000 | 0.966 | 3.3894e-05 | 1.0916 | N/A | N/A |
| 24500 | 0.987 | 3.3559e-05 | 1.1356 | N/A | N/A |
| 24835 | 1.0 | N/A | N/A | 1.1201 | 0.5718 |
| 25000 | 1.007 | 3.3223e-05 | 1.0691 | N/A | N/A |
| 25500 | 1.027 | 3.2888e-05 | 0.949 | N/A | N/A |
| 26000 | 1.047 | 3.2552e-05 | 0.9389 | N/A | N/A |
| 26500 | 1.067 | 3.2217e-05 | 0.9352 | N/A | N/A |
| 27000 | 1.087 | 3.1881e-05 | 0.9251 | N/A | N/A |
| 27500 | 1.107 | 3.1546e-05 | 0.9181 | N/A | N/A |
| 28000 | 1.127 | 3.1210e-05 | 0.9674 | N/A | N/A |
| 28500 | 1.148 | 3.0874e-05 | 0.9783 | N/A | N/A |
| 29000 | 1.168 | 3.0539e-05 | 0.9192 | N/A | N/A |
| 29500 | 1.188 | 3.0203e-05 | 0.9751 | N/A | N/A |
| 30000 | 1.208 | 2.9868e-05 | 0.9555 | N/A | N/A |
| 30500 | 1.228 | 2.9532e-05 | 0.917 | N/A | N/A |
| 31000 | 1.248 | 2.9197e-05 | 0.9963 | N/A | N/A |
| 31500 | 1.268 | 2.8861e-05 | 0.9668 | N/A | N/A |
| 32000 | 1.289 | 2.8526e-05 | 0.9155 | N/A | N/A |
| 32500 | 1.309 | 2.8190e-05 | 0.8989 | N/A | N/A |
| 33000 | 1.329 | 2.7855e-05 | 0.9209 | N/A | N/A |
| 33500 | 1.349 | 2.7519e-05 | 0.9203 | N/A | N/A |
| 34000 | 1.369 | 2.7183e-05 | 0.9156 | N/A | N/A |
| 34500 | 1.389 | 2.6848e-05 | 0.9333 | N/A | N/A |
| 35000 | 1.409 | 2.6512e-05 | 0.904 | N/A | N/A |
| 35500 | 1.429 | 2.6177e-05 | 0.9448 | N/A | N/A |
| 36000 | 1.45 | 2.5841e-05 | 0.9495 | N/A | N/A |
| 36500 | 1.47 | 2.5506e-05 | 0.9248 | N/A | N/A |
| 37000 | 1.49 | 2.5170e-05 | 0.9464 | N/A | N/A |
| 37500 | 1.51 | 2.4835e-05 | 0.9223 | N/A | N/A |
| 38000 | 1.53 | 2.4499e-05 | 0.9354 | N/A | N/A |
| 38500 | 1.55 | 2.4163e-05 | 0.9917 | N/A | N/A |
| 39000 | 1.57 | 2.3828e-05 | 0.921 | N/A | N/A |
| 39500 | 1.59 | 2.3492e-05 | 0.9341 | N/A | N/A |
| 40000 | 1.611 | 2.3157e-05 | 0.9443 | N/A | N/A |
| 40500 | 1.631 | 2.2821e-05 | 0.9163 | N/A | N/A |
| 41000 | 1.651 | 2.2486e-05 | 0.9375 | N/A | N/A |
| 41500 | 1.671 | 2.2150e-05 | 0.9126 | N/A | N/A |
| 42000 | 1.691 | 2.1815e-05 | 0.9315 | N/A | N/A |
| 42500 | 1.711 | 2.1479e-05 | 0.9323 | N/A | N/A |
| 43000 | 1.731 | 2.1144e-05 | 0.9468 | N/A | N/A |
| 43500 | 1.752 | 2.0808e-05 | 0.9616 | N/A | N/A |
| 44000 | 1.772 | 2.0472e-05 | 0.9257 | N/A | N/A |
| 44500 | 1.792 | 2.0137e-05 | 0.9199 | N/A | N/A |
| 45000 | 1.812 | 1.9801e-05 | 0.9237 | N/A | N/A |
| 45500 | 1.832 | 1.9466e-05 | 0.9252 | N/A | N/A |
| 46000 | 1.852 | 1.9130e-05 | 0.9198 | N/A | N/A |
| 46500 | 1.872 | 1.8795e-05 | 0.9251 | N/A | N/A |
| 47000 | 1.892 | 1.8459e-05 | 0.8996 | N/A | N/A |
| 47500 | 1.913 | 1.8124e-05 | 0.8661 | N/A | N/A |
| 48000 | 1.933 | 1.7788e-05 | 0.9119 | N/A | N/A |
| 48500 | 1.953 | 1.7453e-05 | 0.9093 | N/A | N/A |
| 49000 | 1.973 | 1.7117e-05 | 0.8893 | N/A | N/A |
| 49500 | 1.993 | 1.6781e-05 | 0.9163 | N/A | N/A |
| 49670 | 2.0 | N/A | N/A | 1.0531 | 0.6141 |
| 50000 | 2.013 | 1.6446e-05 | 0.7699 | N/A | N/A |
| 50500 | 2.033 | 1.6110e-05 | 0.7102 | N/A | N/A |
| 51000 | 2.054 | 1.5775e-05 | 0.7316 | N/A | N/A |
| 51500 | 2.074 | 1.5439e-05 | 0.7196 | N/A | N/A |
| 52000 | 2.094 | 1.5104e-05 | 0.706 | N/A | N/A |
| 52500 | 2.114 | 1.4768e-05 | 0.7302 | N/A | N/A |
| 53000 | 2.134 | 1.4433e-05 | 0.719 | N/A | N/A |
| 53500 | 2.154 | 1.4097e-05 | 0.6997 | N/A | N/A |
| 54000 | 2.174 | 1.3761e-05 | 0.7478 | N/A | N/A |
| 54500 | 2.194 | 1.3426e-05 | 0.7221 | N/A | N/A |
| 55000 | 2.215 | 1.3090e-05 | 0.6901 | N/A | N/A |
| 55500 | 2.235 | 1.2755e-05 | 0.7306 | N/A | N/A |
| 56000 | 2.255 | 1.2419e-05 | 0.6928 | N/A | N/A |
| 56500 | 2.275 | 1.2084e-05 | 0.7387 | N/A | N/A |
| 57000 | 2.295 | 1.1748e-05 | 0.6852 | N/A | N/A |
| 57500 | 2.315 | 1.1413e-05 | 0.7277 | N/A | N/A |
| 58000 | 2.335 | 1.1077e-05 | 0.6858 | N/A | N/A |
| 58500 | 2.356 | 1.0742e-05 | 0.6819 | N/A | N/A |
| 59000 | 2.376 | 1.0406e-05 | 0.7408 | N/A | N/A |
| 59500 | 2.396 | 1.0070e-05 | 0.7442 | N/A | N/A |
| 60000 | 2.416 | 9.7349e-06 | 0.7565 | N/A | N/A |
| 60500 | 2.436 | 9.3994e-06 | 0.7308 | N/A | N/A |
| 61000 | 2.456 | 9.0638e-06 | 0.6853 | N/A | N/A |
| 61500 | 2.476 | 8.7283e-06 | 0.7294 | N/A | N/A |
| 62000 | 2.496 | 8.3927e-06 | 0.7037 | N/A | N/A |
| 62500 | 2.517 | 8.0572e-06 | 0.7001 | N/A | N/A |
| 63000 | 2.537 | 7.7216e-06 | 0.7086 | N/A | N/A |
| 63500 | 2.557 | 7.3861e-06 | 0.7082 | N/A | N/A |
| 64000 | 2.577 | 7.0505e-06 | 0.7219 | N/A | N/A |
| 64500 | 2.597 | 6.7150e-06 | 0.6902 | N/A | N/A |
| 65000 | 2.617 | 6.3794e-06 | 0.6866 | N/A | N/A |
| 65500 | 2.637 | 6.0439e-06 | 0.7045 | N/A | N/A |
| 66000 | 2.658 | 5.7083e-06 | 0.7051 | N/A | N/A |
| 66500 | 2.678 | 5.3728e-06 | 0.6698 | N/A | N/A |
| 67000 | 2.698 | 5.0372e-06 | 0.6839 | N/A | N/A |
| 67500 | 2.718 | 4.7017e-06 | 0.6723 | N/A | N/A |
| 68000 | 2.738 | 4.3661e-06 | 0.6874 | N/A | N/A |
| 68500 | 2.758 | 4.0306e-06 | 0.7674 | N/A | N/A |
| 69000 | 2.778 | 3.6951e-06 | 0.7345 | N/A | N/A |
| 69500 | 2.798 | 3.3595e-06 | 0.6404 | N/A | N/A |
| 70000 | 2.819 | 3.0240e-06 | 0.6571 | N/A | N/A |
| 70500 | 2.839 | 2.6884e-06 | 0.6858 | N/A | N/A |
| 71000 | 2.859 | 2.3529e-06 | 0.7133 | N/A | N/A |
| 71500 | 2.879 | 2.0173e-06 | 0.7026 | N/A | N/A |
| 72000 | 2.899 | 1.6818e-06 | 0.7178 | N/A | N/A |
| 72500 | 2.919 | 1.3462e-06 | 0.6567 | N/A | N/A |
| 73000 | 2.939 | 1.0107e-06 | 0.6738 | N/A | N/A |
| 73500 | 2.96 | 6.7512e-07 | 0.6513 | N/A | N/A |
| 74000 | 2.98 | 3.3957e-07 | 0.6676 | N/A | N/A |
| 74500 | 3.0 | 4.0266e-09 | 0.7003 | N/A | N/A |
| 74505 | 3.0 | N/A | N/A | 1.0927 | 0.6313 |

</details>


## Framework Versions

- **Transformers:** 5.14.0.dev0
- **PyTorch:** 2.13.0+cu130