canbingol commited on
Commit
3c139a0
·
verified ·
1 Parent(s): 4f516e2

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +14 -24
README.md CHANGED
@@ -10,45 +10,37 @@ language:
10
  - en
11
  base_model:
12
  - google/gemma-3-1b-pt
13
- new_version: canbingol/gemma3_1B_base-tr-cpt-1epoch_stage2
14
  ---
15
 
16
- # Model Card: Gemma3-1B Turkish CPT (Experimental)
17
 
18
  ## Overview
19
- This is an experimental Turkish Continued Pretraining (CPT) model based on `google/gemma-3-1b-pt`.
20
 
21
- The model was continued-pretrained for **0.075 epoch** on a Turkish corpus (~19.5M tokens) to improve Turkish language modeling performance and domain adaptation characteristics.
22
 
23
- This release is intended for **research and experimentation purposes only**.
 
 
24
 
25
  ---
26
 
27
  ## Base Model
 
28
  - `google/gemma-3-1b-pt`
29
 
30
  ---
31
 
32
- ## Training Data
33
 
34
  - Dataset: `canbingol/vngrs-web-corpus-200k`
35
  - Subset Used: First 15,000 samples
36
- - Total Tokens: ~19.5M tokens
37
- - Training Objective: Continued Pretraining (Causal LM)
38
- - Effective Epochs: 0.075
39
-
40
-
41
- ## Limitations & Warnings
42
-
43
- - This model is **not instruction-tuned**.
44
- - It may produce raw or unaligned outputs.
45
- - Safety alignment from instruction-tuned variants is not included.
46
- - Performance gains may be domain-specific and limited due to relatively small CPT token size (~19.5M).
47
-
48
- This model is **not intended for production use** without further evaluation and alignment.
49
-
50
  ---
51
 
 
52
  ## Citation
53
 
54
  If you use this model, please cite the base model:
@@ -57,15 +49,13 @@ If you use this model, please cite the base model:
57
 
58
  ---
59
 
60
- ## Usage
61
-
62
- Below is a minimal example using 🤗 Transformers:
63
 
64
  ```python
65
  import torch
66
  from transformers import AutoModelForCausalLM, AutoTokenizer
67
 
68
- model_id = "canbingol/gemma3_1B_base-tr-cpt-1epoch_stage1"
69
 
70
  device = "cuda" if torch.cuda.is_available() else "cpu"
71
 
 
10
  - en
11
  base_model:
12
  - google/gemma-3-1b-pt
 
13
  ---
14
 
15
+ # Model Card: Gemma3-1B Turkish CPT (15K Subset, 3 Epoch)
16
 
17
  ## Overview
 
18
 
19
+ This model is a Turkish Continued Pretraining (CPT) variant of `google/gemma-3-1b-pt`.
20
 
21
+ The base model was further trained for **3 epochs** on the first **15,000 samples** of a Turkish web corpus to improve Turkish language modeling capability and domain familiarity.
22
+
23
+ This release is intended for **research and experimental use**.
24
 
25
  ---
26
 
27
  ## Base Model
28
+
29
  - `google/gemma-3-1b-pt`
30
 
31
  ---
32
 
33
+ ## Training Setup
34
 
35
  - Dataset: `canbingol/vngrs-web-corpus-200k`
36
  - Subset Used: First 15,000 samples
37
+ - Training Objective: Continued Pretraining (Causal LM / Next-Token Prediction)
38
+ - Epochs: 3
39
+ - Data Regime: Plain text (no instruction formatting)
40
+ - Token Count (approximate): ~19.5M tokens
 
 
 
 
 
 
 
 
 
 
41
  ---
42
 
43
+
44
  ## Citation
45
 
46
  If you use this model, please cite the base model:
 
49
 
50
  ---
51
 
52
+ ## Usage Example
 
 
53
 
54
  ```python
55
  import torch
56
  from transformers import AutoModelForCausalLM, AutoTokenizer
57
 
58
+ model_id = "canbingol/gemma3_1B_base-tr-cpt-3epoch_15k_data"
59
 
60
  device = "cuda" if torch.cuda.is_available() else "cpu"
61