Duplicate from ayousanz/piper-plus-tsukuyomi-chan

Co-authored-by: yousan <ayousanz@users.noreply.huggingface.co>

.gitattributes

@@ -0,0 +1,41 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+samples/sample_01_konnichiwa.wav filter=lfs diff=lfs merge=lfs -text
+samples/sample_02_wagahai.wav filter=lfs diff=lfs merge=lfs -text
+samples/sample_03_onsei.wav filter=lfs diff=lfs merge=lfs -text
+samples/sample_ja.wav filter=lfs diff=lfs merge=lfs -text
+samples/sample_en.wav filter=lfs diff=lfs merge=lfs -text
+samples/sample_es.wav filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,157 @@
+---
+license: other
+license_name: tsukuyomi-chan-corpus
+license_link: https://tyc.rei-yumesaki.net/material/corpus/#terms3
+language:
+- ja
+- en
+- zh
+- es
+- fr
+- pt
+pipeline_tag: text-to-speech
+tags:
+- tts
+- vits
+- multilingual
+- japanese
+- piper
+- prosody
+---
+
+# Piper Plus つくよみちゃん (Multilingual 6-Language + Prosody)
+
+[つくよみちゃんコーパス](https://tyc.rei-yumesaki.net/material/corpus/)を使用して6言語マルチリンガルベースモデルからファインチューニングした多言語TTSモデルです。
+
+prosody features (A1/A2/A3) に対応し、日本語・英語・中国語・スペイン語・フランス語・ポルトガル語の6言語でつくよみちゃんの声質で音声合成が可能です。
+
+## Sample Audio
+
+| テキスト | 言語 | 音声 |
+|----------|------|------|
+| こんにちは、つくよみちゃんです。 | JA | [sample_ja](samples/sample_ja.wav) |
+| Hello, how are you today? | EN | [sample_en](samples/sample_en.wav) |
+| 你好，今天天气很好。 | ZH | [sample_zh](samples/sample_zh.wav) |
+| Hola, como estas hoy? | ES | [sample_es](samples/sample_es.wav) |
+| Bonjour, comment allez-vous? | FR | [sample_fr](samples/sample_fr.wav) |
+| Ola, como voce esta hoje? | PT | [sample_pt](samples/sample_pt.wav) |
+
+## Model Details
+
+| 項目 | 値 |
+|------|-----|
+| アーキテクチャ | VITS |
+| 言語 | 日本語 (ja), 英語 (en), 中国語 (zh), スペイン語 (es), フランス語 (fr), ポルトガル語 (pt) |
+| サンプルレート | 22050 Hz |
+| 品質 | medium |
+| 音素タイプ | multilingual |
+| 音素数 | 173 |
+| prosody_dim | 16 |
+| 話者数 | 1 (つくよみちゃん) |
+| 言語数 | 6 |
+| ベースモデル | [ayousanz/piper-plus-base](https://huggingface.co/ayousanz/piper-plus-base) (6言語マルチリンガル版、571話者) |
+| ファインチューニング | 500 epochs, lr=2e-5, freeze-dp |
+| 学習データ | つくよみちゃんコーパス (100発話, ~11分) |
+| ONNXエクスポート | stochastic + EMA + emb_lang後処理 + FP16 (38MB) |
+
+## Inference Results
+
+| テキスト | 言語 | 音声長 |
+|---------|------|--------|
+| こんにちは、つくよみちゃんです。 | JA | 3.05s |
+| Hello, how are you today? | EN | 2.54s |
+| 你好，今天天气很好。 | ZH | 1.21s |
+| Hola, como estas hoy? | ES | 2.86s |
+| Bonjour, comment allez-vous? | FR | 2.11s |
+| Ola, como voce esta hoje? | PT | 2.24s |
+
+## Usage
+
+### 推論（テキスト直接入力）
+
+```bash
+# 日本語
+CUDA_VISIBLE_DEVICES="" uv run python -m piper_train.infer_onnx \
+  --model tsukuyomi-chan-6lang-fp16.onnx \
+  --config config.json \
+  --output-dir ./output \
+  --text "こんにちは、今日は良い天気ですね。" \
+  --language ja-en-zh-es-fr-pt \
+  --speaker-id 0 \
+  --noise-scale 0.667
+
+# 英語
+CUDA_VISIBLE_DEVICES="" uv run python -m piper_train.infer_onnx \
+  --model tsukuyomi-chan-6lang-fp16.onnx \
+  --config config.json \
+  --output-dir ./output \
+  --text "Hello, how are you today?" \
+  --language ja-en-zh-es-fr-pt \
+  --speaker-id 0 \
+  --noise-scale 0.667
+
+# 混合テキスト（コードスイッチング）
+CUDA_VISIBLE_DEVICES="" uv run python -m piper_train.infer_onnx \
+  --model tsukuyomi-chan-6lang-fp16.onnx \
+  --config config.json \
+  --output-dir ./output \
+  --text "今日はgood morningですね" \
+  --language ja-en-zh-es-fr-pt \
+  --speaker-id 0 \
+  --noise-scale 0.667
+```
+
+### 推奨推論パラメータ
+
+| パラメータ | 値 | 説明 |
+|-----------|-----|------|
+| `--noise-scale` | 0.667 | デフォルト推奨値 |
+| `--speaker-id` | 0 | 単一話者 |
+| `--language` | ja-en-zh-es-fr-pt | 6言語自動検出 |
+
+## Training
+
+### ベースモデル
+
+[ayousanz/piper-plus-base](https://huggingface.co/ayousanz/piper-plus-base) の6言語マルチリンガルベースモデル（571話者、508,187発話、75 epoch）からファインチューニング。
+
+### 転移学習方式
+
+1. `--resume-from-multispeaker-checkpoint` でベースモデルからロード
+   - `emb_g` (571話者分の話者埋め込み) を自動除去
+   - `emb_lang` に conditioning 分布補正を適用
+   - `--freeze-dp` を自動有効化（Duration Predictor の catastrophic forgetting 防止）
+2. 500 epoch ファインチューニング (100発話、lr=2e-5)
+3. ONNX エクスポート前に `emb_lang[0]` (JA) → `emb_lang[1:5]` (EN/ZH/ES/FR/PT) にコピーして声質統一
+
+### 学習コマンド
+
+```bash
+uv run python -m piper_train \
+  --dataset-dir /path/to/tsukuyomi-dataset \
+  --prosody-dim 16 \
+  --accelerator gpu --devices 1 --precision 32-true \
+  --max_epochs 500 --batch-size 4 --samples-per-speaker 4 \
+  --checkpoint-epochs 50 --quality medium \
+  --base_lr 2e-5 --disable_auto_lr_scaling \
+  --ema-decay 0.9995 \
+  --max-phoneme-ids 400 \
+  --no-wavlm \
+  --resume-from-multispeaker-checkpoint /path/to/base_model.ckpt \
+  --default_root_dir /path/to/output
+```
+
+## Files
+
+- `tsukuyomi-chan-6lang-fp16.onnx` - ONNXモデル (38MB, FP16, emb_lang後処理済み)
+- `config.json` - モデル設定 (173音素マップ、6言語)
+- `samples/` - サンプル音声
+
+## Credits
+
+- **つくよみちゃんコーパス**: [夢前黎](https://tyc.rei-yumesaki.net/)
+- **Piper TTS**: [rhasspy/piper](https://github.com/rhasspy/piper)
+
+## License
+
+ライセンスは、[つくよみちゃんコーパス│声優統計コーパス（JVSコーパス準拠）](https://tyc.rei-yumesaki.net/material/corpus/)に準じます

config.json

@@ -0,0 +1,590 @@
+{
+    "dataset": "piper",
+    "audio": {
+        "sample_rate": 22050,
+        "quality": "dataset-tsukuyomi-finetune-6lang"
+    },
+    "espeak": {
+        "voice": "multilingual"
+    },
+    "language": {
+        "code": "multilingual"
+    },
+    "inference": {
+        "noise_scale": 0.667,
+        "length_scale": 1,
+        "noise_w": 0.8
+    },
+    "phoneme_type": "multilingual",
+    "phoneme_map": {},
+    "phoneme_id_map": {
+        "_": [
+            0
+        ],
+        "^": [
+            1
+        ],
+        "$": [
+            2
+        ],
+        "?": [
+            3
+        ],
+        "": [
+            4
+        ],
+        "": [
+            5
+        ],
+        "": [
+            6
+        ],
+        "#": [
+            7
+        ],
+        "[": [
+            8
+        ],
+        "]": [
+            9
+        ],
+        "a": [
+            10
+        ],
+        "i": [
+            11
+        ],
+        "u": [
+            12
+        ],
+        "e": [
+            13
+        ],
+        "o": [
+            14
+        ],
+        "A": [
+            15
+        ],
+        "I": [
+            16
+        ],
+        "U": [
+            17
+        ],
+        "E": [
+            18
+        ],
+        "O": [
+            19
+        ],
+        "": [
+            20
+        ],
+        "": [
+            21
+        ],
+        "": [
+            22
+        ],
+        "": [
+            23
+        ],
+        "": [
+            24
+        ],
+        "N": [
+            25
+        ],
+        "": [
+            26
+        ],
+        "": [
+            27
+        ],
+        "": [
+            28
+        ],
+        "": [
+            29
+        ],
+        "": [
+            30
+        ],
+        "q": [
+            31
+        ],
+        "k": [
+            32
+        ],
+        "": [
+            33
+        ],
+        "": [
+            34
+        ],
+        "g": [
+            35
+        ],
+        "": [
+            36
+        ],
+        "": [
+            37
+        ],
+        "t": [
+            38
+        ],
+        "": [
+            39
+        ],
+        "d": [
+            40
+        ],
+        "": [
+            41
+        ],
+        "p": [
+            42
+        ],
+        "": [
+            43
+        ],
+        "b": [
+            44
+        ],
+        "": [
+            45
+        ],
+        "": [
+            46
+        ],
+        "": [
+            47
+        ],
+        "s": [
+            48
+        ],
+        "": [
+            49
+        ],
+        "z": [
+            50
+        ],
+        "j": [
+            51
+        ],
+        "": [
+            52
+        ],
+        "f": [
+            53
+        ],
+        "h": [
+            54
+        ],
+        "": [
+            55
+        ],
+        "v": [
+            56
+        ],
+        "n": [
+            57
+        ],
+        "": [
+            58
+        ],
+        "m": [
+            59
+        ],
+        "": [
+            60
+        ],
+        "r": [
+            61
+        ],
+        "": [
+            62
+        ],
+        "w": [
+            63
+        ],
+        "y": [
+            64
+        ],
+        "ɑ": [
+            65
+        ],
+        "æ": [
+            66
+        ],
+        "ʌ": [
+            67
+        ],
+        "ə": [
+            68
+        ],
+        "ɔ": [
+            69
+        ],
+        "ɛ": [
+            70
+        ],
+        "ɚ": [
+            71
+        ],
+        "ɜ": [
+            72
+        ],
+        "ɪ": [
+            73
+        ],
+        "ʊ": [
+            74
+        ],
+        "ː": [
+            75
+        ],
+        "": [
+            76
+        ],
+        "l": [
+            77
+        ],
+        "ɡ": [
+            78
+        ],
+        "ŋ": [
+            79
+        ],
+        "ɹ": [
+            80
+        ],
+        "ʃ": [
+            81
+        ],
+        "ʒ": [
+            82
+        ],
+        "θ": [
+            83
+        ],
+        "ð": [
+            84
+        ],
+        "": [
+            85
+        ],
+        "": [
+            86
+        ],
+        "ˈ": [
+            87
+        ],
+        "ˌ": [
+            88
+        ],
+        " ": [
+            89
+        ],
+        ",": [
+            90
+        ],
+        ".": [
+            91
+        ],
+        ";": [
+            92
+        ],
+        ":": [
+            93
+        ],
+        "!": [
+            94
+        ],
+        "-": [
+            95
+        ],
+        "'": [
+            96
+        ],
+        "": [
+            97
+        ],
+        "": [
+            98
+        ],
+        "": [
+            99
+        ],
+        "": [
+            100
+        ],
+        "": [
+            101
+        ],
+        "ɕ": [
+            102
+        ],
+        "": [
+            103
+        ],
+        "": [
+            104
+        ],
+        "ʂ": [
+            105
+        ],
+        "ɻ": [
+            106
+        ],
+        "": [
+            107
+        ],
+        "x": [
+            108
+        ],
+        "ɤ": [
+            109
+        ],
+        "": [
+            110
+        ],
+        "": [
+            111
+        ],
+        "": [
+            112
+        ],
+        "": [
+            113
+        ],
+        "": [
+            114
+        ],
+        "": [
+            115
+        ],
+        "": [
+            116
+        ],
+        "": [
+            117
+        ],
+        "": [
+            118
+        ],
+        "": [
+            119
+        ],
+        "": [
+            120
+        ],
+        "": [
+            121
+        ],
+        "": [
+            122
+        ],
+        "": [
+            123
+        ],
+        "": [
+            124
+        ],
+        "": [
+            125
+        ],
+        "": [
+            126
+        ],
+        "": [
+            127
+        ],
+        "": [
+            128
+        ],
+        "": [
+            129
+        ],
+        "": [
+            130
+        ],
+        "": [
+            131
+        ],
+        "": [
+            132
+        ],
+        "": [
+            133
+        ],
+        "": [
+            134
+        ],
+        "": [
+            135
+        ],
+        "": [
+            136
+        ],
+        "": [
+            137
+        ],
+        "": [
+            138
+        ],
+        "": [
+            139
+        ],
+        "": [
+            140
+        ],
+        "ɨ": [
+            141
+        ],
+        "": [
+            142
+        ],
+        "": [
+            143
+        ],
+        "": [
+            144
+        ],
+        "": [
+            145
+        ],
+        "": [
+            146
+        ],
+        "ɲ": [
+            147
+        ],
+        "ɾ": [
+            148
+        ],
+        "": [
+            149
+        ],
+        "β": [
+            150
+        ],
+        "ɣ": [
+            151
+        ],
+        "ʝ": [
+            152
+        ],
+        "¡": [
+            153
+        ],
+        "¿": [
+            154
+        ],
+        "": [
+            155
+        ],
+        "": [
+            156
+        ],
+        "": [
+            157
+        ],
+        "ø": [
+            158
+        ],
+        "œ": [
+            159
+        ],
+        "ɥ": [
+            160
+        ],
+        "ʁ": [
+            161
+        ],
+        "—": [
+            162
+        ],
+        "–": [
+            163
+        ],
+        "…": [
+            164
+        ],
+        "«": [
+            165
+        ],
+        "»": [
+            166
+        ],
+        "ã": [
+            167
+        ],
+        "ẽ": [
+            168
+        ],
+        "ĩ": [
+            169
+        ],
+        "õ": [
+            170
+        ],
+        "ũ": [
+            171
+        ],
+        "ʎ": [
+            172
+        ]
+    },
+    "num_symbols": 173,
+    "num_speakers": 1,
+    "speaker_id_map": {},
+    "piper_version": "1.5.4",
+    "num_languages": 6,
+    "language_id_map": {
+        "ja": 0,
+        "en": 1,
+        "zh": 2,
+        "es": 3,
+        "fr": 4,
+        "pt": 5
+    },
+    "prosody_num_symbols": 11,
+    "prosody_id_map": {
+        "0": [
+            0
+        ],
+        "1": [
+            1
+        ],
+        "2": [
+            2
+        ],
+        "3": [
+            3
+        ],
+        "4": [
+            4
+        ],
+        "5": [
+            5
+        ],
+        "6": [
+            6
+        ],
+        "7": [
+            7
+        ],
+        "8": [
+            8
+        ],
+        "9": [
+            9
+        ],
+        "10": [
+            10
+        ]
+    }
+}

samples/sample_en.wav

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:b35ac23f8f7f0e431412688332aebe13dc32cc3d2175a4bdecd9fe7d18c29fc3
+size 112172

samples/sample_es.wav

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:232cbc96fdf051fbbbef5897e3be4fdc368d8e784187b781675f49b53e3d5332
+size 125996

samples/sample_ja.wav

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:a56635dd0fe01cafb1b0f4f2ab45a427496e1727ca32fa5fb3d2cd9d654afe05
+size 134700

tsukuyomi-chan-6lang-fp16.onnx

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7c8e909967370a01230ca173fd4541364f30aa3178597b6000122bec92597644
+size 39216913