Chatterbox Nano voice encoders for Core AI
The encoders that turn a short recording into a Chatterbox Nano voice, for
OpenReader and Apple's Core AI runtime. They are Chatterbox's own: the S3
speech tokenizer, CAM++ speaker encoder and voice encoder from the official
ResembleAI/chatterbox-nano weights, revision
71ccd1d0081b430592cea481f4307e764e07bc64, with Chatterbox source revision
5de7a54aa4e5e2baadb0182dde554908b48b85c2. No recordings are included.
The pack is separate from the Nano runtime because only someone making a voice needs it. OpenReader runs it once per voice and stores the result with the reference, so narration never loads it.
voice_encoder.aimodel holds five functions:
| Function | Input | Output |
|---|---|---|
tokenizer_mel |
10β15 s at 16 kHz | S3 tokenizer log-mel |
tokenizer |
that mel, padded to a multiple of four, and its valid length | S3 speech tokens |
speaker |
10 s at 16 kHz | CAM++ x-vector, 192 |
prompt_mel |
10 s at 24 kHz | S3 prompt features, 500 Γ 80 |
voice_mel |
trimmed speech at 16 kHz | voice-encoder mel, 40 bands |
The voice encoder's LSTM, its projection and T3's speaker projection are FP32
host tensors in voice-encoder/, named with shapes and checksums in
voice-encoder.json. An unrolled recurrent layer is slow to specialise on
both Core AI backends, and this one is small enough to run on the host.
Weights are FP16 and compute is FP32, on CPU only. The FP32 program was
517 MB. Half-precision compute flipped one S3 token in 1,875 more often than
the CPU's own FP32 noise did. The GPU lowering fails inside MPS. The pack
expects upstream's preprocessing: a 24 kHz reference at -27 LUFS, a 16 kHz
copy of it, and librosa's 20 dB silence trim before voice_mel.
Against upstream PyTorch on a Mac CPU, from identical inputs across five clips:
- S3 tokens agree on at least 99.7%.
- The x-vector cosine is at least 0.9999987.
- Mel errors are at most 1.1e-4.
On an iPhone 15 Pro, OpenReader's Release build reproduced the Mac's
conditionals exactly. See clone-results.json.
MIT licensed, as upstream; see LICENSE.txt.
Model tree for richardr1126/chatterbox-nano-voice-encoder-coreai
Base model
ResembleAI/chatterbox-nano