rufaelfekadu/diactc-combined

DiaCTC — Wav2Vec2 (ClArTTS + ArVoice)

Fine-tuned Wav2Vec2 CTC model for Arabic diacritic restoration from speech + undiacritized transcript, trained on ClArTTS and ArVoice.

This checkpoint is used with DiaCTC constrained WFST decoding. Try the interactive demo: DiaCTC Space.

Installation

1. Clone the repository and create a virtual environment:

git clone https://github.com/rufaelfekadu/DiaCTC.git
cd DiaCTC
python3 -m venv .venv
source .venv/bin/activate

2. Install the package:

pip install -e .

3. Install k2 (required for WFST/CTC lattice decoding). It must match your CUDA and PyTorch versions:

# Example for CUDA 12.8 / PyTorch 2.9.1
pip install k2==1.24.4.dev20251118+cuda12.8.torch2.9.1 -f https://k2-fsa.github.io/k2/cuda.html

Usage

from diactc.models import Wav2Vec2DiacritizationModel

model = Wav2Vec2DiacritizationModel(
    "rufaelfekadu/diactc-wav2vec2-clartts",
    device="cuda",
    use_blank_token=True
)
text, rtf = model.diacritize(
    undiacritized_text,
    audio_path,
    constrained=True,
    method="wfst",
)

Base model: jonatasgrosman/wav2vec2-large-xlsr-53-arabic.

Results (combined training)

Test set	WER ↓	DER ↓
ClArTTS	13.05	3.80
ArVoice	30.36	8.69

Demo: DiaCTC Space