谢谢作者，这个项目还是很有意义的。
#2
by dremit9105 - opened
我用T4显卡跑，这个是兼顾质量和速度最好的选择，kj的w4a8没你这个快
These mixed convrot quantizations are faster because the INT8 layers are mixed with INT4 layers, preserving much of the INT4 speed advantage. Your T4 graphics card has double the INT4 compute throughput compared to INT8, plus the benefit of the decreased model size and reduced memory bandwidth bottleneck, which is why you're seeing that speed boost. Glad you find them useful 💚 Kijai's W4A8 format is geared more toward RAM/VRAM-constrained systems (INT4 model size with INT8 activations) to achieve near-INT8 quality while fitting into tighter memory budgets.