MiniTranslator-2B GGUF

MiniTranslator-2B ηš„ GGUF η‰ˆζœ¬οΌŒη”± MiniTranslator-2B(BF16 safetensorsοΌ‰θ½¬ζ’δΈŽι‡εŒ–γ€‚

ζ–‡δ»Ά

ζ–‡δ»Ά ι‡εŒ– 倧小 SHA-256
MiniTranslator-2B-F16.gguf F16 5,039,006,496 B 236b69f8d40370ac390ff6f48283349d66318d654094a643d3641f148aa8e958
MiniTranslator-2B-Q8_0.gguf Q8_0 2,679,710,496 B 5999502dd1e520b18fcfa5bade61b64b7b119fa991d16f15b816f4f3b46d1997
MiniTranslator-2B-Q6_K.gguf Q6_K 2,070,225,696 B df7895713c691700c36b1db458a8998cb2ac8cd60e8cff1cb8976a3fe500dd93
MiniTranslator-2B-Q4_K_M.gguf Q4_K_M 1,561,318,176 B e7ebfd5120803c59163ed0cb44725c1e43e97be8434b031474c36ef32a867114

用法

llama-server -m MiniTranslator-2B-Q4_K_M.gguf -c 4096 --reasoning off --port 8080

ηΏ»θ―‘θ―·ζ±‚εΊ”ε…³ι—­ζ€θ€ƒοΌŒε¦εˆ™ζ¨‘εž‹ε―θƒ½ε…ˆθΎ“ε‡Ί <think> ε—γ€‚θΎƒζ–°ζž„ε»Ίη”¨ --reasoning offοΌ› ζ—§ζž„ε»ΊεŽ»ζŽ‰θ―₯ε‚ζ•°οΌŒεΉΆεœ¨θ―·ζ±‚δ½“εŠ  "chat_template_kwargs": {"enable_thinking": false}。

curl http://127.0.0.1:8080/v1/chat/completions \
  -H 'Content-Type: application/json' \
  -d '{
    "messages": [{"role": "user", "content": "ε°†δ»₯δΈ‹ζ–‡ζœ¬ηΏ»θ―‘ζˆθ‹±ζ–‡οΌŒεͺθΎ“ε‡Ίθ―‘ζ–‡οΌš\n\nδΊΊε·₯ζ™Ίθƒ½ζ­£εœ¨ζ”Ήε˜δΈ–η•Œγ€‚"}],
    "temperature": 0,
    "max_tokens": 4096
  }'

llama-cli ε―ΉεΊ”ε‚ζ•°οΌš

llama-cli -m MiniTranslator-2B-Q4_K_M.gguf --reasoning off \
  -p 'ε°†δ»₯δΈ‹ζ–‡ζœ¬ηΏ»θ―‘ζˆθ‹±ζ–‡οΌŒεͺθΎ“ε‡Ίθ―‘ζ–‡οΌšδΊΊε·₯ζ™Ίθƒ½ζ­£εœ¨ζ”Ήε˜δΈ–η•Œγ€‚'

转捒俑息

  • ζΊζƒι‡οΌšMiniTranslator-2B/model.safetensors(sha256 7e75a2304bd5b9931c331138f89b7fd8f8673f19c2f3916d09a8332c208a8185οΌ‰
  • 转捒:llama.cpp convert_hf_to_gguf.py,--outtype f16 --model-name MiniTranslator-2B
  • ι‡εŒ–οΌšllama.cpp llama-quantize(F16 为母本,Q8_0/Q6_K/Q4_K_M δΎζ¬‘ι‡εŒ–οΌ‰

ζ¨‘εž‹ε‘γ€θ―„ζ΅‹η»“ζžœδΈŽζŽ¨θζη€Ίθ―θ§δΈ»δ»“εΊ“ README.md / README_zh.md。

Downloads last month
-
GGUF
Model size
3B params
Architecture
llama
Hardware compatibility
Log In to add your hardware

4-bit

6-bit

8-bit

16-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. πŸ™‹ Ask for provider support

Model tree for Caldalis/MiniTranslator-2B-GGUF

Quantized
(2)
this model