Instructions to use Caldalis/MiniTranslator-2B-GGUF with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use Caldalis/MiniTranslator-2B-GGUF with Transformers:
# Use a pipeline as a high-level helper # Warning: Pipeline type "translation" is no longer supported in transformers v5. # You must load the model directly (see below) or downgrade to v4.x with: # pip install "transformers<5.0.0" from transformers import pipeline pipe = pipeline("translation", model="Caldalis/MiniTranslator-2B-GGUF")# pip install -U transformers accelerate # Load model directly from transformers import AutoModel model = AutoModel.from_pretrained("Caldalis/MiniTranslator-2B-GGUF", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use Caldalis/MiniTranslator-2B-GGUF with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf Caldalis/MiniTranslator-2B-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf Caldalis/MiniTranslator-2B-GGUF:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf Caldalis/MiniTranslator-2B-GGUF:Q4_K_M # Run inference directly in the terminal: llama cli -hf Caldalis/MiniTranslator-2B-GGUF:Q4_K_M
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf Caldalis/MiniTranslator-2B-GGUF:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama cli -hf Caldalis/MiniTranslator-2B-GGUF:Q4_K_M
Use Docker
docker model run hf.co/Caldalis/MiniTranslator-2B-GGUF:Q4_K_M
- Llama new
- LM Studio
- Jan
- Ollama
How to use Caldalis/MiniTranslator-2B-GGUF with Ollama:
ollama run hf.co/Caldalis/MiniTranslator-2B-GGUF:Q4_K_M
- Unsloth Desktop
- Pi
How to use Caldalis/MiniTranslator-2B-GGUF with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf Caldalis/MiniTranslator-2B-GGUF:Q4_K_M
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:9931/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "Caldalis/MiniTranslator-2B-GGUF:Q4_K_M" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use Caldalis/MiniTranslator-2B-GGUF with Docker Model Runner:
docker model run hf.co/Caldalis/MiniTranslator-2B-GGUF:Q4_K_M
- Lemonade
How to use Caldalis/MiniTranslator-2B-GGUF with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull Caldalis/MiniTranslator-2B-GGUF:Q4_K_M
Run and chat with the model
lemonade run user.MiniTranslator-2B-GGUF-Q4_K_M
List all available models
lemonade list
- Hermes Agent
How to use Caldalis/MiniTranslator-2B-GGUF with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf Caldalis/MiniTranslator-2B-GGUF:Q4_K_M
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:9931/v1 hermes config set model.default Caldalis/MiniTranslator-2B-GGUF:Q4_K_M
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use Caldalis/MiniTranslator-2B-GGUF with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf Caldalis/MiniTranslator-2B-GGUF:Q4_K_M
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:9931/v1 \ --custom-model-id "Caldalis/MiniTranslator-2B-GGUF:Q4_K_M" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
MiniTranslator-2B GGUF
MiniTranslator-2B η GGUF ηζ¬οΌη± MiniTranslator-2BοΌBF16 safetensorsοΌθ½¬ζ’δΈιεγ
ζδ»Ά
| ζδ»Ά | ιε | ε€§ε° | SHA-256 |
|---|---|---|---|
MiniTranslator-2B-F16.gguf |
F16 | 5,039,006,496 B | 236b69f8d40370ac390ff6f48283349d66318d654094a643d3641f148aa8e958 |
MiniTranslator-2B-Q8_0.gguf |
Q8_0 | 2,679,710,496 B | 5999502dd1e520b18fcfa5bade61b64b7b119fa991d16f15b816f4f3b46d1997 |
MiniTranslator-2B-Q6_K.gguf |
Q6_K | 2,070,225,696 B | df7895713c691700c36b1db458a8998cb2ac8cd60e8cff1cb8976a3fe500dd93 |
MiniTranslator-2B-Q4_K_M.gguf |
Q4_K_M | 1,561,318,176 B | e7ebfd5120803c59163ed0cb44725c1e43e97be8434b031474c36ef32a867114 |
η¨ζ³
llama-server -m MiniTranslator-2B-Q4_K_M.gguf -c 4096 --reasoning off --port 8080
ηΏ»θ―θ―·ζ±εΊε
³ιζθοΌε¦ε樑εε―θ½ε
θΎεΊ <think> εγθΎζ°ζε»Ίη¨ --reasoning offοΌ
ζ§ζε»Ίε»ζθ―₯εζ°οΌεΉΆε¨θ―·ζ±δ½ε "chat_template_kwargs": {"enable_thinking": false}γ
curl http://127.0.0.1:8080/v1/chat/completions \
-H 'Content-Type: application/json' \
-d '{
"messages": [{"role": "user", "content": "ε°δ»₯δΈζζ¬ηΏ»θ―ζθ±ζοΌεͺθΎεΊθ―ζοΌ\n\nδΊΊε·₯ζΊθ½ζ£ε¨ζΉεδΈηγ"}],
"temperature": 0,
"max_tokens": 4096
}'
llama-cli ε―ΉεΊεζ°οΌ
llama-cli -m MiniTranslator-2B-Q4_K_M.gguf --reasoning off \
-p 'ε°δ»₯δΈζζ¬ηΏ»θ―ζθ±ζοΌεͺθΎεΊθ―ζοΌδΊΊε·₯ζΊθ½ζ£ε¨ζΉεδΈηγ'
转ζ’δΏ‘ζ―
- ζΊζιοΌ
MiniTranslator-2B/model.safetensorsοΌsha2567e75a2304bd5b9931c331138f89b7fd8f8673f19c2f3916d09a8332c208a8185οΌ - 转ζ’οΌllama.cpp
convert_hf_to_gguf.pyοΌ--outtype f16 --model-name MiniTranslator-2B - ιεοΌllama.cpp
llama-quantizeοΌF16 δΈΊζ―ζ¬οΌQ8_0/Q6_K/Q4_K_M δΎζ¬‘ιεοΌ
樑εε‘γθ―ζ΅η»ζδΈζ¨θζη€Ίθ―θ§δΈ»δ»εΊ README.md / README_zh.mdγ
- Downloads last month
- -
4-bit
6-bit
8-bit
16-bit