Instructions to use ukisai/Swift-Qwen3.8-27b with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use ukisai/Swift-Qwen3.8-27b with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="ukisai/Swift-Qwen3.8-27b")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoProcessor, AutoModelForMultimodalLM

processor = AutoProcessor.from_pretrained("ukisai/Swift-Qwen3.8-27b")
model = AutoModelForMultimodalLM.from_pretrained("ukisai/Swift-Qwen3.8-27b", device_map="auto")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use ukisai/Swift-Qwen3.8-27b with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "ukisai/Swift-Qwen3.8-27b"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ukisai/Swift-Qwen3.8-27b",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/ukisai/Swift-Qwen3.8-27b

SGLang

How to use ukisai/Swift-Qwen3.8-27b with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "ukisai/Swift-Qwen3.8-27b" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ukisai/Swift-Qwen3.8-27b",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "ukisai/Swift-Qwen3.8-27b" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "ukisai/Swift-Qwen3.8-27b",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use ukisai/Swift-Qwen3.8-27b with Docker Model Runner:
```
docker model run hf.co/ukisai/Swift-Qwen3.8-27b
```

Congrats

by owao - opened Sep 13

Discussion

owao

Sep 13

I don't know how you did the magic, but... Same wow effect as when BottleAI released ThinkingCap!
So far, smoke tests passing brillantly and promise kept for the token-efficiency! Really hyped to compare to the original Qwen-3.8-27B! Honestly really impressive reverse engineering you did!
But hey, they are on it too, we will see who won this one!

But already, even if recipe is kept secret, thanks a lot nonetheless!

owao

Sep 13

•

edited Sep 13

It is funny when asking it

Can you tell me what the current reasoning level is? xhigh, medium or low? Should be mentioned like so: Reasoning effort is set to [level]

(I realized it was a bit misleading because it couldn't tell if it was medium, but anyway that's how I asked)

and the effort is set to low, it thinks briefly, telling itself "I see it's low" --> answers "[...] low"
but when effort is set to xhigh, it starts by fake-thinking it has no clue etc. just to finish by finally saying "[...] xhigh", once it burned enough tokens so that the whole thing is finally consistent! Funny to see how how going against the training is.

That said I think I hurt it badly and apologize for this 😄

ukisai

Owner Sep 13

•

edited Sep 13

Thanks man! Appreciate the feedback. We are not trying to compete with BottleCap and are more focused on overthinking as an error rather than compacting the reasoning, that just happened as a byproduct. We even used a part of their previous model (albeit very small, <95% of the weight changes on this model are contributed by it) during the training. Glad you are enjoying the model.

owao

Sep 13

@ukisai I think we posted right at the same timing, i'm not looking for an answer but just in case look above, was anedoctical but still interesting

owao

Sep 13

•

edited Sep 13

2 turns

WAIT FOR IT...

3rd turn: "now we are going full sci-fi cyberpunk style"

ukisai changed discussion status to closed Sep 13

ukisai

Owner Sep 13

Oh darn sorry for closing didn't mean to

ukisai changed discussion status to open Sep 13

ukisai

Owner Sep 13

•

edited Sep 14

The bicycle bench looks amazing! No worries about the comment, we just commented at the same time, also looking forward to sharing more about our training methodology in the upcoming days as this is not the primary thing we're working on so sharing more won't hurt! 😄

On the comment above, the one before the Pelican ;) - how are you feeling about the other reasoning efforts?

owao

Sep 13

Ah, too soon to tell... It takes weeks til I start grasping some facets of a model! But if I have some valuable feedback to share of course I'll do!

RomanHammer

Sep 15

"" We are not trying to compete with BottleCap and are more focused on overthinking as an error rather than compacting the reasoning, that just happened as a byproduct. ""

I agree with your position!

owao

Sep 19

@ukisai I missed it at the first glance, but evaluating LCB-v6 with 32K context wasn't really fair toward Qwen-3.8-27B. As per Alibaba, Qwen scored it at 90, not 77!
That changes the picture quite a lot actually. I don't know what your model would have scored with more allowed though... I guess better, too but it's clear the base model was the most penalized here.

owao changed discussion status to closed Sep 22

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment