Image-Text-to-Text
Transformers
Safetensors
laguna
text-generation
multimodal
vision-language
laguna-xs-2.1
moonvit
bf16
blackwell
conversational
custom_code
Eval Results
Instructions to use webbrain-one/Laguna-XS-2.1-Vision-BF16 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use webbrain-one/Laguna-XS-2.1-Vision-BF16 with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("image-text-to-text", model="webbrain-one/Laguna-XS-2.1-Vision-BF16", trust_remote_code=True) messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("webbrain-one/Laguna-XS-2.1-Vision-BF16", trust_remote_code=True) model = AutoModelForCausalLM.from_pretrained("webbrain-one/Laguna-XS-2.1-Vision-BF16", trust_remote_code=True, device_map="auto") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=40) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use webbrain-one/Laguna-XS-2.1-Vision-BF16 with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "webbrain-one/Laguna-XS-2.1-Vision-BF16" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "webbrain-one/Laguna-XS-2.1-Vision-BF16", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker
docker model run hf.co/webbrain-one/Laguna-XS-2.1-Vision-BF16
- SGLang
How to use webbrain-one/Laguna-XS-2.1-Vision-BF16 with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "webbrain-one/Laguna-XS-2.1-Vision-BF16" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "webbrain-one/Laguna-XS-2.1-Vision-BF16", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "webbrain-one/Laguna-XS-2.1-Vision-BF16" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "webbrain-one/Laguna-XS-2.1-Vision-BF16", "messages": [ { "role": "user", "content": [ { "type": "text", "text": "Describe this image in one sentence." }, { "type": "image_url", "image_url": { "url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg" } } ] } ] }' - Docker Model Runner
How to use webbrain-one/Laguna-XS-2.1-Vision-BF16 with Docker Model Runner:
docker model run hf.co/webbrain-one/Laguna-XS-2.1-Vision-BF16
Document packaged vision artifacts and licenses
Browse files- LICENSE_KIMI_K2.6 +27 -0
- README.md +31 -20
- VISION_ADAPTER_MANIFEST.json +37 -0
LICENSE_KIMI_K2.6
ADDED
|
@@ -0,0 +1,27 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
Modified MIT License
|
| 2 |
+
|
| 3 |
+
Copyright (c) 2026 Moonshot AI
|
| 4 |
+
|
| 5 |
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
| 6 |
+
of this software and associated documentation files (the “Software”), to deal
|
| 7 |
+
in the Software without restriction, including without limitation the rights
|
| 8 |
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
| 9 |
+
copies of the Software, and to permit persons to whom the Software is
|
| 10 |
+
furnished to do so, subject to the following conditions:
|
| 11 |
+
|
| 12 |
+
The above copyright notice and this permission notice shall be included in all
|
| 13 |
+
copies or substantial portions of the Software.
|
| 14 |
+
|
| 15 |
+
THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
| 16 |
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
| 17 |
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
| 18 |
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
| 19 |
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
| 20 |
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
| 21 |
+
SOFTWARE.
|
| 22 |
+
|
| 23 |
+
Our only modification part is that, if the Software (or any derivative works
|
| 24 |
+
thereof) is used for any of your commercial products or services that have
|
| 25 |
+
more than 100 million monthly active users, or more than 20 million US dollars
|
| 26 |
+
(or equivalent in other currencies) in monthly revenue, you shall prominently
|
| 27 |
+
display "Kimi K2.6" on the user interface of such product or service.
|
README.md
CHANGED
|
@@ -1,5 +1,6 @@
|
|
| 1 |
---
|
| 2 |
-
license:
|
|
|
|
| 3 |
library_name: transformers
|
| 4 |
inference: false
|
| 5 |
pipeline_tag: image-text-to-text
|
|
@@ -19,7 +20,7 @@ base_model:
|
|
| 19 |
|
| 20 |

|
| 21 |
|
| 22 |
-
**Laguna XS 2.1 with sight.** A
|
| 23 |
that connects Poolside's agentic coding model to the MoonViT vision encoder from
|
| 24 |
[Kimi-K2.6](https://huggingface.co/moonshotai/Kimi-K2.6) through a trained
|
| 25 |
PatchMerger projector.
|
|
@@ -43,11 +44,10 @@ checkpoint is text-only. This project adds a basic MoonViT vision bridge while
|
|
| 43 |
keeping both the language backbone and vision tower frozen.
|
| 44 |
|
| 45 |
> [!IMPORTANT]
|
| 46 |
-
>
|
| 47 |
-
>
|
| 48 |
-
>
|
| 49 |
-
>
|
| 50 |
-
> a ready-to-serve vision-language checkpoint.
|
| 51 |
|
| 52 |
| Component | Detail |
|
| 53 |
|---|---|
|
|
@@ -67,10 +67,20 @@ keeping both the language backbone and vision tower frozen.
|
|
| 67 |
- [x] Pass backward and 2,048-token memory gates before the full run.
|
| 68 |
- [x] Cache 100,000 MoonViT training examples.
|
| 69 |
- [x] Finish the 100,000-example projector run.
|
| 70 |
-
- [
|
| 71 |
- [ ] Assemble multimodal configuration, processor, and serving integration.
|
| 72 |
- [ ] Pass final single-GPU loading, image inference, and regression gates.
|
| 73 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 74 |
## Provenance
|
| 75 |
|
| 76 |
The packaged text backbone is copied from
|
|
@@ -79,17 +89,17 @@ The packaged text backbone is copied from
|
|
| 79 |
Every copied file was checked against its upstream Git blob or LFS SHA-256
|
| 80 |
fingerprint, then independently rechecked after upload.
|
| 81 |
|
| 82 |
-
Projector training
|
| 83 |
-
MoonViT-3d features.
|
| 84 |
-
verified BF16 backbone above
|
| 85 |
-
|
| 86 |
|
| 87 |
## Usage
|
| 88 |
|
| 89 |
A serving command is intentionally not published yet. The current repository
|
| 90 |
-
contains
|
| 91 |
-
|
| 92 |
-
|
| 93 |
|
| 94 |
## Method credit
|
| 95 |
|
|
@@ -104,10 +114,11 @@ deployment artifacts.
|
|
| 104 |
## License
|
| 105 |
|
| 106 |
The redistributed Laguna XS 2.1 BF16 backbone remains subject to the included
|
| 107 |
-
[OpenMDW-1.1 license](./LICENSE.md) and Poolside's source notices.
|
| 108 |
-
|
| 109 |
-
|
| 110 |
-
|
|
|
|
| 111 |
|
| 112 |
## Acknowledgements
|
| 113 |
|
|
@@ -115,7 +126,7 @@ Built on [Poolside's Laguna XS 2.1 BF16](https://huggingface.co/poolside/Laguna-
|
|
| 115 |
and [Moonshot AI's Kimi-K2.6](https://huggingface.co/moonshotai/Kimi-K2.6), with
|
| 116 |
the vision-attachment method inspired by
|
| 117 |
[Baseten's GLM-5.2-Vision-NVFP4](https://huggingface.co/baseten/GLM-5.2-Vision-NVFP4).
|
| 118 |
-
These teams were not involved in this
|
| 119 |
not direct issues with this repository to them.
|
| 120 |
|
| 121 |
## Want this model on your inference provider?
|
|
|
|
| 1 |
---
|
| 2 |
+
license: other
|
| 3 |
+
license_name: openmdw-1.1-and-kimi-modified-mit
|
| 4 |
library_name: transformers
|
| 5 |
inference: false
|
| 6 |
pipeline_tag: image-text-to-text
|
|
|
|
| 20 |
|
| 21 |

|
| 22 |
|
| 23 |
+
**Laguna XS 2.1 with sight.** A vision-language development checkpoint
|
| 24 |
that connects Poolside's agentic coding model to the MoonViT vision encoder from
|
| 25 |
[Kimi-K2.6](https://huggingface.co/moonshotai/Kimi-K2.6) through a trained
|
| 26 |
PatchMerger projector.
|
|
|
|
| 44 |
keeping both the language backbone and vision tower frozen.
|
| 45 |
|
| 46 |
> [!IMPORTANT]
|
| 47 |
+
> The pinned BF16 text backbone, frozen MoonViT tower, and final 100K-example
|
| 48 |
+
> projector are complete, packaged, and fingerprint-verified. Multimodal model
|
| 49 |
+
> code, processor/serving integration, and final image-inference validation are
|
| 50 |
+
> still pending, so this is not yet a stock ready-to-serve checkpoint.
|
|
|
|
| 51 |
|
| 52 |
| Component | Detail |
|
| 53 |
|---|---|
|
|
|
|
| 67 |
- [x] Pass backward and 2,048-token memory gates before the full run.
|
| 68 |
- [x] Cache 100,000 MoonViT training examples.
|
| 69 |
- [x] Finish the 100,000-example projector run.
|
| 70 |
+
- [x] Add and fingerprint-verify the frozen MoonViT tower and final projector.
|
| 71 |
- [ ] Assemble multimodal configuration, processor, and serving integration.
|
| 72 |
- [ ] Pass final single-GPU loading, image inference, and regression gates.
|
| 73 |
|
| 74 |
+
## Included vision artifacts
|
| 75 |
+
|
| 76 |
+
| File | Contents | Size | SHA-256 |
|
| 77 |
+
|---|---|---:|---|
|
| 78 |
+
| `vision_tower.safetensors` | Frozen MoonViT-3d tower, 329 tensors, all BF16 | 833,765,768 bytes | `befe801bd7dfe8bf5630fef56a7f53c2235065599ca9eea4d995040e2e6fd183` |
|
| 79 |
+
| `mm_projector.safetensors` | Final step-782 PatchMerger projector, 6 tensors, all BF16 | 61,360,104 bytes | `7837384f18be69a4f875ca44a8ed69ec186501d70896daf4b305fa77547974de` |
|
| 80 |
+
|
| 81 |
+
Machine-readable provenance is in
|
| 82 |
+
[`VISION_ADAPTER_MANIFEST.json`](VISION_ADAPTER_MANIFEST.json).
|
| 83 |
+
|
| 84 |
## Provenance
|
| 85 |
|
| 86 |
The packaged text backbone is copied from
|
|
|
|
| 89 |
Every copied file was checked against its upstream Git blob or LFS SHA-256
|
| 90 |
fingerprint, then independently rechecked after upload.
|
| 91 |
|
| 92 |
+
Projector training used the frozen BF16 Laguna XS 2.1 backbone and frozen
|
| 93 |
+
MoonViT-3d features. This repository now pairs the final trained projector and
|
| 94 |
+
frozen tower with the verified BF16 backbone above. End-to-end behavior will be
|
| 95 |
+
validated after the remaining multimodal integration is assembled.
|
| 96 |
|
| 97 |
## Usage
|
| 98 |
|
| 99 |
A serving command is intentionally not published yet. The current repository
|
| 100 |
+
contains the complete text backbone, frozen vision tower, and final projector,
|
| 101 |
+
but not the multimodal processor and serving assembly that connects them. A
|
| 102 |
+
tested quickstart will be added after integration passes the final gates.
|
| 103 |
|
| 104 |
## Method credit
|
| 105 |
|
|
|
|
| 114 |
## License
|
| 115 |
|
| 116 |
The redistributed Laguna XS 2.1 BF16 backbone remains subject to the included
|
| 117 |
+
[OpenMDW-1.1 license](./LICENSE.md) and Poolside's source notices. The included
|
| 118 |
+
MoonViT tower remains subject to the included
|
| 119 |
+
[Kimi-K2.6 Modified MIT terms](./LICENSE_KIMI_K2.6). The newly trained projector
|
| 120 |
+
is documented in the manifest above. Downstream users remain responsible for
|
| 121 |
+
complying with both upstream licenses.
|
| 122 |
|
| 123 |
## Acknowledgements
|
| 124 |
|
|
|
|
| 126 |
and [Moonshot AI's Kimi-K2.6](https://huggingface.co/moonshotai/Kimi-K2.6), with
|
| 127 |
the vision-attachment method inspired by
|
| 128 |
[Baseten's GLM-5.2-Vision-NVFP4](https://huggingface.co/baseten/GLM-5.2-Vision-NVFP4).
|
| 129 |
+
These teams were not involved in this development checkpoint; please do
|
| 130 |
not direct issues with this repository to them.
|
| 131 |
|
| 132 |
## Want this model on your inference provider?
|
VISION_ADAPTER_MANIFEST.json
ADDED
|
@@ -0,0 +1,37 @@
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 1 |
+
{
|
| 2 |
+
"schema_version": 1,
|
| 3 |
+
"package_kind": "vision_language_checkpoint_assets",
|
| 4 |
+
"text_backbone": {
|
| 5 |
+
"model_id": "poolside/Laguna-XS-2.1",
|
| 6 |
+
"revision": "e9df9a59996d790b94b70f3fef343fe1d9e34bdf",
|
| 7 |
+
"variant": "BF16",
|
| 8 |
+
"packaged_in_repo": true
|
| 9 |
+
},
|
| 10 |
+
"vision_tower": {
|
| 11 |
+
"filename": "vision_tower.safetensors",
|
| 12 |
+
"source_model": "moonshotai/Kimi-K2.6",
|
| 13 |
+
"source_revision": "7eb5002f6aadc958aed6a9177b7ed26bb94011bb",
|
| 14 |
+
"source_prefix": "vision_tower.",
|
| 15 |
+
"tensor_count": 329,
|
| 16 |
+
"dtypes": ["BF16"],
|
| 17 |
+
"parameter_count": 416866032,
|
| 18 |
+
"size_bytes": 833765768,
|
| 19 |
+
"sha256": "befe801bd7dfe8bf5630fef56a7f53c2235065599ca9eea4d995040e2e6fd183"
|
| 20 |
+
},
|
| 21 |
+
"projector": {
|
| 22 |
+
"filename": "mm_projector.safetensors",
|
| 23 |
+
"training_step": 782,
|
| 24 |
+
"training_examples": 100000,
|
| 25 |
+
"training_epochs": 1,
|
| 26 |
+
"training_world_size": 4,
|
| 27 |
+
"tensor_count": 6,
|
| 28 |
+
"dtypes": ["BF16"],
|
| 29 |
+
"parameter_count": 30679808,
|
| 30 |
+
"size_bytes": 61360104,
|
| 31 |
+
"sha256": "7837384f18be69a4f875ca44a8ed69ec186501d70896daf4b305fa77547974de"
|
| 32 |
+
},
|
| 33 |
+
"serving": {
|
| 34 |
+
"multimodal_integration_packaged": false,
|
| 35 |
+
"end_to_end_validated": false
|
| 36 |
+
}
|
| 37 |
+
}
|