Instructions to use simaai/LFM2-1.2B-Autoround-Safetensors with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use simaai/LFM2-1.2B-Autoround-Safetensors with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="simaai/LFM2-1.2B-Autoround-Safetensors")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("simaai/LFM2-1.2B-Autoround-Safetensors")
model = AutoModelForCausalLM.from_pretrained("simaai/LFM2-1.2B-Autoround-Safetensors", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use simaai/LFM2-1.2B-Autoround-Safetensors with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "simaai/LFM2-1.2B-Autoround-Safetensors"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "simaai/LFM2-1.2B-Autoround-Safetensors",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/simaai/LFM2-1.2B-Autoround-Safetensors

SGLang

How to use simaai/LFM2-1.2B-Autoround-Safetensors with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "simaai/LFM2-1.2B-Autoround-Safetensors" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "simaai/LFM2-1.2B-Autoround-Safetensors",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "simaai/LFM2-1.2B-Autoround-Safetensors" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "simaai/LFM2-1.2B-Autoround-Safetensors",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use simaai/LFM2-1.2B-Autoround-Safetensors with Docker Model Runner:
```
docker model run hf.co/simaai/LFM2-1.2B-Autoround-Safetensors
```

LFM2-1.2B Autoround

Pre-LLiMa Hugging Face checkpoint for Sima.ai deployment experiments. It is based on LiquidAI/LFM2-1.2B at source revision 933cee00d754fb3bfe06c644c0cb95453f2d8bb2 and remains subject to that model's MIT license and limitations.

Quantization

Component	Method	Weight format	Details
Decoder Linear layers	AutoRound	symmetric INT4, G256	92 targets; `lm_head` excluded
`lm_head`	GPTQ	symmetric INT4, G256	static act-order, block size 128, dampening 0.01
Mixed-precision exceptions	—	source dtype	Non-Linear parameters remain at source precision

Calibration used HuggingFaceH4/ultrachat_200k, train_sft[:512], rendered with the source chat template. Token IDs were deterministically concatenated and packed into 512 full 1024-token spans, with no shuffle. AutoRound used 200 iterations and batch size 1.

Evaluation

Full wikitext-2-raw-v1 evaluation used the wikitext lm-eval task, no example limit, batch size 1, CUDA, and the same evaluator for both checkpoints on 2026-07-19.

Checkpoint	Word perplexity	Status
`LiquidAI/LFM2-1.2B` source	34.597716	Full run
This UltraChat checkpoint	38.401202	Full run
Absolute degradation	3.803485	Lower is better
Relative degradation	10.993%	`100 * (quantized / source - 1)`

UltraChat is the selected calibration corpus because the intended deployment prioritizes instruction-following data. Finite quantization scales and a Transformers chat-generation smoke test both passed.

Reproduction

This directory contains the exact quantize.py, recipe.yaml, and versions.txt.

python quantize.py \
  --model-path /project/mlasw/share/huggingface/models--LiquidAI--LFM2-1.2B \
  --output-dir /path/to/output

Environment

Exact Python, CUDA, Torch, Transformers, llm-compressor, AutoRound, and compressed-tensors versions are recorded in versions.txt.

Deployment

This is the pre-LLiMa quantized Hugging Face artifact. Compile it separately for the target Sima.ai platform and keep compiler output separate. No upload is authorized as part of the current local batch.

Limitations

WikiText perplexity measures language-model likelihood and does not directly measure instruction following. Quantization quality can vary with language, domain, prompt format, context length, safety requirements, and runtime. Validate the intended deployment workload independently.