Instructions to use maum-ai/Llama-3-MAAL-8B-Instruct-v0.1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use maum-ai/Llama-3-MAAL-8B-Instruct-v0.1 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="maum-ai/Llama-3-MAAL-8B-Instruct-v0.1")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("maum-ai/Llama-3-MAAL-8B-Instruct-v0.1")
model = AutoModelForCausalLM.from_pretrained("maum-ai/Llama-3-MAAL-8B-Instruct-v0.1", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use maum-ai/Llama-3-MAAL-8B-Instruct-v0.1 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "maum-ai/Llama-3-MAAL-8B-Instruct-v0.1"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "maum-ai/Llama-3-MAAL-8B-Instruct-v0.1",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/maum-ai/Llama-3-MAAL-8B-Instruct-v0.1

SGLang

How to use maum-ai/Llama-3-MAAL-8B-Instruct-v0.1 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "maum-ai/Llama-3-MAAL-8B-Instruct-v0.1" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "maum-ai/Llama-3-MAAL-8B-Instruct-v0.1",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "maum-ai/Llama-3-MAAL-8B-Instruct-v0.1" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "maum-ai/Llama-3-MAAL-8B-Instruct-v0.1",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use maum-ai/Llama-3-MAAL-8B-Instruct-v0.1 with Docker Model Runner:
```
docker model run hf.co/maum-ai/Llama-3-MAAL-8B-Instruct-v0.1
```
Browse Quantizations to use this model in llama.cpp, Ollama, LM Studio, or any compatible app.

Llama-3-MAAL-8B-Instruct-v0.1

we release MAAL, Multilingual Adaptive Augmentation Language-model which comprises a groundbreaking fusion of multilingual capabilities and adaptive augmentation techniques.

Developed by: maum.ai Brain NLP. Jaeyoon Jung, Jinjoo Lee, Yongjae Lee, Dongjun Lee, Woosung Joo
Language(s) (NLP): Korean, English (currently, bilingual)

Model Description

Version 0.1 uses cross-lingual training to transfer instruction-following capabilities from English to Korean.

We Trained this model on an 8 H100-80G for 1 day with cross-lingual training dataset
we recommend using the fixed system prompt for the model unless you fine-tune it

너는 마음에이아이의 챗봇 MAAL이다. 고객의 질문에 친절하게 답하여라.

sample inference code (GPU)

import transformers
import torch

model_id = "maum-ai/Llama-3-MAAL-8B-Instruct-v0.1"
model = transformers.AutoModelForCausalLM.from_pretrained(model_id).to("cuda")
tokenizer = transformers.AutoTokenizer.from_pretrained(model_id)
streamer = transformers.TextStreamer(tokenizer, skip_prompt=True, skip_special_tokens=True)

# we recommend using the fixed prompt for the model unless you fine-tune it
prompt = "너는 마음에이아이의 챗봇 MAAL이다. 고객의 질문에 친절하게 답하여라."
instruction = "사과 한 박스에는 사과가 30개 들어있는데, 처음에는 사과 3박스가 있었고, 내가 사과 5개를 먹었어. 남은 사과는 총 몇개야?"

messages = [
    {"role": "system", "content": f"{prompt}"},
    {"role": "user", "content": f"{instruction}"}
    ]

inputs = tokenizer.apply_chat_template(
    messages,
    tokenize=True,
    return_tensors='pt').to("cuda")
outputs = model.generate(inputs, streamer=streamer, max_new_tokens=1024, pad_token_id=tokenizer.eos_token_id)

Evaluation Results

As the main goal of version 0.1 is to transfer instruction-following capabilities from English to Korean without utilizing continuous pre-training, etc., we select LogicKor as our evaluation method to assess the Korean instruction skills.

We compare our model with a similar parameter model (less than 13B) that has been fine-tuned on the Korean dataset. * denotes our self-report result.

Model	single-turn(↑)	multi-turn(↑)	average(↑)
maum-ai/Llama-3-MAAL-8B-Instruct-v0.1*	5.80	4.66	5.23
maywell/Synatra-kiqu-10.7B	5.71	4.73	5.22
yanolja/EEVE-Korean-Instruct-10.8B-v1.0	5.78	3.92	4.85
nlpai-lab/KULLM3	4.61	4.83	4.72
MLP-KTLim/llama3-Bllossom*	2.11	1.57	1.84

Limitations

Due to this model being trained on a small dataset, it has several limitations.

Hard to generate diverse Korean texts
lack of Korean knowledge & Culture (localization)
Not work with Image inputs and video inputs

Todo

we will solve these limitations one by one by upgrading this model like as...

Enhance the Korean generation through Vocabulary Expansion & Continuous pre-training. (more Korean corpus!)
Localize with cultural adaptation method and additional Korean knowledge data. similar idea
Develop a Vision Language Model that can handle both video and image inputs. similar idea

Downloads last month: 24

Safetensors

Model size

8B params

Tensor type

BF16

Model tree for maum-ai/Llama-3-MAAL-8B-Instruct-v0.1

Base model

meta-llama/Meta-Llama-3-8B-Instruct

Finetuned

(1194)

this model

Merges

2 models

Quantizations

4 models