Hugging Face's logo

principled-intelligence
/
scope-guard-v2-4B-q-2609

Text Generation
Transformers
Safetensors
English
qwen3_5
image-text-to-text
scope-guard
guardrail
safety-classifier
text-classification
conversational
🇪🇺 Region: EU
Model card Files
xet
 Community
2

ScopeGuard-2: A Governance SLM for Scope Classification

scopeguard-2-banner

ScopeGuard-2 is the second generation of our small language model (SLM) for scope classification in AI governance. Like its predecessor, it is trained to make consistent decisions based on service policies, with the low latency needed to check every incoming request.

We retain the previous generation's focus on a small, specialized model. This is a deliberate design choice: keeping inference costs and latency low makes ScopeGuard practical to deploy as part of a production application's request handling. ScopeGuard-2 builds on that approach with improved classification accuracy and faster inference.

Table of contents

Resources

What ScopeGuard-2 does

Given a user request, ScopeGuard-2 decides whether it falls within the AI service's scope or should be declined, redirected, or passed to a person. You define that scope through the service's supported use cases, allowed topics, and interaction rules.

ScopeGuard-2 can be used as an early gate in enterprise AI deployments to ensure requests are handled only when they match the intended purpose of the system. The new release, also identifies requests that call for a predefined reply or human oversight.

Given a conversation, ScopeGuard-2 assigns the last user message to one of seven classes, each mapped to a different action:

Class Meaning Typical action
Directly Supported Clearly within the service's stated functionalities. Answer.
Potentially Supported Adjacent to the stated functionalities, and no rule applies. Answer with care.
Predefined Answer Matches a scenario with a pre-written response. Send the fixed reply.
Human Oversight Meets an escalation criterion, or needs human judgment. Route to a person.
Out of Scope Outside the service's remit. Off-topic, not harmful. Decline politely.
Restricted Explicitly forbidden by the service's constraints, or harmful. Refuse.
Chit Chat Greetings, thanks, small talk. Reply briefly.

You can also request a one-sentence rationale, passages from your description that support the decision, and a suggested reply in the user's language.

scopeguard-2-overview-transparent

Show me the code

Install 

pip install "orbitals[scope-guard-vllm]>=0.6.1"

# Or, to use transformers as a backend
# pip install "orbitals[scope-guard-hf]>=0.6.1"

Use 

from orbitals.scope_guard_v2 import ScopeGuardV2

if __name__ == "__main__":
  sg = ScopeGuardV2(
      backend="vllm",
      model="principled-intelligence/scope-guard-v2-4B-q-2609",
  )
  
  ai_service_description = """
  You are a virtual assistant for a parcel delivery service.
  You can only answer questions about package tracking.
  Never respond to requests for refunds.
  """
  
  user_query = "If the package hasn't arrived by tomorrow, can I get my money back?"
  result = sg.validate(
      user_query,
      ai_service_description=ai_service_description,
      output_fields=["reasoning", "scope_class"],
  )
  
  print(f"Scope: {result.scope_class.value}")
  print(f"Reasoning: {result.reasoning}")
  
  # Scope: Restricted
  # Reasoning: The user is asking about getting a refund, which is explicitly forbidden by the service constraints.

output_fields accepts any subset of evidences, reasoning, scope_class, and suggested_response. The scope_class field is always included. When generating output, requesting scope_class alone gives the lowest latency and highest accuracy in our evaluation (see Inference speed and deployment).

Structured AI service description (recommended)

The structured description has dedicated fields for constraints, escalation criteria and predefined responses, which map directly onto Restricted, Human Oversight and Predefined Answer. ScopeGuard reads them more reliably than the same rules buried in free text.

from orbitals.scope_guard_v2 import ScopeGuardV2
from orbitals.types import AIServiceDescriptionV2, PredefinedResponse

if __name__ == "__main__":
  sg = ScopeGuardV2(
      backend="vllm",
      model="principled-intelligence/scope-guard-v2-4B-q-2609",
      output_fields=["scope_class"],
  )
  
  ai_service_description = AIServiceDescriptionV2(
      identity_role="PackAssist, a virtual assistant that helps customers track their parcels.",
      context="A parcel-delivery company. Users are customers awaiting or sending parcels.",
      functionalities=[
          "Retrieve tracking updates",
          "Explain tracking events and estimated delivery windows",
      ],
      constraints=["Never process or promise refunds"],
      escalation_criteria=["The customer reports a lost or stolen parcel"],
      predefined_responses=[
          PredefinedResponse(
              trigger="Asks for the customer service phone number",
              response="You can reach us at +1 555 0100, Mon-Fri 9-18.",
          ),
      ],
  )
  
  conversation = [
      {"role": "user", "content": "I ordered a package, tracking number 1234567890"},
      {"role": "assistant", "content": "It is in transit. What would you like to know?"},
      {"role": "user", "content": "If it doesn't arrive tomorrow, can I get a refund?"},
  ]
  
  result = sg.validate(conversation, ai_service_description=ai_service_description)
  print(f"Scope: {result.scope_class.value}")
  
  # Scope: Restricted

For conversations with multiple turns, ScopeGuard classifies the last user message using earlier turns as context. batch_validate classifies many conversations in one call, and AsyncScopeGuardV2 offers the same methods for async code.

If vLLM is unavailable on your machine, use the Transformers backend. It is slower, so prefer vLLM for serving:

sg = ScopeGuardV2(
    backend="hf",
    model="principled-intelligence/scope-guard-v2-4B-q-2609",
)

Probabilities instead of labels

classify skips generation and returns a probability for each class from a single forward pass. It is the fastest way to run ScopeGuard and lets you route requests using a confidence threshold:

result = sg.classify(conversation, ai_service_description=ai_service_description)
print(result.scope_class.value, result.confidence)
print(result.probabilities)

Serve it

orbitals can start vLLM and an HTTP API in front of it:

pip install "orbitals[scope-guard-serve]>=0.6.1"
orbitals scope-guard-v2 serve principled-intelligence/scope-guard-v2-4B-q-2609 --port 8000

Then call it from Python with ScopeGuardV2(backend="api", api_url="http://localhost:8000"), or directly over HTTP. See the Orbitals documentation for the available endpoints and configuration options.

ScopeGuard model family

What's new in ScopeGuard-2

  • Seven classes instead of five. Predefined Answer and Human Oversight identify requests that your application should handle with a fixed reply or pass to a person.
  • Conversation context. The last user message is classified with the rest of the conversation as context.
  • A richer service description, with constraints, escalation criteria and predefined responses as separate fields.
  • Choose what the model returns: the class alone or with a rationale, supporting evidence, and a suggested reply. You can also request a probability for each class.
  • Faster and more accurate. Against scope-guard-4B-q-2601, 1.3 percentage points higher mean balanced accuracy on the six benchmarks containing both safe and unsafe prompts, with about one-seventh the median latency (34 vs 241 ms).

How we evaluated ScopeGuard

We compared both ScopeGuard-2 models against:

  • General-purpose models: GPT-6.1 Sol, Claude Opus 5.5, Gemini 4 Argon, and Jev 1.13.0
  • Safety classifiers: NVIDIA Nemotron-3.5-Content-Safety and Shieldstral-1.0-3B
  • Our previous generation: scope-guard-4B-q-2601, scope-guard-4B-g-2601, and ScopeGuard Pro

We used unsafe-eval, our open-source framework for reproducible AI safety evaluation. Each model labels each prompt as safe or unsafe. For ScopeGuard, Restricted and Human Oversight count as unsafe. Every other class, including Out of Scope, counts as safe.

We report balanced accuracy, the mean of the percentage of unsafe prompts caught and the percentage of safe prompts allowed through. This gives safe and unsafe prompts equal weight regardless of their proportions in each benchmark.

Tasks

  1. General safety classification Detect whether a user request violates generic safety policies (toxicity, violence, illegal activity, abuse).

  2. Custom safety classification Detect whether a user request violates policies defined by the service owner, which differ across products and organizations.

Results

Balanced accuracy (%) on six public benchmarks. Aegis 2.0, OpenAI Moderation, ToxicChat, and WildGuardMix cover general safety. CoSApien and DynaGuardrail test whether the model follows explicit policies supplied as part of the input.

Benchmark scope-guard-v2-pro-2609 scope-guard-v2-4B-q-2609 ScopeGuard Pro scope-guard-4B-q-2601 scope-guard-4B-g-2601 GPT-6.1 Sol Claude Opus 5.5 Gemini 4 Argon Nemotron-3.5-Content-Safety Shieldstral-1.0-3B Jev 1.13.0
CoSApien 94.6 93.7 92.2 89.5 85.6 93.1 91.9 91.7 88.4 81.5 93.7
ToxicChat 92.2 91.4 92.0 90.9 89.4 79.7 84.3 77.0 90.3 90.7 85.2
DynaGuardrail 90.6 90.8 87.2 89.1 87.1 90.3 91.2 90.1 89.6 81.4 90.6
WildGuardMix 88.1 87.4 81.6 86.8 83.7 88.9 89.8 83.8 86.2 90.6 90.7
Aegis 2.0 80.8 80.8 85.1 84.4 84.9 83.3 84.0 84.5 84.1 83.6 84.0
OpenAI Moderation 83.4 79.0 81.8 74.5 78.6 85.7 84.4 85.2 74.1 73.2 84.0
Mean balanced accuracy 88.3 87.2 86.6 85.9 84.9 86.8 87.6 85.4 85.5 83.5 88.0

ScopeGuard-2 4B scores 87.2% on average, compared with 85.9% for the previous Qwen-based release. Its score is within half a point of Claude Opus 5.5 at 87.6% and above GPT-6.1 Sol at 86.8%, while its median response time is 34 ms compared with 2,976 ms and 2,786 ms. ScopeGuard-2 Pro and Jev have the highest average scores in this comparison, at 88.3% and 88.0%, respectively.

On custom policies, ScopeGuard-2 4B scores 93.7% on CoSApien and 90.8% on DynaGuardrail. ScopeGuard-2 Pro scores 94.6% and 90.6%, respectively. These results are comparable to the best general-purpose models in this evaluation.

On SimpleSafetyTests and HarmBench, which contain only unsafe prompts and are excluded from the six-benchmark mean above, ScopeGuard-2 4B catches 99.0% and 98.4% of prompts, respectively.

Inference speed and deployment

A guardrail adds latency to every user interaction. We measured ScopeGuard-2 4B locally on a single NVIDIA GeForce RTX 5090 (32 GB), processing one request at a time. Hosted API response times include the network round trip and depend on network conditions and serving load.

The table reports median response times across all timed requests on the six benchmarks above, including CoSApien and DynaGuardrail. ScopeGuard-2 results use classification mode.

Model Median response time
scope-guard-v2-pro-2609 118 ms
scope-guard-v2-4B-q-2609 34 ms
ScopeGuard Pro 450 ms
scope-guard-4B-q-2601 241 ms
scope-guard-4B-g-2601 314 ms
GPT-6.1 Sol 2,786 ms
Claude Opus 5.5 2,976 ms
Gemini 4 Argon 2,851 ms
Nemotron-3.5-Content-Safety 40 ms
Shieldstral-1.0-3B 11 ms
Jev 1.13.0 371 ms

Mean balanced accuracy versus median response time for ScopeGuard-2 and the other models in the comparison

Output configuration

Generating additional fields takes longer. For ScopeGuard-2 4B, generating the class alone takes 87 ms, compared with 34 ms in classification mode. Adding reasoning raises the median to 338 ms.

ScopeGuard-2 4B output configuration Mean balanced accuracy (%) Median latency
Classification mode (classify) 87.17 34 ms
Generate class only 87.16 87 ms
Generate reasoning and class 86.88 338 ms
Generate evidence, reasoning, and class 86.06 529 ms
Generate all fields, including a suggested reply 85.72 633 ms

These measurements use the same six benchmarks. Classification mode and generating the class alone have nearly identical accuracy. Requesting additional fields reduced accuracy in this evaluation, though explanations and supporting evidence can help you review decisions.

Intended use

ScopeGuard is intended for:

  • Scope enforcement for customer-facing assistants: answer, decline, send a fixed reply, or hand off to a person, depending on the class
  • Inline guardrailing for agentic systems (pre-checks before tool execution)
  • Enterprise governance layers where behavior must follow explicit boundaries and policies
  • Analytics and routing pipelines, where the class and its rationale support monitoring and reporting

Limitations

  • ScopeGuard-2 is designed for governance decisions, not for open-ended generation.
  • Training data is predominantly English. suggested_response follows the user's language, but classification quality in other languages has not been measured for ScopeGuard-2.
  • Performance on safety benchmarks can be sensitive to prompting and evaluation conditions; strong results on one benchmark may not transfer automatically to all setups.
Using ScopeGuard-2 without orbitals

orbitals builds the prompt, restricts decoding to the requested JSON keys and parses the result. Without the library you do all three yourself. The prompt has to match the training prompt byte for byte, because a different one lowers accuracy and raises no error.

The code below renders the prompt with the model's tokenizer and sends it to the completions endpoint of a vLLM server. This is the same request the orbitals API backend makes.

pip install vllm openai
vllm serve principled-intelligence/scope-guard-v2-4B-q-2609 --port 8000

import json

from huggingface_hub import hf_hub_download
from openai import OpenAI
from transformers import AutoTokenizer

MODEL = "principled-intelligence/scope-guard-v2-4B-q-2609"
SYSTEM_PROMPT = open(hf_hub_download(MODEL, "system_prompt.txt"), encoding="utf-8").read()
tokenizer = AutoTokenizer.from_pretrained(MODEL)
client = OpenAI(base_url="http://localhost:8000/v1", api_key="unused")

CLASSES = [
    "Directly Supported",
    "Potentially Supported",
    "Predefined Answer",
    "Human Oversight",
    "Out of Scope",
    "Restricted",
    "Chit Chat",
]

# The model emits the keys in this order, whatever order you request them in.
FIELD_SCHEMAS = {
    "evidences": {"anyOf": [{"type": "array", "items": {"type": "string"}}, {"type": "null"}]},
    "reasoning": {"type": "string"},
    "scope_class": {"enum": CLASSES},
    "suggested_response": {"anyOf": [{"type": "string"}, {"type": "null"}]},
}


def render_prompt(description: str, conversation: list[dict], fields: list[str]) -> str:
    *history, last = conversation
    dump = "".join(f"{m['role'].upper()}:\n{m['content']}\n\n" for m in history)
    dump += f"LAST MESSAGE (USER):\n{last['content']}\n"
    user_turn = (
        f"**START OF THE AI SERVICE DESCRIPTION**\n\n{description}\n\n"
        "**END OF THE AI SERVICE DESCRIPTION**\n\n\n"
        f"**START OF THE CONVERSATION DUMP**\n\n{dump}\n\n"
        "**END OF THE CONVERSATION DUMP**\n\n\n"
        f"**REQUESTED OUTPUT FIELDS**\n\n{json.dumps(fields)}"
    )
    messages = [
        {"role": "system", "content": SYSTEM_PROMPT},
        {"role": "user", "content": user_turn},
    ]
    return tokenizer.apply_chat_template(
        messages, tokenize=False, add_generation_prompt=True, enable_thinking=False
    )


def validate(description: str, conversation: list[dict], output_fields=("scope_class",)) -> dict:
    fields = [f for f in FIELD_SCHEMAS if f in output_fields or f == "scope_class"]
    schema = {
        "type": "object",
        "properties": {f: FIELD_SCHEMAS[f] for f in fields},
        "required": fields,
        "additionalProperties": False,
    }
    completion = client.completions.create(
        model=MODEL,
        prompt=render_prompt(description, conversation, fields),
        temperature=0,
        max_tokens=1024,
        extra_body={"structured_outputs": {"json": schema}},
    )
    return json.loads(completion.choices[0].text)

Here it is with the structured description from Show me the code:

description = json.dumps(
    {
        "identity_role": "PackAssist, a virtual assistant that helps customers track their parcels.",
        "context": "A parcel-delivery company. Users are customers awaiting or sending parcels.",
        "knowledge_scope": None,
        "functionalities": [
            "Retrieve tracking updates",
            "Explain tracking events and estimated delivery windows",
        ],
        "constraints": ["Never process or promise refunds"],
        "predefined_responses": [
            {
                "trigger": "Asks for the customer service phone number",
                "response": "You can reach us at +1 555 0100, Mon-Fri 9-18.",
            }
        ],
        "escalation_criteria": ["The customer reports a lost or stolen parcel"],
        "response_guidelines": None,
    },
    separators=(",", ":"),
    ensure_ascii=False,
)

conversation = [
    {"role": "user", "content": "I ordered a package, tracking number 1234567890"},
    {"role": "assistant", "content": "It is in transit. What would you like to know?"},
    {"role": "user", "content": "If it doesn't arrive tomorrow, can I get a refund?"},
]

print(validate(description, conversation, ["reasoning"]))

# {'reasoning': 'The user is asking about the possibility of receiving a refund, which directly
#  violates the constraint that the AI must never process or promise refunds.', 'scope_class': 'Restricted'}

Common pitfalls

  • A structured description must match AIServiceDescriptionV2.model_dump_json() exactly: all eight keys in order, null for unset ones, dumped as in the example.
  • The last message in the conversation must come from the user.
  • Set max_tokens. The completions endpoint defaults to 16 tokens and cuts the JSON short.
  • With transformers, pass eos_token_id=tokenizer.eos_token_id to generate, or the model keeps writing after the JSON.

Class probabilities

classify requests scope_class alone, appends {"scope_class": " to the prompt and reads the log-probabilities of the next token. The seven class names each start with a different token, so normalizing over those tokens gives one probability per class:

import math

PREFIX = '{"scope_class": "'
prefix_len = len(tokenizer.encode(PREFIX, add_special_tokens=False))
first_tokens = {
    c: tokenizer.decode(tokenizer.encode(PREFIX + c, add_special_tokens=False)[prefix_len])
    for c in CLASSES
}


def classify(description: str, conversation: list[dict]) -> dict[str, float]:
    completion = client.completions.create(
        model=MODEL,
        prompt=render_prompt(description, conversation, ["scope_class"]) + PREFIX,
        temperature=0,
        max_tokens=1,
        logprobs=20,
    )
    top = completion.choices[0].logprobs.top_logprobs[0]
    scores = {c: math.exp(top.get(t, -math.inf)) for c, t in first_tokens.items()}
    total = sum(scores.values())
    return {c: s / total for c, s in scores.items()}


print({c: round(p, 4) for c, p in classify(description, conversation).items()})

# {'Directly Supported': 0.0066, 'Potentially Supported': 0.0046, 'Predefined Answer': 0.004,
#  'Human Oversight': 0.0003, 'Out of Scope': 0.0005, 'Restricted': 0.9839, 'Chit Chat': 0.0001}

Cite ScopeGuard

If you use ScopeGuard in academic work or evaluations, please cite it. Our technical report is coming soon.

Integration and contact

ScopeGuard-2 is released under the terms in LICENSE. If you are thinking about integrating ScopeGuard into your pipeline, or want to learn more about ScopeGuard-2 Pro, write to orbitals@principled-intelligence.com.

Principled Intelligence

Downloads last month
149
Safetensors
Model size
5B params
Tensor type
BF16
·
Inference Providers NEW
Text Generation
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for principled-intelligence/scope-guard-v2-4B-q-2609

Base model

Qwen/Qwen3.5-4B-Base
Finetuned
Qwen/Qwen3.5-4B
Finetuned
unsloth/Qwen3.5-4B
Finetuned
(259)
this model