source/docs/barunaction-model-card.md

BarunAction-35M candidate-v2 model card

BarunAction-35M is a 35,072,768-parameter local compiler for typed personal-action proposals. It takes a request, explicit tool schemas, JSON context, and a reference time, then generates one Action IR v1 JSON object. The reference package hash-verifies the checkpoint, validates inputs and outputs deterministically, and never invokes a real tool.

Candidate-v2 is useful as a compact structured-generation research model. It is not an autonomous assistant, safety classifier, permission system, or production-ready executor.

Model details

Field	Value
Model	BarunAction-35M candidate-v2
Base	harrrshall/BarunLM-35M revision ef3e483a9fd7d906ecf2a7929babeffaf82d1d16
Parameters	35,072,768
Layers / hidden size	12 / 448
Attention heads / KV heads	7 / 1
Attention pattern	three local layers, then one global layer
Local window	256 tokens
Feed-forward width	1,228
Vocabulary	16,384 byte-level BPE tokens
Maximum context	2,048 tokens
Output	Action IR v1
License	Apache-2.0 code and released weights

The architecture also uses partial RoPE, QK normalization, gated attention outputs, tied input/output embeddings, and a learned residual selector every four layers. The complete model configuration is barun_config.json.

Intended use

Appropriate uses include:

• research on compact typed-action generation;
• local proposal compilation for explicitly supplied schemas;
• deterministic parser, schema, and failure-mode experiments; and
• sandboxed demonstrations where external application code owns every permission and side effect.

Do not use the model to directly send messages, edit calendars, modify devices, or call production services. A real integration needs independent authentication, authorization, confirmation, identity, rate limiting, audit logging, and tool-specific validation. Valid JSON is not permission to act.

The package's simulator is in-memory only. It has no calendar, mail, map, contacts, settings, device-control, or network integration.

Training summary

Candidate-v2 is a fresh one-epoch, full-parameter, response-only SFT run from the pinned base. It saw 7,937 Mobile Actions-derived examples exactly once and completed 126 optimizer steps with seed 17, batch size 63, AdamW, peak learning rate 1e-4, 12 warmup steps, cosine decay, and BF16 CUDA autocast. Prompt and padding labels were masked; target tokens and one EOS token were supervised. No row was truncated or dropped.

The run used no synthetic teacher data, distillation, preference pairs, adapters, DPO, or RL. The exact method, data hashes, optimizer settings, and reproduction path are in docs/training.md and training/candidate-v2-recipe.json.

Evaluation

The released float checkpoint was selected and scored on a grouped 756-row Mobile Actions development population derived only from public source-training rows. All rows are CALL examples over one seven-tool schema family.

Model	Exact parameters	Parse valid	Schema valid	Strict AST exact
BarunAction-35M candidate-v2	35,072,768	756/756	755/756	602/756 (79.63%)
Qwen2.5-0.5B-Instruct matched baseline	494,032,768	755/756	754/756	663/756 (87.70%)

Qwen is 14.09 times larger. BarunAction retains 90.80% of its exact-match rate, but Qwen leads by 61 rows, or 8.07 percentage points. Paired outcomes were 19 BarunAction-only wins, 80 Qwen-only wins, 583 both correct, and 74 both wrong.

The Qwen model is Qwen/Qwen2.5-0.5B-Instruct revision 7ae557604adf67be50417f59c2c2f167def9a775: exactly 494,032,768 unique BF16 parameters across 290 tensors, approximately 0.494B—not 500B and not 500 billion. The comparison matched semantic examples, one training pass, effective batch 63, 126 updates, seed 17, final-checkpoint scoring, and greedy decoding. It did not match pretraining, prior instruction tuning, tokenizer control tokens, learning rates, architecture, or prior development-selection budget.

The hypothesis that candidate-v2 would beat this larger matched baseline failed. The useful result is compactness with substantial retained task accuracy, not an accuracy victory.

Full definitions, per-scenario results, raw continuations, and comparison limitations are in docs/benchmark.md, docs/evaluation.md, and benchmarks/evidence/.

Evaluation boundary

The reported score is reused development evidence because the population selected candidate-v2. The 961 official Mobile Actions evaluation rows were never read, parsed, materialized, trained on, or scored. No official, hidden, or independent test result is claimed.

Because every development target is CALL, the benchmark has zero denominator for false-action, abstention, clarification, confirmation, unsafe-request, ambiguity, and out-of-scope behavior. A zero-valued aggregate field with a zero denominator is not a safety measurement.

The result does not establish competence on renamed or unseen schemas, contextual revisions, disfluencies, prompt injection, broad function calling, or real-world action execution. Calendar datetime and map argument fidelity are the weakest observed development slices.

Input contract

The runtime contract versions are:

• prompt: barunaction-local-prompt-v1;
• tool declaration: barunaction-tool-schema-v1; and
• result: barunaction-inference-result-v1.

Every request must provide:

1. non-empty request text;
2. one or more strict, uniquely named tool declarations;
3. a JSON object for context, including {} when empty; and
4. a timezone-aware ISO-8601 now value.

Tool declarations specify name, description, typed arguments, required names, whether extra arguments are allowed, and whether the tool is side-effecting. The validator supports strict JSON scalars, enums, arrays, and nested objects. Duplicate or unknown schema fields, undeclared required arguments, non-finite numbers, naive timestamps, and reserved role-token text fail closed.

Empty context is omitted from the rendered prompt. Non-empty context is canonicalized to strict JSON. The runtime accepts exactly one generated JSON object and performs no Markdown stripping, substring extraction, coercion, defaulting, or repair.

Output contract

The four syntactically supported decisions are CALL, CONFIRM, CLARIFY, and ABSTAIN; call modes are SINGLE, SERIAL, and PARALLEL. For example:

{
  "calls": [
    {
      "args": {"query": "Cubbon Park, Bengaluru"},
      "tool": "show_map"
    }
  ],
  "decision": "CALL",
  "mode": "SINGLE"
}

The released model was supervised only on CALL, so the other decisions are runtime schema capabilities rather than demonstrated model behavior. Every successfully parsed result still sets policy.execution_permitted to false.

Quick start

python -m venv .venv
source .venv/bin/activate
pip install -e .

barunaction download --output ./models/BarunAction-35M
barunaction verify --checkpoint ./models/BarunAction-35M

barunaction infer \
  --checkpoint ./models/BarunAction-35M \
  --tools examples/barunaction_tools.example.json \
  --context examples/barunaction_empty_context.example.json \
  --now 2026-08-03T20:00:00+05:30 \
  --request "Turn on the flashlight" \
  --device cpu

A successful proposal exits with status 0 but does not authorize execution. Input, checkpoint, generation, parse, or schema failure returns structured JSON and exits with status 2.

Python:

import json

from barunaction import BarunActionCompiler, download_candidate_checkpoint

downloaded = download_candidate_checkpoint("./models/BarunAction-35M")
compiler = BarunActionCompiler(downloaded.path, device="cpu")

with open("examples/barunaction_tools.example.json", encoding="utf-8") as source:
    tools = json.load(source)

result = compiler.infer(
    request="Turn on the flashlight",
    tool_schemas=tools,
    context={},
    now="2026-08-03T20:00:00+05:30",
)
print(result.to_dict())

The download and loader both verify the candidate's required hashes.

Sandboxed demonstration

The weight-free demo exercises strict validation and the in-memory policy boundary:

barunaction demo

It loads no model, requires no network, and performs no external side effect. Model quality is not measured by this demo.

Checkpoint identity

File	Bytes	SHA-256
barun_config.json	448	9b3a1d71baa95a198744d250f9629231738d942570b8685c44307fd83dd33565
model.safetensors	140,304,464	fdb95ccf58a095e0d321be998924318b35ee59a334f6dd97d8726d2cf80021d3
tokenizer.json	1,136,031	70ded9605fccd09c2340ca7e225361eab0ae8b4dbbb0d6e26343ab5183979db6
checkpoint_manifest.json	549	c743ab7c4d33ae75c6b0aa4547458a961b92766da8fcf85fd148fda2ebb5530a

The float package also carries LICENSE, NOTICE, and MODEL_CARD.md. Exact immutable artifact identities and verification commands are in docs/barunaction-retrieval.md.

Int8 derivative

A CPU dynamic-int8 derivative is retained only for Darwin ARM64, PyTorch 2.13.0, and QNNPACK. It quantizes 87 internal Linear modules; embeddings, norms, attention operations, KV state, and the tied output head remain floating point. It scored 607/756 on the same development IDs and passed a frozen retention check, but the small timing smoke was slower than float. No general speed, portability, or int8-accuracy-improvement claim is made. See docs/int8-quantization.md.

Data and license

Candidate-v2 was post-trained on google/mobile-actions revision e920309bc2acbc2e99a5e3201cf37df2b9fd9151, licensed CC BY 4.0. Training data is not bundled with the weights and retains its upstream terms. Code and released model weights are Apache-2.0. See NOTICE and the data card.

Citation

Use CITATION.cff or:

@software{harrrshall2026barunaction35m,
  author  = {Harrrshall},
  title   = {BarunAction-35M: A Compact Typed Personal-Action Compiler},
  year    = {2026},
  version = {candidate-v2},
  url     = {https://github.com/harrrshall/barunaction-35m},
  license = {Apache-2.0}
}