SOC Analyst Tool-Use Model

Fine-tuned language model for structured SOC analyst tool calling, trained with QLoRA on SOC-AgentBench.

Published by Aria AI Security Research Team.

Model Details

Property Value
Base Model Qwen/Qwen2.5-1.5B-Instruct
Method QLoRA (r=16, alpha=32)
Training Data SOC-AgentBench train split (84 episodes)
Last train loss 0.454 (ZeroGPU LoRA, 3 epochs)
Max Length 2048 tokens
Task Tool-calling SOC investigation

Supported Tools

  • search_logs(query, start_time, end_time)
  • get_host_context(hostname)
  • get_user_context(username)
  • lookup_mitre_technique(technique_id)
  • enrich_ip(ip)
  • build_timeline(events)
  • submit_incident_report(report)

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer

model_id = "alirezaaminzadeh/soc-analyst-tool-use"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, device_map="auto")

messages = [
    {"role": "system", "content": "You are a SOC analyst agent..."},
    {"role": "user", "content": "Investigate: Suspicious LSASS memory access on WS-104"},
]
text = tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)
inputs = tokenizer(text, return_tensors="pt").to(model.device)
outputs = model.generate(**inputs, max_new_tokens=1024)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

Benchmark Results

Metric Score
Technique Macro F1 0.68
Evidence Precision 0.72
Root Cause Accuracy 0.58
Tool Success Rate 0.88

Related

License

Apache 2.0

Downloads last month
26
Safetensors
Model size
2B params
Tensor type
BF16
·
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for alirezaaminzadeh/soc-analyst-tool-use

Finetuned
(2031)
this model

Spaces using alirezaaminzadeh/soc-analyst-tool-use 5

Collection including alirezaaminzadeh/soc-analyst-tool-use

Article mentioning alirezaaminzadeh/soc-analyst-tool-use