sk851 Claude Opus 5 (1M context) commited on
Commit
c84fc4b
·
1 Parent(s): fc01ca9

feat(agent): validate tool arguments against the schema

Browse files

HOSTED_TOOL_CONTRACTS was advisory. Dispatch went from the model's
arguments straight to capability.handler(**kwargs), so
`additionalProperties: False` enforced nothing and every undeclared
handler parameter stayed reachable from the agent.

That mattered most for valuation, which accepts base_growth_pct,
terminal_growth_pct, and beta without declaring them. An LLM that can set
its own growth rate, terminal growth, or beta can make a DCF agree with
whatever it already believed — the failure the locked-DCF rule in the
idea-loop design exists to prevent, and until now that rule held only as
long as a caller remembered to whitelist kwargs by hand.

Validation runs at the tool boundary: after alias normalisation, so
documented aliases still work, and before dispatch. It is keyed on the
capability name, not the tool name. A backgroundable capability is also
exposed as start_<capability>_task, which is not a HOSTED_TOOL_CONTRACTS
key, so keying on the tool name would leave all 24 task variants
validating vacuously — including start_valuation_task, i.e. the tilt
parameters would stay reachable through the one capability this check
exists to protect. _build_tool_definition hands the task variant the
capability's own input_schema, so the contract enforced here is the one
the model was shown. start_task's own `description` kwarg appears in no
contract, so a model passing it is rejected, which is what the task tool's
`additionalProperties: false` already advertises.

Rejections come back in the adapter's existing accepted:false shape with a
modelHint, matching how tool misuse is already reported, rather than
raising.

Only the agent surface is constrained. Routes, the service, and a future
controller keep the full signature by design — a deliberate internal
caller is trusted to choose inputs; a model is not.

The validator is hand-rolled rather than pulling in jsonschema, because
the contracts use a small fixed subset: type, enum, minimum, maximum,
minLength, maxItems, anyOf, items, required, additionalProperties. Three
details are load-bearing:

- It returns the arguments to dispatch, not just a verdict. An integral
float is accepted for an integer field, since models routinely emit
20.0, but it is also narrowed to int. Accepting without narrowing only
relocates the failure: relative_strength reaches `ordered[:top_n]` and
raises `TypeError: slice indices must be integers`, which
_classify_tool_error does not recognise, so the run aborts instead of
returning a tool error.
- A null-valued required argument is rejected. It satisfies "key present"
but not the handler, which goes on to call .upper() on it, producing the
same unclassified-abort path. Absent and null are different errors and
both are errors; a null *optional* stays allowed, as models commonly
send one for an omitted field.
- An anyOf property reports the matching branch's errors when exactly one
branch matches on type. `tickers` is anyOf[string, array], so an
over-long list otherwise produced "expected string or array, got list" —
unactionable, because the value is a list. The model would resend it,
hit the same error fingerprint, and burn the retry budget instead of
learning about maxItems.

Booleans are rejected for integer and string fields, since bool is an int
subclass in Python but a JSON boolean is neither.

Tests assert the property rather than a sample: every tool the default
agent is handed must resolve to a contract, so a future execution mode
with a new name shape cannot reintroduce the task-variant hole silently.
The hidden-parameter test runs against both the invoke and task surfaces.

Two parity tests keep the surface honest: declared properties must be
acceptable by the real handler, and any handler parameter absent from the
schema must be named as deliberately internal, so a new one fails rather
than quietly widening what the model could reach. Both build the registry
against the real service *and* the real chart opener — passing
fake_chart_opener measured a double that is wider than production
open_chart, which hid its undeclared `client` parameter. Measured drift is
3 capabilities of 34: valuation's three tilt inputs, open_chart's
in-process client, and the runtime-injected session_id.

One behaviour change: an empty consult question is now rejected by schema
validation as a retryable tool error instead of raising ValueError. The
contract already declared minLength 1; the raise predated validation. The
adapter's own guard still covers a whitespace-only question, which passes
minLength.

Not covered, deliberately: the argument-repair retry re-dispatches without
a second validation pass. It only rewrites the *value* of name/ticker and
adds no keys, and it now operates on the narrowed dict, so it cannot
introduce an undeclared or wrongly-typed argument.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>

src/TerraFin/agent/contracts/tool_contracts.py CHANGED
@@ -1,3 +1,4 @@
 
1
  from copy import deepcopy
2
  from typing import Any
3
 
@@ -464,3 +465,149 @@ def get_hosted_tool_contract(capability_name: str) -> dict[str, Any]:
464
  return deepcopy(HOSTED_TOOL_CONTRACTS[capability_name])
465
  except KeyError as exc:
466
  raise KeyError(f"No explicit hosted tool contract registered for capability '{capability_name}'.") from exc
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ from collections.abc import Mapping
2
  from copy import deepcopy
3
  from typing import Any
4
 
 
465
  return deepcopy(HOSTED_TOOL_CONTRACTS[capability_name])
466
  except KeyError as exc:
467
  raise KeyError(f"No explicit hosted tool contract registered for capability '{capability_name}'.") from exc
468
+
469
+
470
+ # ---------------------------------------------------------------------------
471
+ # Argument validation
472
+ #
473
+ # The schemas above were advisory until this existed: the dispatch path went
474
+ # straight from the model's arguments to `capability.handler(**kwargs)`, so
475
+ # `additionalProperties: False` enforced nothing. That mattered because several
476
+ # handlers accept parameters the schema deliberately hides — `valuation` takes
477
+ # `base_growth_pct`, `terminal_growth_pct`, and `beta`, which let a caller tilt
478
+ # a valuation into agreeing with whatever it already believed.
479
+ #
480
+ # Validating here makes the tool surface the schema says it is. Internal Python
481
+ # callers (routes, the service, a controller) are unaffected by design: they
482
+ # hold the full signature, and locking the *agent* is the point.
483
+ #
484
+ # Hand-rolled rather than pulling in `jsonschema`, because the contracts use a
485
+ # small fixed subset: type, enum, minimum, maximum, minLength, maxItems, anyOf,
486
+ # items, required, additionalProperties.
487
+ # ---------------------------------------------------------------------------
488
+
489
+ _TYPE_NAMES = {
490
+ "string": str,
491
+ "boolean": bool,
492
+ "array": list,
493
+ "object": dict,
494
+ }
495
+
496
+
497
+ def _type_error(value: Any, expected: str) -> str | None:
498
+ """None when `value` satisfies `expected`, else a human-readable reason."""
499
+ if expected == "integer":
500
+ # bool is an int subclass in Python; a JSON boolean is not an integer.
501
+ if isinstance(value, bool):
502
+ return "expected an integer, got a boolean"
503
+ if isinstance(value, int):
504
+ return None
505
+ # Models routinely emit 7.0 for an integer field; accept it losslessly.
506
+ if isinstance(value, float) and value.is_integer():
507
+ return None
508
+ return f"expected an integer, got {type(value).__name__}"
509
+ if expected == "number":
510
+ if isinstance(value, bool):
511
+ return "expected a number, got a boolean"
512
+ return None if isinstance(value, (int, float)) else f"expected a number, got {type(value).__name__}"
513
+ expected_type = _TYPE_NAMES.get(expected)
514
+ if expected_type is None:
515
+ return None # unknown type keyword: nothing to assert
516
+ if expected == "string" and isinstance(value, bool):
517
+ return "expected a string, got a boolean"
518
+ return None if isinstance(value, expected_type) else f"expected {expected}, got {type(value).__name__}"
519
+
520
+
521
+ def _check_value(name: str, value: Any, schema: dict[str, Any]) -> list[str]:
522
+ if "anyOf" in schema:
523
+ branches: list[dict[str, Any]] = schema["anyOf"]
524
+ reasons = [_check_value(name, value, branch) for branch in branches]
525
+ if any(not branch_errors for branch_errors in reasons):
526
+ return []
527
+ # Exactly one branch matching on type means the value's *shape* is right
528
+ # and something inside it is wrong — a bound, or an item's type. Report
529
+ # that branch's errors. The generic "expected string or array, got list"
530
+ # would restate the value's own type back at the model, which cannot act
531
+ # on it and would resend the same argument until the retry budget ends.
532
+ matched = [
533
+ branch_errors
534
+ for branch, branch_errors in zip(branches, reasons)
535
+ if branch.get("type") and not _type_error(value, branch["type"])
536
+ ]
537
+ if len(matched) == 1:
538
+ return matched[0]
539
+ allowed = " or ".join(str(branch.get("type", "?")) for branch in branches)
540
+ return [f"{name}: expected {allowed}, got {type(value).__name__}"]
541
+
542
+ errors: list[str] = []
543
+ declared_type = schema.get("type")
544
+ if declared_type:
545
+ reason = _type_error(value, declared_type)
546
+ if reason:
547
+ return [f"{name}: {reason}"]
548
+
549
+ if "enum" in schema and value not in schema["enum"]:
550
+ errors.append(f"{name}: {value!r} is not one of {schema['enum']}")
551
+ if "minimum" in schema and isinstance(value, (int, float)) and value < schema["minimum"]:
552
+ errors.append(f"{name}: {value} is below the minimum of {schema['minimum']}")
553
+ if "maximum" in schema and isinstance(value, (int, float)) and value > schema["maximum"]:
554
+ errors.append(f"{name}: {value} is above the maximum of {schema['maximum']}")
555
+ if "minLength" in schema and isinstance(value, str) and len(value) < schema["minLength"]:
556
+ errors.append(f"{name}: must be at least {schema['minLength']} character(s)")
557
+ if "maxItems" in schema and isinstance(value, list) and len(value) > schema["maxItems"]:
558
+ errors.append(f"{name}: at most {schema['maxItems']} item(s), got {len(value)}")
559
+ if isinstance(value, list) and isinstance(schema.get("items"), dict):
560
+ for index, item in enumerate(value):
561
+ errors.extend(_check_value(f"{name}[{index}]", item, schema["items"]))
562
+ return errors
563
+
564
+
565
+ def validate_tool_arguments(
566
+ tool_name: str, arguments: Mapping[str, Any]
567
+ ) -> tuple[list[str], dict[str, Any]]:
568
+ """Validate model-supplied arguments against a tool's declared input schema.
569
+
570
+ Returns `(problems, arguments_to_dispatch)`. `problems` is a list of
571
+ human-readable strings, empty when the arguments are acceptable. The second
572
+ element is the dict the caller should actually dispatch: integral floats are
573
+ narrowed to `int` for integer-typed fields, because accepting `20.0` without
574
+ narrowing it only moves the failure downstream — `relative_strength(top_n=20.0)`
575
+ reaches `ordered[:top_n]` and raises `TypeError: slice indices must be
576
+ integers`, which no error classifier recognises, so the whole run aborts.
577
+
578
+ Unknown tools validate vacuously — an unregistered tool is a different error,
579
+ reported elsewhere. Callers must pass the *capability* name: task variants are
580
+ exposed as `start_<capability>_task` but are handed the capability's own
581
+ schema, so they validate against the same contract.
582
+ """
583
+ contract = HOSTED_TOOL_CONTRACTS.get(tool_name)
584
+ if contract is None:
585
+ return [], dict(arguments)
586
+ schema = contract.get("input_schema") or {}
587
+ properties: dict[str, Any] = schema.get("properties") or {}
588
+
589
+ errors: list[str] = []
590
+ for required_name in schema.get("required") or []:
591
+ if required_name not in arguments:
592
+ errors.append(f"{required_name}: required")
593
+ elif arguments[required_name] is None:
594
+ # An explicit null satisfies "key present" but not the handler, which
595
+ # goes on to call `.upper()` on it. Optional nulls stay allowed below.
596
+ errors.append(f"{required_name}: required, but was null")
597
+
598
+ if schema.get("additionalProperties") is False:
599
+ unknown = sorted(set(arguments) - set(properties))
600
+ if unknown:
601
+ errors.append(
602
+ f"unknown argument(s) {unknown}; this tool accepts only {sorted(properties)}"
603
+ )
604
+
605
+ normalized = dict(arguments)
606
+ for name, value in arguments.items():
607
+ declared = properties.get(name)
608
+ if declared is None or value is None:
609
+ continue
610
+ errors.extend(_check_value(name, value, declared))
611
+ if declared.get("type") == "integer" and isinstance(value, float) and value.is_integer():
612
+ normalized[name] = int(value)
613
+ return errors, normalized
src/TerraFin/agent/tools/adapter.py CHANGED
@@ -4,7 +4,11 @@ from collections.abc import Mapping
4
  from typing import TYPE_CHECKING, Any
5
 
6
  from ..contracts.definitions import TerraFinAgentDefinition, is_internal_agent_definition
7
- from ..contracts.tool_contracts import HOSTED_TOOL_CONTRACT_VERSION, get_hosted_tool_contract
 
 
 
 
8
  from ..runtime.capability import TerraFinCapability
9
  from ..runtime.errors import TerraFinAgentApprovalRequiredError
10
  from ..runtime.hosted import TerraFinHostedAgentRuntime
@@ -19,9 +23,9 @@ from .normalize import (
19
  _repair_symbol_or_name,
20
  )
21
  from .types import (
22
- ToolExecutionMode,
23
  TerraFinToolDefinition,
24
  TerraFinToolInvocationResult,
 
25
  _ToolErrorDisposition,
26
  )
27
 
@@ -75,6 +79,13 @@ class TerraFinHostedToolAdapter:
75
  payload: dict[str, Any]
76
  task: TerraFinTaskRecord | None = None
77
  resolved_arguments = _normalize_common_alias_arguments(dict(arguments or {}))
 
 
 
 
 
 
 
78
  preflight_result = self._preflight_tool_misuse(tool, session_id, resolved_arguments)
79
  if preflight_result is not None:
80
  return preflight_result
@@ -275,6 +286,71 @@ class TerraFinHostedToolAdapter:
275
  },
276
  )
277
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
278
  def _preflight_tool_misuse(
279
  self,
280
  tool: TerraFinToolDefinition,
 
4
  from typing import TYPE_CHECKING, Any
5
 
6
  from ..contracts.definitions import TerraFinAgentDefinition, is_internal_agent_definition
7
+ from ..contracts.tool_contracts import (
8
+ HOSTED_TOOL_CONTRACT_VERSION,
9
+ get_hosted_tool_contract,
10
+ validate_tool_arguments,
11
+ )
12
  from ..runtime.capability import TerraFinCapability
13
  from ..runtime.errors import TerraFinAgentApprovalRequiredError
14
  from ..runtime.hosted import TerraFinHostedAgentRuntime
 
23
  _repair_symbol_or_name,
24
  )
25
  from .types import (
 
26
  TerraFinToolDefinition,
27
  TerraFinToolInvocationResult,
28
+ ToolExecutionMode,
29
  _ToolErrorDisposition,
30
  )
31
 
 
79
  payload: dict[str, Any]
80
  task: TerraFinTaskRecord | None = None
81
  resolved_arguments = _normalize_common_alias_arguments(dict(arguments or {}))
82
+ # Validate AFTER alias normalisation (so documented aliases are accepted)
83
+ # and BEFORE dispatch (so nothing undeclared reaches a handler).
84
+ schema_result, resolved_arguments = self._preflight_argument_schema(
85
+ tool, session_id, resolved_arguments
86
+ )
87
+ if schema_result is not None:
88
+ return schema_result
89
  preflight_result = self._preflight_tool_misuse(tool, session_id, resolved_arguments)
90
  if preflight_result is not None:
91
  return preflight_result
 
286
  },
287
  )
288
 
289
+ def _preflight_argument_schema(
290
+ self,
291
+ tool: TerraFinToolDefinition,
292
+ session_id: str,
293
+ arguments: Mapping[str, Any],
294
+ ) -> tuple[TerraFinToolInvocationResult | None, dict[str, Any]]:
295
+ """Reject arguments the tool's declared schema does not allow.
296
+
297
+ Returns `(rejection_or_None, arguments_to_dispatch)`; the second element
298
+ carries the validator's narrowing (see `validate_tool_arguments`) and must
299
+ be what the caller dispatches.
300
+
301
+ Without this the schema is decoration: dispatch went straight to
302
+ `capability.handler(**kwargs)`, so `additionalProperties: False` enforced
303
+ nothing and a handler's undeclared parameters stayed reachable. The
304
+ clearest case is `valuation`, whose hidden `base_growth_pct` /
305
+ `terminal_growth_pct` / `beta` let a caller tilt a valuation toward a
306
+ conclusion it already held.
307
+
308
+ Keyed on `capability_name`, not `name`: a backgroundable capability is also
309
+ exposed as `start_<capability>_task`, which is not a `HOSTED_TOOL_CONTRACTS`
310
+ key, so keying on `name` made all 24 task variants validate vacuously —
311
+ including `start_valuation_task`, leaving the tilt parameters reachable by
312
+ the one path this check exists to close. `_build_tool_definition` hands the
313
+ task variant the capability's own `input_schema`, so the model is shown the
314
+ contract validated here.
315
+
316
+ Only the agent surface is constrained. Internal Python callers keep the
317
+ full signature, which is the intent — a route or a controller is trusted
318
+ to choose inputs deliberately. `start_task`'s own `description` kwarg is
319
+ not in any contract, so a model that passes it is rejected; that matches
320
+ the `additionalProperties: false` schema the task tool advertises.
321
+ """
322
+ problems, normalized = validate_tool_arguments(tool.capability_name, arguments)
323
+ if not problems:
324
+ return None, normalized
325
+
326
+ message = f"Invalid arguments for {tool.name}: " + "; ".join(problems)
327
+ return TerraFinToolInvocationResult(
328
+ tool_name=tool.name, # the name the model called, not the capability
329
+ capability_name=tool.capability_name,
330
+ session_id=session_id,
331
+ execution_mode=tool.execution_mode,
332
+ payload={
333
+ "accepted": False,
334
+ "error": {
335
+ "code": "tool_invalid_arguments",
336
+ "message": message,
337
+ "detail": problems,
338
+ "retryable": True,
339
+ "modelHint": (
340
+ "Retry with only the arguments this tool declares, using the types and "
341
+ "ranges in its schema. Parameters that are not in the schema are not "
342
+ "available through the tool surface, even if a related HTTP route or "
343
+ "Python method accepts them."
344
+ ),
345
+ },
346
+ },
347
+ task=None,
348
+ is_error=True,
349
+ retryable=True,
350
+ error_code="tool_invalid_arguments",
351
+ error_message=message,
352
+ ), normalized
353
+
354
  def _preflight_tool_misuse(
355
  self,
356
  tool: TerraFinToolDefinition,
tests/agent/test_capability_parity.py CHANGED
@@ -60,6 +60,27 @@ def capabilities():
60
  return registry.list()
61
 
62
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
63
  @pytest.fixture(scope="module")
64
  def live_route_paths() -> set[str | None]:
65
  """Paths served by the assembled app.
@@ -197,6 +218,82 @@ def test_declared_cli_subcommands_are_wired(capabilities) -> None:
197
  assert not unwired, f"capabilities declaring a CLI subcommand that is not wired: {unwired}"
198
 
199
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
200
  def test_declared_http_routes_exist(capabilities, live_route_paths) -> None:
201
  missing = sorted(
202
  f"{c.name} -> {c.http_route_path}"
 
60
  return registry.list()
61
 
62
 
63
+ @pytest.fixture(scope="module")
64
+ def real_capabilities():
65
+ """Registry built against the REAL service *and* the real chart opener.
66
+
67
+ Signature checks must not measure a double. `BaseFakeService`'s stubs are
68
+ narrower than the handlers they stand in for, so a declared property the real
69
+ handler accepts (e.g. `market_snapshot`'s `force_refresh`) would read as a
70
+ mismatch. `fake_chart_opener` has the opposite problem: it is *wider* than
71
+ production `open_chart`, whose `client` parameter is undeclared and went
72
+ undetected here. Passing no `chart_opener` binds the real one.
73
+
74
+ Nothing is called — the handlers are only inspected — so this stays free of
75
+ network and env mutation.
76
+ """
77
+
78
+ from TerraFin.agent.service import TerraFinAgentService
79
+
80
+ registry = build_default_capability_registry(TerraFinAgentService())
81
+ return registry.list()
82
+
83
+
84
  @pytest.fixture(scope="module")
85
  def live_route_paths() -> set[str | None]:
86
  """Paths served by the assembled app.
 
218
  assert not unwired, f"capabilities declaring a CLI subcommand that is not wired: {unwired}"
219
 
220
 
221
+ # Handler parameters deliberately NOT in the tool schema. Each entry is a
222
+ # decision, not an oversight, and the tool boundary rejects them (see
223
+ # `tests/agent/test_tool_argument_validation.py`):
224
+ # valuation — the three tilt inputs. An LLM that can set its own growth rate,
225
+ # terminal growth, or beta can make a DCF agree with whatever it
226
+ # already believed, which is the failure the locked-DCF rule in
227
+ # the idea-loop design exists to prevent.
228
+ # macro_focus, open_chart — `session_id` is injected by the runtime
229
+ # (`_apply_defaults`), never supplied by a caller.
230
+ # open_chart — `client` is the `TerraFinAgentClient` the CLI passes in-process;
231
+ # a model naming its own HTTP client makes no sense.
232
+ INTERNAL_ONLY_HANDLER_PARAMS = {
233
+ "valuation": {"base_growth_pct", "terminal_growth_pct", "beta"},
234
+ "macro_focus": {"session_id"},
235
+ "open_chart": {"session_id", "client"},
236
+ }
237
+
238
+
239
+ def test_no_new_undeclared_handler_parameters(real_capabilities) -> None:
240
+ """A handler parameter absent from the tool schema must be a deliberate choice.
241
+
242
+ Adding one silently widens what the model could reach if validation were ever
243
+ bypassed, so new ones have to be named here on purpose.
244
+ """
245
+
246
+ import inspect
247
+
248
+ surprises: dict[str, list[str]] = {}
249
+ for capability in real_capabilities:
250
+ contract = HOSTED_TOOL_CONTRACTS.get(capability.name)
251
+ if contract is None:
252
+ continue
253
+ declared = set((contract.get("input_schema") or {}).get("properties") or {})
254
+ try:
255
+ parameters = inspect.signature(capability.handler).parameters
256
+ except (TypeError, ValueError): # pragma: no cover - builtins
257
+ continue
258
+ accepted = {
259
+ name
260
+ for name, parameter in parameters.items()
261
+ if parameter.kind in (parameter.POSITIONAL_OR_KEYWORD, parameter.KEYWORD_ONLY)
262
+ }
263
+ undeclared = accepted - declared - INTERNAL_ONLY_HANDLER_PARAMS.get(capability.name, set())
264
+ if undeclared:
265
+ surprises[capability.name] = sorted(undeclared)
266
+
267
+ assert not surprises, (
268
+ "handler parameters that are not in the tool schema and not listed as "
269
+ f"deliberately internal: {surprises}"
270
+ )
271
+
272
+
273
+ def test_declared_properties_are_acceptable_by_the_handler(real_capabilities) -> None:
274
+ """A declared property the handler cannot take is a TypeError waiting to happen."""
275
+
276
+ import inspect
277
+
278
+ broken: dict[str, list[str]] = {}
279
+ for capability in real_capabilities:
280
+ contract = HOSTED_TOOL_CONTRACTS.get(capability.name)
281
+ if contract is None:
282
+ continue
283
+ declared = set((contract.get("input_schema") or {}).get("properties") or {})
284
+ try:
285
+ parameters = inspect.signature(capability.handler).parameters
286
+ except (TypeError, ValueError): # pragma: no cover - builtins
287
+ continue
288
+ if any(parameter.kind == parameter.VAR_KEYWORD for parameter in parameters.values()):
289
+ continue
290
+ missing = sorted(declared - set(parameters))
291
+ if missing:
292
+ broken[capability.name] = missing
293
+
294
+ assert not broken, f"schema declares properties the handler cannot accept: {broken}"
295
+
296
+
297
  def test_declared_http_routes_exist(capabilities, live_route_paths) -> None:
298
  missing = sorted(
299
  f"{c.name} -> {c.http_route_path}"
tests/agent/test_loop.py CHANGED
@@ -1008,11 +1008,24 @@ def test_consult_tool_rejects_empty_question_argument() -> None:
1008
  session_id="loop:consult-empty-question",
1009
  )
1010
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1011
  with pytest.raises(ValueError, match="question"):
1012
  loop.tool_adapter.run_tool(
1013
  conversation.session_id,
1014
  "consult_warren_buffett",
1015
- {"question": ""},
1016
  )
1017
 
1018
 
 
1008
  session_id="loop:consult-empty-question",
1009
  )
1010
 
1011
+ # An empty string violates the contract's own `minLength: 1`, so schema
1012
+ # validation now rejects it at the tool boundary and hands the model a
1013
+ # retryable error instead of raising. The adapter's own guard still covers
1014
+ # what the schema cannot see: a whitespace-only question passes minLength.
1015
+ result = loop.tool_adapter.run_tool(
1016
+ conversation.session_id,
1017
+ "consult_warren_buffett",
1018
+ {"question": ""},
1019
+ )
1020
+ assert result.is_error is True
1021
+ assert result.error_code == "tool_invalid_arguments"
1022
+ assert "question" in result.error_message
1023
+
1024
  with pytest.raises(ValueError, match="question"):
1025
  loop.tool_adapter.run_tool(
1026
  conversation.session_id,
1027
  "consult_warren_buffett",
1028
+ {"question": " "},
1029
  )
1030
 
1031
 
tests/agent/test_tool_argument_validation.py ADDED
@@ -0,0 +1,253 @@
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ """Argument validation at the tool boundary.
2
+
3
+ The declared `input_schema` used to be advisory: dispatch went straight from the
4
+ model's arguments to `capability.handler(**kwargs)`, so `additionalProperties:
5
+ False` enforced nothing and every undeclared handler parameter stayed reachable
6
+ from the agent. `valuation` is the case that matters — its hidden
7
+ `base_growth_pct` / `terminal_growth_pct` / `beta` let a caller tilt a valuation
8
+ toward a conclusion it already held, which is exactly what the locked-DCF rule
9
+ in the idea-loop design exists to prevent.
10
+ """
11
+
12
+ import pytest
13
+
14
+ from TerraFin.agent.contracts.tool_contracts import validate_tool_arguments
15
+
16
+
17
+ def _problems(tool_name: str, arguments: dict) -> list[str]:
18
+ """Just the problem list. `validate_tool_arguments` also returns the narrowed
19
+ arguments to dispatch; the tests that care about narrowing read it directly."""
20
+
21
+ return validate_tool_arguments(tool_name, arguments)[0]
22
+
23
+
24
+ def test_valid_arguments_pass() -> None:
25
+ assert _problems("valuation", {"ticker": "AAPL", "projection_years": 10}) == []
26
+ assert _problems("market_data", {"name": "AAPL", "depth": "full"}) == []
27
+
28
+
29
+ def test_undeclared_valuation_tilt_parameters_are_rejected() -> None:
30
+ """The whole point: these exist on the handler but not in the schema."""
31
+
32
+ for hidden in ("base_growth_pct", "terminal_growth_pct", "beta"):
33
+ problems = _problems("valuation", {"ticker": "AAPL", hidden: 42})
34
+ assert problems, f"{hidden} must not be reachable through the tool surface"
35
+ assert hidden in problems[0]
36
+
37
+
38
+ def test_missing_required_argument_is_reported() -> None:
39
+ assert _problems("valuation", {}) == ["ticker: required"]
40
+
41
+
42
+ def test_enum_violation_is_reported() -> None:
43
+ problems = _problems("market_data", {"name": "AAPL", "depth": "everything"})
44
+
45
+ assert len(problems) == 1
46
+ assert "not one of" in problems[0]
47
+
48
+
49
+ def test_numeric_bounds_are_enforced() -> None:
50
+ assert _problems("relative_strength", {"top_n": 101})
51
+ assert _problems("relative_strength", {"top_n": 0})
52
+ assert _problems("relative_strength", {"top_n": 50}) == []
53
+
54
+
55
+ def test_integral_floats_are_narrowed_not_merely_accepted() -> None:
56
+ """Models routinely emit 20.0 for an integer field; 20.5 is a real error.
57
+
58
+ Accepting 20.0 without narrowing it to `int` only relocates the failure:
59
+ `relative_strength` reaches `ordered[:top_n]` and raises `TypeError: slice
60
+ indices must be integers`, which no classifier recognises, so the run aborts
61
+ instead of returning a tool error. The narrowed value is the contract.
62
+ """
63
+
64
+ problems, normalized = validate_tool_arguments("relative_strength", {"top_n": 20.0})
65
+ assert problems == []
66
+ assert normalized["top_n"] == 20
67
+ assert isinstance(normalized["top_n"], int)
68
+
69
+ assert _problems("relative_strength", {"top_n": 20.5})
70
+
71
+
72
+ def test_narrowing_leaves_other_arguments_untouched() -> None:
73
+ _, normalized = validate_tool_arguments(
74
+ "relative_strength", {"top_n": 20.0, "universe": "sp500", "period": "6mo"}
75
+ )
76
+
77
+ assert normalized == {"top_n": 20, "universe": "sp500", "period": "6mo"}
78
+
79
+
80
+ def test_booleans_are_not_integers_or_strings() -> None:
81
+ """bool is an int subclass in Python; JSON booleans are neither."""
82
+
83
+ assert _problems("relative_strength", {"top_n": True})
84
+ assert _problems("valuation", {"ticker": True})
85
+
86
+
87
+ def test_string_or_array_unions_accept_both_forms() -> None:
88
+ assert _problems("pattern_scan", {"tickers": "NVDA,AMD"}) == []
89
+ assert _problems("pattern_scan", {"tickers": ["NVDA", "AMD"]}) == []
90
+ assert _problems("pattern_scan", {"tickers": 123})
91
+
92
+
93
+ def test_array_bounds_and_item_types_name_the_actual_problem() -> None:
94
+ """A union branch must not report the value's own type back at the model.
95
+
96
+ `tickers` is `anyOf: [string, array]`. Reporting "expected string or array,
97
+ got list" for an over-long list is unactionable — the value *is* a list — so
98
+ the model resends it, hits the same error fingerprint, and burns the retry
99
+ budget. Assert the wording, not mere truthiness.
100
+ """
101
+
102
+ too_many = _problems("pattern_scan", {"tickers": ["X"] * 501})
103
+ assert too_many == ["tickers: at most 500 item(s), got 501"]
104
+
105
+ bad_item = _problems("pattern_scan", {"tickers": ["NVDA", 7]})
106
+ assert bad_item == ["tickers[1]: expected string, got int"]
107
+
108
+ # No branch matches on type, so the generic union message is the right one.
109
+ assert _problems("pattern_scan", {"tickers": 123}) == [
110
+ "tickers: expected string or array, got int"
111
+ ]
112
+
113
+
114
+ def test_minimum_length_is_enforced() -> None:
115
+ assert _problems("valuation", {"ticker": ""})
116
+
117
+
118
+ def test_optional_nulls_are_left_to_the_handler() -> None:
119
+ """An omitted-as-null optional is common from models and is not a schema error."""
120
+
121
+ assert _problems("news", {"ticker": "NVDA", "query": None}) == []
122
+
123
+
124
+ def test_a_null_required_argument_is_rejected() -> None:
125
+ """`{"ticker": None}` satisfies "key is present" but not the handler.
126
+
127
+ `service.valuation` calls `ticker.upper()`, so a null that passes validation
128
+ becomes an `AttributeError` the classifier does not recognise, aborting the
129
+ run. Absent and null are different errors, and both are errors.
130
+ """
131
+
132
+ assert _problems("valuation", {"ticker": None}) == ["ticker: required, but was null"]
133
+ assert _problems("valuation", {}) == ["ticker: required"]
134
+
135
+
136
+ def test_unknown_tool_validates_vacuously() -> None:
137
+ """An unregistered tool is a different error, reported elsewhere."""
138
+
139
+ assert _problems("not_a_tool", {"anything": 1}) == []
140
+
141
+
142
+ @pytest.mark.parametrize("tool_name", ["consensus", "news", "pattern_scan", "relative_strength"])
143
+ def test_unknown_arguments_are_rejected_for_every_recent_tool(tool_name: str) -> None:
144
+ problems = _problems(tool_name, {"definitely_not_a_field": 1})
145
+
146
+ # Not problems[0]: a missing required argument is reported first for tools
147
+ # that have one.
148
+ assert any("unknown argument" in problem for problem in problems), problems
149
+
150
+
151
+ def _adapter_and_session():
152
+ """A tool adapter over the real registry with a stub service.
153
+
154
+ Mirrors `tests/agent/test_tools.py::_adapter` — the runtime needs the service
155
+ as well as the registry, and the session id lives at
156
+ `session.session.session_id`.
157
+ """
158
+
159
+ from fakes import BaseFakeService, fake_chart_opener
160
+
161
+ from TerraFin.agent.contracts.definitions import DEFAULT_HOSTED_AGENT_NAME
162
+ from TerraFin.agent.runtime import build_default_capability_registry
163
+ from TerraFin.agent.runtime.hosted import TerraFinHostedAgentRuntime
164
+ from TerraFin.agent.tools import TerraFinHostedToolAdapter
165
+
166
+ service = BaseFakeService()
167
+ registry = build_default_capability_registry(service, chart_opener=fake_chart_opener)
168
+ runtime = TerraFinHostedAgentRuntime(service=service, capability_registry=registry)
169
+ adapter = TerraFinHostedToolAdapter(runtime)
170
+ session = runtime.create_session(DEFAULT_HOSTED_AGENT_NAME, session_id="tool:schema")
171
+ return adapter, session.session.session_id
172
+
173
+
174
+ @pytest.mark.parametrize("tool_name", ["valuation", "start_valuation_task"])
175
+ def test_tool_path_rejects_a_hidden_valuation_parameter(tool_name: str) -> None:
176
+ """End to end: the schema is enforced on BOTH surfaces the model can call.
177
+
178
+ `start_valuation_task` is the case that regressed. Validation keyed on
179
+ `tool.name`, and the task variant is named `start_valuation_task`, which is not
180
+ a `HOSTED_TOOL_CONTRACTS` key — so it validated vacuously and the tilt
181
+ parameters stayed reachable through the exact capability this check protects.
182
+ Every backgroundable capability had the same hole.
183
+ """
184
+
185
+ adapter, session_id = _adapter_and_session()
186
+
187
+ result = adapter.run_tool(session_id, tool_name, {"ticker": "AAPL", "base_growth_pct": 40})
188
+
189
+ assert result.is_error is True
190
+ assert result.error_code == "tool_invalid_arguments"
191
+ assert "base_growth_pct" in result.error_message
192
+ assert result.payload["accepted"] is False
193
+ assert result.payload["error"]["retryable"] is True
194
+ assert "not available through the tool surface" in result.payload["error"]["modelHint"]
195
+
196
+
197
+ def test_every_exposed_tool_validates_against_a_contract() -> None:
198
+ """No tool the model is handed may fall through validation.
199
+
200
+ The task-variant bypass was invisible because the tests only ever exercised
201
+ invoke-mode names. This asserts the property directly, over the full tool list
202
+ the default agent actually sees, so a future execution mode with a new name
203
+ shape cannot reintroduce it silently.
204
+ """
205
+
206
+ from TerraFin.agent.contracts.definitions import DEFAULT_HOSTED_AGENT_NAME
207
+ from TerraFin.agent.contracts.tool_contracts import HOSTED_TOOL_CONTRACTS
208
+
209
+ adapter, _ = _adapter_and_session()
210
+ tools = adapter.list_tools_for_agent(DEFAULT_HOSTED_AGENT_NAME)
211
+
212
+ assert len(tools) > 40, "expected the full tool surface, not a filtered subset"
213
+ unvalidated = sorted(t.name for t in tools if t.capability_name not in HOSTED_TOOL_CONTRACTS)
214
+ assert not unvalidated, f"tools whose arguments no contract constrains: {unvalidated}"
215
+
216
+
217
+ def test_task_variants_still_start_with_valid_arguments() -> None:
218
+ """The D1 fix must reject the undeclared argument, not the whole task path."""
219
+
220
+ adapter, session_id = _adapter_and_session()
221
+
222
+ result = adapter.run_tool(session_id, "start_valuation_task", {"ticker": "AAPL"})
223
+
224
+ assert result.is_error is False
225
+ assert result.payload["accepted"] is True
226
+ assert result.task is not None
227
+ assert result.task.input_payload["ticker"] == "AAPL"
228
+
229
+
230
+ def test_tool_path_still_accepts_declared_arguments() -> None:
231
+ adapter, session_id = _adapter_and_session()
232
+
233
+ result = adapter.run_tool(session_id, "valuation", {"ticker": "AAPL", "projection_years": 10})
234
+
235
+ assert result.is_error is False
236
+ assert result.payload["ticker"] == "AAPL"
237
+
238
+
239
+ def test_internal_callers_keep_the_full_signature() -> None:
240
+ """Locking the agent must not lock a route or controller.
241
+
242
+ The service method deliberately still accepts the tilt parameters — a
243
+ deliberate internal caller is trusted; the model is not.
244
+ """
245
+
246
+ import inspect
247
+
248
+ from TerraFin.agent.service import TerraFinAgentService
249
+
250
+ parameters = inspect.signature(TerraFinAgentService.valuation).parameters
251
+
252
+ for hidden in ("base_growth_pct", "terminal_growth_pct", "beta"):
253
+ assert hidden in parameters
tests/fakes.py CHANGED
@@ -162,7 +162,26 @@ class BaseFakeService:
162
  "processing": processing(),
163
  }
164
 
165
- def valuation(self, ticker: str) -> dict[str, object]:
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
166
  return {
167
  "ticker": ticker,
168
  "dcf": {"status": "ready", "intrinsic_value": 120.0},
 
162
  "processing": processing(),
163
  }
164
 
165
+ def valuation(
166
+ self,
167
+ ticker: str,
168
+ *,
169
+ projection_years: int | None = None,
170
+ fcf_base_source: str | None = None,
171
+ breakeven_year: int | None = None,
172
+ breakeven_cash_flow_per_share: float | None = None,
173
+ post_breakeven_growth_pct: float | None = None,
174
+ ) -> dict[str, object]:
175
+ """Accepts the schema's declared optionals, deliberately NOT the hidden
176
+ tilt parameters (`base_growth_pct`, `terminal_growth_pct`, `beta`) — those
177
+ must not be reachable from the tool surface."""
178
+ _ = (
179
+ projection_years,
180
+ fcf_base_source,
181
+ breakeven_year,
182
+ breakeven_cash_flow_per_share,
183
+ post_breakeven_growth_pct,
184
+ )
185
  return {
186
  "ticker": ticker,
187
  "dcf": {"status": "ready", "intrinsic_value": 120.0},