feat(phase2): observation-only LLMPolicy entry (D1) + pinned schema_version (P0.1)
- LLMPolicy is now Callable[[Observation], tuple[Action, ...]]: the snapshot is the policy's sole information source; the live CombatantState is no longer passed to new-style policies. - CombatEngine._llm_adapter keeps a compatibility path for pre-D1 two-argument callables: the signature is inspected ONCE at wrap time (_accepts_state) and legacy policies still receive (observation, state). - Observation.to_dict() carries schema_version (OBSERVATION_SCHEMA_VERSION=1) as its first pinned key; bump only on removal/meaning change. - Pinned keyset tests updated (test_observation, test_observation_enriched); new tests: observation-only e2e, legacy 2-arg e2e through the adapter, schema_version pin + JSON-serializability. - README: documents the new entry signature and the schema_version key.
This commit is contained in:
@@ -1951,8 +1951,11 @@ Règles modélisées :
|
|||||||
retour : la politique peut les muter sans affecter le combat ; l'objet est un
|
retour : la politique peut les muter sans affecter le combat ; l'objet est un
|
||||||
dataclass figé au premier niveau, les dictionnaires internes sont des copies
|
dataclass figé au premier niveau, les dictionnaires internes sont des copies
|
||||||
mutables, non deep-freezés). Une politique LLM
|
mutables, non deep-freezés). Une politique LLM
|
||||||
se branche via `CombatEngine(..., llm_policy=...)` (`LLMPolicy(observation,
|
se branche via `CombatEngine(..., llm_policy=...)` (`LLMPolicy(observation)
|
||||||
state) -> tuple[Action, ...]`). Schéma enrichi (E1-E5) :
|
-> tuple[Action, ...]` : la politique ne reçoit QUE le snapshot, jamais
|
||||||
|
l'état vivant ; un callable hérité à deux arguments `(observation, state)`
|
||||||
|
reste accepté via l'adaptateur de compatibilité D1, signature inspectée
|
||||||
|
une fois à la construction). Schéma enrichi (E1-E5) :
|
||||||
- Par combattant : id, side, pos, occupied_cells, hp/hp_max, active/dead,
|
- Par combattant : id, side, pos, occupied_cells, hp/hp_max, active/dead,
|
||||||
dying/stabilized, conditions (noms + rounds_left), resources_left (+ `spell_slot_{level}`),
|
dying/stabilized, conditions (noms + rounds_left), resources_left (+ `spell_slot_{level}`),
|
||||||
weapon_broken/weapon_destroyed/weapon_stolen/weapon_dropped, carried_items
|
weapon_broken/weapon_destroyed/weapon_stolen/weapon_dropped, carried_items
|
||||||
@@ -2050,7 +2053,10 @@ Règles modélisées :
|
|||||||
charge_reason (str | None — None si `can_charge` est vrai ; sinon la
|
charge_reason (str | None — None si `can_charge` est vrai ; sinon la
|
||||||
raison de `ChargeResult` verbatim (`no_line_of_effect` / `no_charge_path`)
|
raison de `ChargeResult` verbatim (`no_line_of_effect` / `no_charge_path`)
|
||||||
ou `no_weapon` si aucune arme de mêlée utilisable).
|
ou `no_weapon` si aucune arme de mêlée utilisable).
|
||||||
- Top-level : round, acting_combatant, combatants, pairs, terrain, elevation,
|
- Top-level : schema_version (P0.1 : version épinglée du schéma d'observation,
|
||||||
|
constante `OBSERVATION_SCHEMA_VERSION` = 1 ; incrémentée seulement si une
|
||||||
|
clé est retirée ou change de sens), round, acting_combatant, combatants,
|
||||||
|
pairs, terrain, elevation,
|
||||||
threatened (grille bool des cases menacées au contact par les ennemis —
|
threatened (grille bool des cases menacées au contact par les ennemis —
|
||||||
AoO), ranged_threatened (U6d : grille bool des cases sous tir ennemi à
|
AoO), ranged_threatened (U6d : grille bool des cases sous tir ennemi à
|
||||||
distance — champ de tir des armes à distance utilisables des ennemis
|
distance — champ de tir des armes à distance utilisables des ennemis
|
||||||
|
|||||||
@@ -121,6 +121,7 @@ Phase 0 documented deviations from PF1e (conventions):
|
|||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import inspect
|
||||||
from collections.abc import Callable, Iterable
|
from collections.abc import Callable, Iterable
|
||||||
from dataclasses import asdict, dataclass, field, replace
|
from dataclasses import asdict, dataclass, field, replace
|
||||||
from itertools import pairwise
|
from itertools import pairwise
|
||||||
@@ -1259,6 +1260,12 @@ class ChargeResult:
|
|||||||
reason: str | None
|
reason: str | None
|
||||||
|
|
||||||
|
|
||||||
|
# Observation schema version (P0.1, docs/phase-2-llm-architecture.md section
|
||||||
|
# 10). Bump only when a top-level key is removed or its meaning changes;
|
||||||
|
# additive keys keep the version.
|
||||||
|
OBSERVATION_SCHEMA_VERSION = 1
|
||||||
|
|
||||||
|
|
||||||
@dataclass(frozen=True)
|
@dataclass(frozen=True)
|
||||||
class Observation:
|
class Observation:
|
||||||
"""Serializable snapshot of combat state for a policy/LLM.
|
"""Serializable snapshot of combat state for a policy/LLM.
|
||||||
@@ -1269,7 +1276,9 @@ class Observation:
|
|||||||
collections are copies (tuples/dicts), never read back by the engine, so a
|
collections are copies (tuples/dicts), never read back by the engine, so a
|
||||||
policy may mutate them freely without touching combat; the snapshot is
|
policy may mutate them freely without touching combat; the snapshot is
|
||||||
JSON-serializable (the top level is a frozen dataclass; the inner dicts are
|
JSON-serializable (the top level is a frozen dataclass; the inner dicts are
|
||||||
plain mutable copies, not deep-frozen).
|
plain mutable copies, not deep-frozen). ``to_dict()`` carries
|
||||||
|
``schema_version`` (``OBSERVATION_SCHEMA_VERSION``) as its first key so a
|
||||||
|
consumer can detect breaking schema changes.
|
||||||
|
|
||||||
``combatants`` holds one dict per combatant in ``self._states`` order;
|
``combatants`` holds one dict per combatant in ``self._states`` order;
|
||||||
``pairs`` holds one dict per (acting combatant x each other combatant) with
|
``pairs`` holds one dict per (acting combatant x each other combatant) with
|
||||||
@@ -1310,6 +1319,7 @@ class Observation:
|
|||||||
def to_dict(self) -> dict[str, object]:
|
def to_dict(self) -> dict[str, object]:
|
||||||
"""Return a JSON-serializable dict with the pinned top-level keys."""
|
"""Return a JSON-serializable dict with the pinned top-level keys."""
|
||||||
return {
|
return {
|
||||||
|
"schema_version": OBSERVATION_SCHEMA_VERSION,
|
||||||
"round": self.round,
|
"round": self.round,
|
||||||
"acting_combatant": self.acting_combatant,
|
"acting_combatant": self.acting_combatant,
|
||||||
"combatants": self.combatants,
|
"combatants": self.combatants,
|
||||||
@@ -1613,7 +1623,7 @@ class CombatEngine:
|
|||||||
*,
|
*,
|
||||||
round_cap: int = 100,
|
round_cap: int = 100,
|
||||||
policy: Policy | None = None,
|
policy: Policy | None = None,
|
||||||
llm_policy: LLMPolicy | None = None,
|
llm_policy: LLMPolicy | LegacyLLMPolicy | None = None,
|
||||||
spell_registry: dict[str, SpellSpec] | None = None,
|
spell_registry: dict[str, SpellSpec] | None = None,
|
||||||
reaction_policy: ReactionPolicy | None = None,
|
reaction_policy: ReactionPolicy | None = None,
|
||||||
monsters_die_at_zero: bool = False,
|
monsters_die_at_zero: bool = False,
|
||||||
@@ -1685,18 +1695,24 @@ class CombatEngine:
|
|||||||
if self._monsters_die_at_zero and not _is_pc_combatant(state.combatant):
|
if self._monsters_die_at_zero and not _is_pc_combatant(state.combatant):
|
||||||
state.dies_at_zero = True
|
state.dies_at_zero = True
|
||||||
|
|
||||||
def _llm_adapter(self, llm_policy: LLMPolicy) -> Policy:
|
def _llm_adapter(self, llm_policy: LLMPolicy | LegacyLLMPolicy) -> Policy:
|
||||||
"""Wrap an ``LLMPolicy`` into a ``Policy``.
|
"""Wrap an ``LLMPolicy`` (or a pre-D1 two-argument callable) into a ``Policy``.
|
||||||
|
|
||||||
At the point of invocation (``actions = self._policy(self, state)`` in
|
At the point of invocation (``actions = self._policy(self, state)`` in
|
||||||
``_take_turn``), the Observation is built for the current state and
|
``_take_turn``), the Observation is built for the current state and
|
||||||
delegated to ``llm_policy(observation, state)``. The adapter consumes
|
delegated. New-style policies get ``llm_policy(observation)`` only;
|
||||||
no RNG and mutates no state.
|
a legacy ``llm_policy(observation, state)`` callable is detected ONCE
|
||||||
|
here (signature inspection, not per turn) and keeps receiving the live
|
||||||
|
state. The adapter consumes no RNG and mutates no state.
|
||||||
"""
|
"""
|
||||||
|
two_arg = _accepts_state(llm_policy)
|
||||||
|
call: Callable[..., tuple[Action, ...]] = llm_policy
|
||||||
|
|
||||||
def adapter(_engine: CombatEngine, state: CombatantState) -> tuple[Action, ...]:
|
def adapter(_engine: CombatEngine, state: CombatantState) -> tuple[Action, ...]:
|
||||||
observation = self._build_observation(state)
|
observation = self._build_observation(state)
|
||||||
return llm_policy(observation, state)
|
if two_arg:
|
||||||
|
return call(observation, state)
|
||||||
|
return call(observation)
|
||||||
|
|
||||||
return adapter
|
return adapter
|
||||||
|
|
||||||
@@ -10572,7 +10588,36 @@ def _ready_payload(action: Action) -> Action:
|
|||||||
|
|
||||||
Policy = Callable[[CombatEngine, CombatantState], tuple[Action, ...]]
|
Policy = Callable[[CombatEngine, CombatantState], tuple[Action, ...]]
|
||||||
|
|
||||||
LLMPolicy = Callable[[Observation, CombatantState], tuple[Action, ...]]
|
# D1 (docs/phase-2-llm-architecture.md section 13): the LLM policy entry is
|
||||||
|
# observation-only -- the snapshot is the policy's sole information source and
|
||||||
|
# the live CombatantState is never handed to a new-style policy. Pre-D1
|
||||||
|
# two-argument callables remain accepted via CombatEngine._llm_adapter.
|
||||||
|
LLMPolicy = Callable[[Observation], tuple[Action, ...]]
|
||||||
|
|
||||||
|
LegacyLLMPolicy = Callable[[Observation, CombatantState], tuple[Action, ...]]
|
||||||
|
|
||||||
|
_LEGACY_LLM_POLICY_ARITY = 2
|
||||||
|
|
||||||
|
|
||||||
|
def _accepts_state(llm_policy: LLMPolicy | LegacyLLMPolicy) -> bool:
|
||||||
|
"""True when ``llm_policy`` is a pre-D1 callable taking (observation, state).
|
||||||
|
|
||||||
|
Counts REQUIRED positional parameters (a defaulted second parameter is a
|
||||||
|
new-style policy that tolerates the 1-argument call). Callables whose
|
||||||
|
signature cannot be inspected default to the new-style 1-argument call.
|
||||||
|
"""
|
||||||
|
try:
|
||||||
|
parameters = list(inspect.signature(llm_policy).parameters.values())
|
||||||
|
except (TypeError, ValueError):
|
||||||
|
return False
|
||||||
|
required_positional = [
|
||||||
|
p
|
||||||
|
for p in parameters
|
||||||
|
if p.default is inspect.Parameter.empty
|
||||||
|
and p.kind
|
||||||
|
in (inspect.Parameter.POSITIONAL_ONLY, inspect.Parameter.POSITIONAL_OR_KEYWORD)
|
||||||
|
]
|
||||||
|
return len(required_positional) >= _LEGACY_LLM_POLICY_ARITY
|
||||||
|
|
||||||
ReactionPolicy = Callable[
|
ReactionPolicy = Callable[
|
||||||
[CombatEngine, CombatantState, ReactionEvent], Action | None
|
[CombatEngine, CombatantState, ReactionEvent], Action | None
|
||||||
|
|||||||
@@ -229,7 +229,7 @@ def test_results_visible_in_next_turn_observation() -> None:
|
|||||||
# the second invocation must already see the first turn's rejection.
|
# the second invocation must already see the first turn's rejection.
|
||||||
seen: list[Observation] = []
|
seen: list[Observation] = []
|
||||||
|
|
||||||
def llm_policy(obs: Observation, state: CombatantState) -> tuple[Action, ...]:
|
def llm_policy(obs: Observation) -> tuple[Action, ...]:
|
||||||
seen.append(obs)
|
seen.append(obs)
|
||||||
if len(seen) == 1:
|
if len(seen) == 1:
|
||||||
return (Action(kind="move"),)
|
return (Action(kind="move"),)
|
||||||
|
|||||||
@@ -1,7 +1,7 @@
|
|||||||
"""Package U8 — final validation: an observation-only LLM policy drives the
|
"""Package U8 — final validation: an observation-only LLM policy drives the
|
||||||
Phase-2 capabilities (U1-U7b) end-to-end through the real engine.
|
Phase-2 capabilities (U1-U7b) end-to-end through the real engine.
|
||||||
|
|
||||||
Each test wires a ``llm_policy`` (``LLMPolicy(observation, state)``) that
|
Each test wires a ``llm_policy`` (``LLMPolicy(observation)``) that
|
||||||
reads ONLY the frozen Observation snapshot — never ``engine.*`` — and emits
|
reads ONLY the frozen Observation snapshot — never ``engine.*`` — and emits
|
||||||
one newly-unblocked tactical decision per capability: ``ready`` (U4),
|
one newly-unblocked tactical decision per capability: ``ready`` (U4),
|
||||||
``delay`` (U3), ``total_defense`` (U5a), ``move`` with an explicit
|
``delay`` (U3), ``total_defense`` (U5a), ``move`` with an explicit
|
||||||
@@ -121,7 +121,7 @@ def _counting(decide: Decide) -> LLMPolicy:
|
|||||||
"""LLMPolicy calling ``decide(id, call_no, obs)`` per turn (obs only)."""
|
"""LLMPolicy calling ``decide(id, call_no, obs)`` per turn (obs only)."""
|
||||||
calls: dict[str, int] = {}
|
calls: dict[str, int] = {}
|
||||||
|
|
||||||
def policy(obs: Observation, _state: CombatantState) -> tuple[Action, ...]:
|
def policy(obs: Observation) -> tuple[Action, ...]:
|
||||||
cid = str(_me(obs)["id"])
|
cid = str(_me(obs)["id"])
|
||||||
n = calls.get(cid, 0)
|
n = calls.get(cid, 0)
|
||||||
calls[cid] = n + 1
|
calls[cid] = n + 1
|
||||||
|
|||||||
@@ -9,9 +9,12 @@ Covers the pinned schema:
|
|||||||
- Terrain/elevation layers.
|
- Terrain/elevation layers.
|
||||||
- Frozen copies: mutating live state after construction does not change the
|
- Frozen copies: mutating live state after construction does not change the
|
||||||
already-built Observation.
|
already-built Observation.
|
||||||
- ``to_dict()`` is JSON-serializable and exposes no ``engine.*`` callables.
|
- ``to_dict()`` is JSON-serializable, carries the pinned ``schema_version``
|
||||||
|
key, and exposes no ``engine.*`` callables.
|
||||||
- The ``llm_policy`` adapter builds the Observation at the point of policy
|
- The ``llm_policy`` adapter builds the Observation at the point of policy
|
||||||
invocation and delegates to ``llm_policy(observation, state)``.
|
invocation and delegates to ``llm_policy(observation)`` (D1: observation
|
||||||
|
only); a legacy ``llm_policy(observation, state)`` callable keeps working
|
||||||
|
through the compatibility adapter.
|
||||||
"""
|
"""
|
||||||
|
|
||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
@@ -20,7 +23,13 @@ import json
|
|||||||
|
|
||||||
import pytest
|
import pytest
|
||||||
|
|
||||||
from pf1e_simulator.combat import Action, CombatantState, CombatEngine, Observation
|
from pf1e_simulator.combat import (
|
||||||
|
OBSERVATION_SCHEMA_VERSION,
|
||||||
|
Action,
|
||||||
|
CombatantState,
|
||||||
|
CombatEngine,
|
||||||
|
Observation,
|
||||||
|
)
|
||||||
from pf1e_simulator.conditions import SHAKEN
|
from pf1e_simulator.conditions import SHAKEN
|
||||||
from pf1e_simulator.dice import parse_dice
|
from pf1e_simulator.dice import parse_dice
|
||||||
from pf1e_simulator.effects import ActiveCondition
|
from pf1e_simulator.effects import ActiveCondition
|
||||||
@@ -441,11 +450,11 @@ def test_observation_frozen_copies() -> None:
|
|||||||
|
|
||||||
|
|
||||||
def test_llm_policy_wrapper_e2e() -> None:
|
def test_llm_policy_wrapper_e2e() -> None:
|
||||||
"""Given: a CombatEngine with an llm_policy that reads ONLY the observation
|
"""Given: a CombatEngine with an observation-only llm_policy (D1 signature)
|
||||||
When: the battle runs
|
When: the battle runs
|
||||||
Then: the policy's decisions are honored and the transcript is coherent."""
|
Then: the policy's decisions are honored and the transcript is coherent."""
|
||||||
|
|
||||||
def llm_policy(obs: Observation, state: CombatantState) -> tuple:
|
def llm_policy(obs: Observation) -> tuple:
|
||||||
# Read only the observation: full_attack if the target is adjacent,
|
# Read only the observation: full_attack if the target is adjacent,
|
||||||
# otherwise move toward it.
|
# otherwise move toward it.
|
||||||
target_id = obs.combatants[obs.pairs[0]["target_index"]]["id"]
|
target_id = obs.combatants[obs.pairs[0]["target_index"]]["id"]
|
||||||
@@ -470,6 +479,65 @@ def test_llm_policy_wrapper_e2e() -> None:
|
|||||||
assert any("move" in line for line in result.transcript)
|
assert any("move" in line for line in result.transcript)
|
||||||
|
|
||||||
|
|
||||||
|
def test_legacy_two_arg_llm_policy_still_works() -> None:
|
||||||
|
"""Given: a pre-D1 llm_policy taking (observation, state)
|
||||||
|
When: the battle runs through the compatibility adapter
|
||||||
|
Then: the policy receives BOTH arguments and its decisions are honored."""
|
||||||
|
seen_states: list[CombatantState] = []
|
||||||
|
|
||||||
|
def legacy_policy(obs: Observation, state: CombatantState) -> tuple:
|
||||||
|
seen_states.append(state)
|
||||||
|
assert state.combatant.id == obs.combatants[obs.acting_combatant]["id"]
|
||||||
|
target_id = obs.combatants[obs.pairs[0]["target_index"]]["id"]
|
||||||
|
return (Action(kind="move", target_id=target_id),)
|
||||||
|
|
||||||
|
hero = make_combatant("hero")
|
||||||
|
foe = make_combatant("foe")
|
||||||
|
hero_state = make_state(hero, "players", (0, 0))
|
||||||
|
foe_state = make_state(foe, "monsters", (0, 3))
|
||||||
|
engine = CombatEngine(
|
||||||
|
ScriptedRng([10, 9, 8, 15, 3, 5]),
|
||||||
|
make_grid(),
|
||||||
|
[hero_state, foe_state],
|
||||||
|
round_cap=1,
|
||||||
|
llm_policy=legacy_policy,
|
||||||
|
)
|
||||||
|
|
||||||
|
result = engine.run()
|
||||||
|
assert seen_states, "legacy policy was never invoked"
|
||||||
|
assert any("move" in line for line in result.transcript)
|
||||||
|
|
||||||
|
|
||||||
|
def test_observation_schema_version_pinned() -> None:
|
||||||
|
"""Given: a built Observation
|
||||||
|
When: to_dict() is called
|
||||||
|
Then: the top-level key set is exactly the pinned keys and
|
||||||
|
schema_version == OBSERVATION_SCHEMA_VERSION, JSON-serializable."""
|
||||||
|
hero = make_combatant("hero")
|
||||||
|
foe = make_combatant("foe")
|
||||||
|
hero_state = make_state(hero, "players", (0, 0))
|
||||||
|
foe_state = make_state(foe, "monsters", (0, 3))
|
||||||
|
engine = make_engine([hero_state, foe_state])
|
||||||
|
|
||||||
|
data = engine._build_observation(hero_state).to_dict()
|
||||||
|
assert data["schema_version"] == OBSERVATION_SCHEMA_VERSION
|
||||||
|
assert set(data) == {
|
||||||
|
"schema_version",
|
||||||
|
"round",
|
||||||
|
"acting_combatant",
|
||||||
|
"combatants",
|
||||||
|
"pairs",
|
||||||
|
"terrain",
|
||||||
|
"elevation",
|
||||||
|
"threatened",
|
||||||
|
"ranged_threatened",
|
||||||
|
"terrain_semantics",
|
||||||
|
"condition_definitions",
|
||||||
|
"initiative_order",
|
||||||
|
}
|
||||||
|
json.dumps(data) # must not raise
|
||||||
|
|
||||||
|
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
# 12. policy + llm_policy → ValueError
|
# 12. policy + llm_policy → ValueError
|
||||||
# ---------------------------------------------------------------------------
|
# ---------------------------------------------------------------------------
|
||||||
|
|||||||
@@ -1051,7 +1051,8 @@ def test_observation_enriched_terrain_semantics_no_callable_no_share() -> None:
|
|||||||
def test_observation_enriched_e5_existing_keys_preserved() -> None:
|
def test_observation_enriched_e5_existing_keys_preserved() -> None:
|
||||||
"""Given: an Observation built for the L5 fields
|
"""Given: an Observation built for the L5 fields
|
||||||
When: the top-level dict is inspected
|
When: the top-level dict is inspected
|
||||||
Then: all pre-existing top-level keys remain (round, acting_combatant,
|
Then: all pre-existing top-level keys remain (schema_version, round,
|
||||||
|
acting_combatant,
|
||||||
combatants, pairs, terrain, elevation) and the new keys
|
combatants, pairs, terrain, elevation) and the new keys
|
||||||
(threatened, ranged_threatened, terrain_semantics,
|
(threatened, ranged_threatened, terrain_semantics,
|
||||||
condition_definitions, initiative_order) are added — exact key set."""
|
condition_definitions, initiative_order) are added — exact key set."""
|
||||||
@@ -1063,6 +1064,7 @@ def test_observation_enriched_e5_existing_keys_preserved() -> None:
|
|||||||
|
|
||||||
obs = _snapshot(engine, hero_state)
|
obs = _snapshot(engine, hero_state)
|
||||||
assert set(obs) == {
|
assert set(obs) == {
|
||||||
|
"schema_version",
|
||||||
"round",
|
"round",
|
||||||
"acting_combatant",
|
"acting_combatant",
|
||||||
"combatants",
|
"combatants",
|
||||||
@@ -1138,7 +1140,7 @@ def _best_usable_attack(obs: Observation) -> dict[str, object] | None:
|
|||||||
return best
|
return best
|
||||||
|
|
||||||
|
|
||||||
def _enriched_llm_policy(obs: Observation, state: CombatantState) -> tuple[Action, ...]:
|
def _enriched_llm_policy(obs: Observation) -> tuple[Action, ...]:
|
||||||
"""Read ONLY the observation — never state.combatant.* nor engine.*."""
|
"""Read ONLY the observation — never state.combatant.* nor engine.*."""
|
||||||
cast = _castable_spell_action(obs)
|
cast = _castable_spell_action(obs)
|
||||||
if cast is not None:
|
if cast is not None:
|
||||||
|
|||||||
Reference in New Issue
Block a user