Kyle Luecke

research / agentic safety & runtime verification · v1.0.0

Typed Boundaries for Small-Model Web Agents

Preregistered BrowserGym/MiniWoB++ evaluation and N=300 synthetic cohort testing whether compact typed DOM projections and deterministic pre-execution gates keep 0.5B–3B models safe without frontier token budgets.

v1.0.0 PREPRINT5/5 BITCOIN OTSFILTERS · DOES NOT REPAIR

Question

When small local language models (0.5B–3B parameters) interact with web interfaces, how much invalid-action risk and observation token overhead can be eliminated purely by enforcing a typed observation/action boundary?

Method

Five preregistered experimental stages anchored to the Bitcoin blockchain via OpenTimestamps (.ots) prior to evaluation, comparing raw accessibility-tree prompts against compact typed projections and pre-execution schema/affordance gates across BrowserGym/MiniWoB++ and an N=300 adversarial synthetic cohort.

Evidence

Tracks the reference agent across 0.5B–3B model scales using 3.1×–3.8× fewer observation tokens while executing 0/300 invalid actions in the gated cohort (versus 152/300 invalid actions executed unassisted).

Boundary retained

Reported plainly as a safety filter, not a capability amplifier: a deterministic pre-execution gate blocks malformed or out-of-scope actions before side effects occur, but it does not repair a model's underlying task plan when semantic reasoning fails.