Skip to main content

Is Recurrent State a New Attack Surface?

· 7 min read
p4r4d0xb0x
p4r4d0xb0x
Rustacean, AI, OSS Enthusiast

cover placeholder

In Transformer systems, security discussions have centered on prompts, tool arguments, retrieved documents, and output filters. Operating recurrent or SSM-family systems adds another question: who creates the hidden state passed to the model’s next input, how long is it retained, and across which boundaries is it reused?

This question does not mean that “recurrent state is inherently vulnerable.” The mere existence of state does not establish an attack. But if state persists across sessions, is updated by mixing inputs with different trust levels, or affects authorization decisions for tool calls, it should be treated as both a new security asset and an attack surface. The papers and projects below address different layers, so they should not be equated directly.

Turning state poisoning into a threat model​

Term explainer: state poisoning

A threat in which an attacker plants malicious or misleading information in internal state through input or stored records, then attempts to steer the model’s behavior in a desired direction on later, otherwise normal inputs.

Term explainer: provenance

Information that tracks where data came from and which transformations it underwent. In security, it is important for preventing data from different sources from being treated with the same level of trust.

If a general recurrent update is written as h_t = f(h_{t-1}, x_t), there are three possible attack points. First, when malicious x_t is recorded into the state. Second, when contaminated h_t is used for a later decision. Third, when the state crosses to another user, session, or tool. This model does not prove an attack mathematically, but it provides a checklist for design review.

What Mamba’s selectivity implies for security​

Mamba proposes a selective SSM that adjusts state-space parameters according to the input, selectively propagating or forgetting information. The paper abstract claims efficient sequence modeling and content-dependent state updates, not a security mechanism.

What the source says: Mamba targets linear scaling and input selectivity on long sequences and reports performance across multiple tasks.

Security interpretation: Selective updates are not an authentication mechanism that distinguishes useful information from information planted by an attacker. Instead, because there is now a function deciding what remains in the state, there is greater reason to audit and validate state updates. We should check which tokens significantly change the state, whether state controls sensitive tool calls, and whether it is initialized at session boundaries. The Mamba paper alone cannot support a claim that a specific poisoning attack succeeds; that part is unverified.

Hidden State Poisoning: evidence that requires direct verification​

“Hidden State Poisoning” is a designated source directly related to this article’s threat model. The arXiv abstract reports HiSPA and the RoBench-25 evaluation, in which a short trigger contaminates hidden states in Mamba-family models and weakens information retrieval. However, details such as the attack procedure, success rate, and scope of vulnerability across specific models require additional review of the paper’s body and experiments, so this article treats them only cautiously.

What can safely be said beyond the verified scope is a conceptual question. If external input accumulates in a hidden state and later affects decision-making, input filtering alone may not be sufficient. Once the state itself is trusted, an attacker gains persistence that can affect multiple later steps rather than just a single input. This is not a reproduction of the paper’s result, but an analytical hypothesis applying the threat indicated by its title to recurrent systems.

When a precise source is available, the items to verify include the method of state access, outputs observable to the attacker, whether the state is initialized, attack persistence, and the baselines used in defense experiments.

Agent security context: how trajeckt handles state​

trajeckt is described as a gateway that, rather than allowing each agent tool call independently, checks pre-sealed allowed trajectories and data flows during execution. The repository documentation gives a database-read → summarize → external-transfer flow as an example, blocking the final exfiltration in a multi-step sequence.

This design does not claim to inspect recurrent hidden state itself. It does, however, provide an important security principle: manage trajectories and provenance outside the agent instead of trusting only the agent’s reported current context or internal plan. Even if recurrent state is poisoned, limiting tool permissions and data sinks through external policy can reduce the blast radius.

Limitation: As the trajeckt documentation states, semantic attacks, attacks hidden inside allowed causal paths, and issues that cannot replace existing RBAC and argument validation remain. Trajectory enforcement must therefore not be interpreted as proof of hidden-state integrity.

Valmis: trust boundaries created by isolation and proxies​

The Valmis repository emphasizes container isolation, credential protection through a proxy, agent memory and embeddings, and multi-step workflows. Here, memory is described not as recurrent hidden state but as an external memory layer using pgvector and embeddings.

This distinction matters for security. External memory permits the design of search and write permissions, database audit logs, tenant separation, and deletion policies. Runtime hidden state, by contrast, lives inside the model’s computation graph, making the same forms of access control and forensics difficult. Valmis’s proxy prevents the agent from directly seeing raw credentials, but the proxy and host become new trust centers—an existing limitation noted in the earlier article.

Boundary between source and interpretation: The Valmis README describes a security-oriented architecture but does not prove an external security audit or defense against hidden-state poisoning. This article does not exaggerate product features into evidence of attack prevention.

Operational checklist for a “new attack surface”​

  1. Lifetime: Is state reused across requests, users, or tenants? The safer default is to initialize it at session boundaries.
  2. Provenance: Do trusted system events and user text enter the same update path? Provenance should be recorded separately.
  3. Authorization: Does state directly determine tool selection, external writes, or credential use? If so, do not authorize based on state alone; add an external policy.
  4. Observability: Can the magnitude, cause, and timing of state changes be recorded? Logging the raw state can create a new leak of sensitive information, so consider hashes, statistics, and sampling.
  5. Recovery: If contamination is suspected, can the state be discarded and regenerated from a verified checkpoint?
  6. Separation from external memory: Are records in a search database and ephemeral hidden state being treated as if they had the same trust level?

Conclusion and limitations​

Recurrent state does not automatically become a vulnerability, but persistence, opacity, and influence over authorization provide sufficient reason to model it as a new attack surface. Mamba demonstrates the technical possibility of selective state updates but provides no security guarantee. Hidden State Poisoning points to a direct threat hypothesis, although its precise bibliography and experiments are not verified in this draft. trajeckt provides a context for limiting harm through trajectory and data-flow controls outside the agent, while Valmis illustrates trust boundaries involving containers, proxies, and external memory.

The practical conclusion is therefore not the simplistic prescription to encrypt all state. It is to separate the state’s lifetime and provenance, revalidate sensitive actions through external policy, and build a structure in which contaminated state can be discarded and recovered. In particular, the attack details and model-specific reproducibility of Hidden State Poisoning, as well as Valmis’s real-world operational security, remain unverified until the original sources and code are reviewed further.

Sources​

// COMMENTS

Comments