prompt-injection

One threat we’re particularly focused on is prompt injection, and we handle it in layers. The model is trained to recognize and resist it. The harness marks anything coming from an untrusted source. Deterministic code checks the result. And an ensemble of classifiers runs where the agent can't reach them.

By enforcing capability-based security policies,
CaMeL effectively prevents unintended data
flows and actions. Even with a compromised
cloud storage file containing prompt injections
targeting the Quarantined LLM, the capabilities system acts as a robust and granular control mechanism, preventing data exfiltration, as
well as other prohibited actions. This approach
provides a secure environment for LLM agents,
preventing harm by design, without requiring
modifications to the underlying LLM itself, and
offering a significantly more fine-grained and
robust defense compared to isolation and/or
adversarial training strategies alone.