Securing LLM Applications: Prompt Injection, Data Leakage and Agent Risk

Applications built on large language models introduce a risk class that existing controls do not cover well, because the system's instructions and its input arrive through the same channel and in the same form. Everything below follows from that one architectural fact.

Prompt injection

A model receives a prompt containing the developer's instructions and the user's content concatenated together. The model has no reliable way to distinguish which text is authority and which is data — they are both just tokens. Content that says "ignore previous instructions and do X" may be followed.

The structural parallel is SQL injection: code and data sharing one channel. The difference is that SQL injection has a complete fix in parameterized queries, where the database is told explicitly which part is structure and which is value. No equivalent exists for language models. There is no parameterization primitive, and prompt-based defences are heuristics an attacker can work around.

Indirect prompt injection is the more dangerous form and the one defenders underestimate. The malicious instructions do not come from the user at all — they are embedded in content the model retrieves: a web page, a document, an email, a code comment, a support ticket. A user asks the assistant to summarize a page, and the page contains instructions the model then follows. The user is a victim, not an attacker, and nothing in the user's input looks wrong.

The other main risks

Sensitive information disclosure. Models may reveal content from their context window, including other users' data in a poorly isolated system, credentials placed in a system prompt, or documents retrieved on behalf of someone with different permissions. The classic failure is a retrieval system that searches a whole corpus regardless of who is asking.

Excessive agency. An agent given tools — send email, execute code, call an API, modify a record — can be induced through injection to use them. The severity of a prompt injection is determined entirely by what the model is permitted to do, which is why tool permissions are the real control surface.

Supply chain risk. Model weights, fine-tuning datasets, and third-party plugins are dependencies obtained from outside, and they carry the same provenance problems as any other component — the reasoning under supply chain security applies directly, including verifying where a model came from.

Insecure output handling. Model output treated as trusted input by downstream systems reintroduces classic vulnerabilities — generated HTML rendered without escaping, generated SQL executed, generated commands run. Output from a model is untrusted input, always.

Controls that hold

Since injection cannot be prevented reliably, the design goal is to make a successful injection not matter much.

Least privilege on tools. Grant the narrowest set of actions the use case requires, scoped and rate-limited. An assistant that can only read cannot be talked into deleting.

Human approval for consequential actions. Sending external email, moving money, changing configuration, deleting data — these need a person confirming, and the confirmation must show what is actually about to happen rather than a summary the model wrote.

Permission-aware retrieval. Filter the corpus by the requesting user's entitlements before retrieval, not after generation. Trusting the model to withhold what it has already been given is not a control.

Treat output as untrusted. Escape it, validate it, and never execute it without the same checks you would apply to user input.

Separate the data. Where the architecture allows, keep untrusted retrieved content in a different processing path from instructions, and do not give a model that reads untrusted content the tools that would make injection valuable.

Input and output filtering, and a WAF in front of the application, are worth having as defence in depth. Both are pattern matching against an adversary who can rephrase freely, so neither is a boundary to rely on.

Monitoring

Log prompts, retrieved context, tool invocations, and outputs, with the caveat that these logs now contain whatever sensitive data flowed through the system and need protection accordingly. Alert on tool use outside normal patterns, on refusals clustering around one user, and on retrieved content containing instruction-like text. Feed it into your SIEM like any other application telemetry, and red team the deployment specifically — conventional application testing does not cover this.

Exam angles

Know prompt injection as instructions and data sharing one channel, with indirect injection arriving through retrieved content. Recognize sensitive information disclosure, excessive agency, insecure output handling, and model supply chain as the companion risks. Understand that no complete preventive fix exists, so least privilege on tools, human approval for consequential actions, permission-aware retrieval, and treating output as untrusted are the controls. The severity of an injection equals the privilege of the agent.