Definition: what is prompt injection?
Prompt injection is an attack in which instructions are inserted into material an AI application processes, attempting to redirect it away from its intended task. The malicious text may appear in a user's message or inside a document, web page, retrieved passage or tool response. Its purpose is to influence behaviour, not merely to provide information.
This distinction matters because language models process instructions and ordinary text in closely connected ways. A sentence inside a document can resemble a command even though the document has no authority to command the application. A secure workflow must preserve the boundary between trusted instructions and untrusted content rather than assume the model will always recognise that boundary itself.
What authoritative guidance says
The OWASP Top 10 for LLM Applications, 2025 edition lists prompt injection as LLM01 and distinguishes direct and indirect forms. Direct injection arrives through user input. Indirect injection arrives through external material the application encounters, including retrieved content. The entry describes prevention and mitigation, not a numerical guarantee of safety.
OWASP also treats excessive agency as a separate risk. That connection is important: an assistant that only drafts text has different consequences from one allowed to read private files, send messages or change records. An injection becomes more damaging when the system has powerful permissions and weak checks around actions.
No attack-success percentage is claimed here. Effectiveness depends on the model, inputs, tool permissions, application design and test conditions. A dramatic example from one system is not a reliable estimate for another. The examples below are fictional scenarios designed to explain the mechanism without targeting a real service.
Direct versus indirect prompt injection
Direct injection
A person interacting with an assistant might ask it to ignore its existing rules or reveal restricted information. The attacker controls the message sent to the application. For an employee, this often looks like an obviously inappropriate request. For developers, the harder issue is preventing such requests from changing what tools or information the caller may access.
Indirect injection
An assistant may be asked to summarise a web page or search a knowledge base. An attacker places misleading instructions inside the source material. The employee's request is legitimate, but the retrieved content is hostile. A model that mistakes source text for an authorised command may produce a manipulated answer or attempt an action the user never requested.
Why retrieval does not automatically solve it
Retrieval-augmented generation can supply relevant documents, but retrieval is not a trust guarantee. A source can be relevant to the topic and still contain malicious instructions. Even an internal document can include copied external text. Security depends on how the application handles that content, not simply where it stores it.
Example: an onboarding assistant reads a poisoned document
Imagine a fictional HR assistant that drafts onboarding checklists from approved policy documents. A malicious passage is added to a document that otherwise contains plausible information. The passage tells the assistant to replace the official help-desk contact with an unrelated address and to omit mention of the change.
The employee asks an ordinary question: create an onboarding checklist for a new colleague. If the assistant treats the passage as a command, its output may direct the colleague to the wrong destination. If it treats the passage as untrusted source content, it can flag the inconsistency or cite the approved contact without adopting the instruction.
Now consider a more capable assistant that can send onboarding emails automatically. The same mistake could become an external communication rather than a draft caught by a reviewer. This is why the organisation should constrain actions independently of model behaviour. A document should not be able to grant permission to send messages or change recipients.
Recognising suspicious source content
Potential warning signs include instructions to ignore earlier rules, impersonate an administrator, hide information from the user, reveal secrets or perform an unrelated action. Suspicious text may claim urgency or authority. It may also appear in comments, metadata or other fields that a reader does not normally notice but the application extracts.
Not every instruction-like sentence is malicious. A training manual legitimately contains instructions for its readers. The question is whether the content is trying to control the assistant beyond the user's authorised task. Employees should report suspicious material rather than rely on appearance alone; some attacks are not conspicuous to an ordinary reader.
Controls and what they can realistically do
| Control | Practical purpose | What it does not guarantee |
|---|---|---|
| Label external content as untrusted | Make the instruction boundary explicit | That the model always respects the label |
| Restrict tool permissions | Limit actions available to the assistant | That generated text is accurate |
| Validate actions outside the model | Enforce allowed recipients and operations | That every misleading draft is detected |
| Require approval for consequential actions | Let a person inspect the intended action | Safety if the reviewer rubber-stamps it |
| Test adversarial documents | Reveal weaknesses before deployment | Immunity to future attacks |
| Log and investigate failures | Support monitoring and incident response | Prevention without active follow-up |
These are complementary controls. A strong warning in a system prompt is useful, but it is not a substitute for an application refusing an unauthorised action. Likewise, an approved-source list reduces exposure without proving that every approved file is clean.
How to reduce prompt-injection risk at work
Step 1: identify what the assistant can read and do
List its inputs, connected systems and action permissions. Distinguish reading a document from editing a record or sending a message. Identify confidential information the assistant can access. The consequences of an injection depend on these capabilities, so start with a capability inventory rather than a clever defensive prompt.
Step 2: keep instructions separate from source material
Make clear that documents, search results and tool outputs are evidence to inspect, not authority to redefine the task. Applications should maintain that distinction throughout the workflow. Employees should avoid pasting unrelated instructions into the same field as trusted task requirements without identifying what is quoted source material.
Step 3: minimise permissions
Give the assistant only the access required for the approved task. A summarisation tool usually does not need permission to delete files or send external emails. Separate sensitive actions from routine drafting. Where practical, restrict destinations, record types and operations through checks that do not depend on the model deciding what is permissible.
Step 4: review proposed actions, not just prose
Before a consequential action, display the actual recipient, data and operation for approval. A reassuring explanation is insufficient if the underlying action differs. Reviewers need enough context to verify the request. If a system cannot show what it is about to do, that is a reason to reconsider automatic execution.
Step 5: test safely with synthetic content
Use an authorised sandbox and fictional documents. Include ordinary documents, suspicious passages and mixed cases. Check whether the assistant stays on task, flags conflicts and refuses unauthorised actions. Do not test by exposing real credentials or sending information to an external destination. Record failures so changes can be retested consistently.
Step 6: monitor changes and incidents
Repeat relevant tests after changing the model, tools, retrieval source or permissions. Give employees a clear reporting channel for unexpected behaviour. Preserve appropriate logs while respecting privacy rules. A new model version or connector can change the risk even when the visible assistant interface looks unchanged.
Step 7: define a stop condition
Document when the workflow must pause for security review. Examples include an attempted unauthorised send, a changed destination or unexpected access to private material. Do not continue simply because the final answer looks plausible. Investigate the attempted action and limit capabilities while the issue is assessed.
A practical employee checklist
Before using an AI summary to act, verify the source, destination and requested action. Ask whether the output includes instructions absent from the legitimate task. Check important contacts against an approved directory rather than a generated answer. Keep confidential data in approved tools and escalate suspicious content to your security team.
For HR, particularly sensitive workflows include candidate records, employee documents and onboarding communications. An assistant may help draft material, but people and application controls should retain authority over disclosures and decisions. See AI governance for assigning an owner and setting permitted uses.
Frequently asked questions
Is prompt injection the same as a jailbreak?
The concepts overlap, but they are not identical. A jailbreak generally aims to bypass model safeguards. Prompt injection redirects an application's intended behaviour through input or source content. Some attacks do both. For a workplace team, focus on the boundary violated and the actions possible rather than the label alone.
Can a prompt prevent every injection?
No prompt should be treated as a universal security guarantee. Clear instructions can help, but permissions, independent validation, review and adversarial testing are essential. The system must remain safe even when the model generates an unexpected response.
Is an internal knowledge base automatically safe?
No. Internal sources can contain copied content, compromised edits or instructions inappropriate for the assistant. Access controls and editorial review help, but retrieved material should still be handled as source data rather than authority to change application rules.
Key takeaways and sources
Prompt injection exploits confusion between instructions and content. Its practical impact grows with the assistant's permissions. Reduce risk through clear boundaries, restricted capabilities, independent action checks and regular testing. Employees should verify consequential outputs and report unexpected behaviour.
Primary references: OWASP LLM01: Prompt Injection and OWASP LLM06: Excessive Agency. Continue with AI agents and human oversight.