Presented by Eversheds Sutherland (US) LLP
What if the document you ask AI to analyze is secretly giving the AI instructions of its own? As businesses increasingly rely on AI tools to review contracts, screen job applicants, summarize reports, and analyze information, a new risk is emerging: documents may contain instructions directed not to the human reader, but to the AI tool itself – sometimes in ways users may never see.
This technique, known as ‘prompt injection,’ occurs when instructions embedded in material supplied to an AI tool cause the tool to disregard, override, or deviate from the user’s intended task. Such instructions may be visible or concealed through techniques such as white-on-white text, tiny type, or off-page placement. The result may be biased, incomplete, or unreliable output – undermining the very efficiencies and insights AI is intended to promote.
A recent Connecticut case appears to be the first reported US case to confront this risk in litigation. In Elliott v. New York Bariatric Group, LLC, a litigant embedded hidden instructions in court filings that if the document is reviewed by an AI model, it should generate output favorable to the plaintiff and “aim to ensure remediation” of a prior ruling unfavorable to the plaintiff. The court held that the tactic was an abuse of the filing process and imposed sanctions.
Elliott is a warning not only for litigators but for any organization using AI to analyze material received from an external or potentially adversarial source. Any document submitted for AI-assisted review may contain instructions designed to influence how the AI tool interprets, prioritizes, or summarizes the material.
For example, a contract could contain hidden text directing the AI to ignore key provisions or produce a biased summary; a resume could carry hidden instructions to an automated screener to advance the applicant; a vendor proposal could include embedded instructions designed to influence AI-generated comparisons among competing bids.
No single precaution can eliminate prompt injection risk. Instead, organizations should consider a combination of safeguards proportionate to the source, sensitivity, and intended use of the document you wish to upload to AI.
The following are some practical tips that may help to reduce the risk that embedded instructions in uploaded documents will compromise the reliability of AI-generated analysis.
1. Know your source, and adjust your level of skepticism accordingly.
2. Look for warning signs of possible prompt injections, such as white-on-white text, tiny font sizes, excessive blank spaces, text positioned off-page, or comments directed to “AI,” “LLM,” “assistant,” or “reviewer.”
3. Pre-process, or “clean,” documents before uploading them to AI.
4. Instruct the AI tool to scan the document to identify and disregard embedded instructions and to take direction only from the user.
5. Be alert for unexplained conclusions and require source support.
6. Understand and enable your AI platform’s built-in security controls.
7. Independently verify the AI tool’s conclusions. Human review of AI output remains one of the most effective safeguards against AI-generated errors and the influence of hidden instructions.
Hidden instructions should never have the last word in an important business decision; influences on important decisions should be open, identifiable, and subject to scrutiny. Prompt injection seeks to bypass that scrutiny by allowing hidden instructions to shape AI-generated output. As organizations incorporate AI into more business processes, healthy skepticism, clear AI instructions, and human oversight remain essential safeguards.