ChatGPT can remember lies: Researchers discover dangerous vulnerability
GhostWriter tricks AI into forwarding other people's data (Photo: Pexels)
Researchers have discovered a new way to attack ChatGPT and other AI services through their long-term memory feature. The technique, called GhostWriter, allows attackers to quietly poison stored memories and influence how an AI chatbot behaves in future conversations, according to a reference to a scientific paper published on the arXiv preprint server.
How GhostWriter targets AI's long-term memory
The researchers explain that modern personal AI agents combine the capabilities of a conversational chatbot with task execution, such as sending messages, scheduling meetings, or editing code.
When these systems interact with untrusted information sources, they become vulnerable to memory poisoning attacks.
In their paper, researchers George Torres, Sharad Shrestha, and Satyajayant Misra describe how GhostWriter works. The technique involves planting false memories or hidden instructions inside an AI model's long-term memory.
The attack consists of two stages:
- Injection: The attacker secretly inserts a malicious payload into the target AI agent.
- Activation: The poisoned memory is later retrieved from storage when the AI processes a legitimate user request.
One example described by the researchers involves an AI assistant that manages email. The attacker implants a hidden instruction telling the model to regularly summarize emails from banks or financial institutions and secretly forward those summaries to the attacker's email address.
Experiments showed that GhostWriter achieved an injection success rate of nearly 98% and an average activation rate of about 60% against modern AI agents.
LLM agents without memory process every interaction in isolation. LLM agents with persistent memory can store information and retrieve it later when needed (Illustration: George Torres, Sharad Shrestha, and Satyajayant Misra)
How can the threat be stopped?
According to the researchers, the vulnerability exists because AI systems lack security-focused memory management. To address the issue, they developed a defense system called Agentic Memory Sentry (AM-Sentry).
AM-Sentry relies on two key mechanisms:
- Memory-saving policy: Controls how new information is written to long-term memory.
- Memory-retrieval screen: Checks stored instructions immediately before they are retrieved and executed.
According to the team's experiments, AM-Sentry significantly reduces the success rate of GhostWriter attacks while preserving the AI agent's overall functionality and performance.
The researchers believe that these protection strategies could be adapted to improve the security of other AI systems with long-term memory.
