Attacking RAG-pipelines
Large Language Models are powerful, but they have a fundamental limitation: their knowledge is frozen at the moment they are trained. They cannot reliably access new information, organization-specific data, or rapidly-changing facts unless we perform costly retraining or fine-tuning. Retrieval-Augmented Generation (RAG) solves this problem by injecting external knowledge into the model at runtime without modifying the model itself. This makes LLMs far more adaptable, accurate, and aligned with real business needs.
Knowledge Base Leakage
When sensitive data is stored within an RAG, we can try to extract it by probing the AI-agent.
When gathering information from RAG systems, precision in our prompts is everything.
Example:
Ingestion Poisoning
Ingestion Poisoning occurs when we upload a document to poison the knowledge base during ingestion, so that future retrievals return poisoned or misleading content. This can provide us with sensitive information or even system access, depending on what information we plant in the knowledge base.
Password reset example
We can also upload files that the AI will use to update its knowledge base:
We upload the following prompt
After it ingested the uploaded file, we can now ask again how to reset the password:
Retrieval Hijacking
poisoning and embedding collision, we needed to identify and target frequent queries. With retrieval hijacking, we actually want the opposite. We want to embed our instructions in documents that only match specific queries that we control. This way, our malicious instructions won't be flagged by automated systems or noticed by regular users. They'll only be triggered when we deliberately query for them.
We can think of it this way: instead of poisoning a well-traveled road that everyone uses, we're creating a hidden path that only we know about.
input filters often only apply to user input, not to retrieved context. When we upload a document and it gets ingested into the knowledge base, its contents become part of the retrieved context that's added to the model's prompt. The model treats this retrieved context as trusted internal data, not as potentially malicious user input.
Last updated