Attacking Embeddings
Developers often believe embeddings work like hashes; one-way functions that can't be reversed. This leads them to relax security, reasoning that "even if an attacker captures the embeddings, they can't do anything with them". That assumption is wrong. Embeddings are designed to preserve semantic meaning, and where meaning is preserved, information can be recovered.
Extract script from Weaviate
The following script extracts all embeddings from a Weaviate database that has no authentication:
import requests
import numpy as np
import pandas as pd
from pathlib import Path
from tqdm import tqdm
WEAVIATE_URL = "http://localhost:8080/v1/graphql"
COLLECTION = "DocChunk"
PAGE_SIZE = 200
OUT_DIR = Path("./export")
OUT_DIR.mkdir(exist_ok=True)
def gql(query: str):
resp = requests.post(WEAVIATE_URL, json={"query": query})
resp.raise_for_status()
data = resp.json()
if "errors" in data:
raise RuntimeError(data["errors"])
return data["data"]
print(f"Connected to Weaviate via raw HTTP. Collection: {COLLECTION}")
count_query = """
{
Aggregate {
DocChunk {
meta {
count
}
}
}
}
"""
count_res = gql(count_query)
total = count_res["Aggregate"]["DocChunk"][0]["meta"]["count"]
print(f"Total objects: {total}")
all_ids = []
all_chunk_ids = []
all_vectors = []
cursor = None
fetched = 0
pbar = tqdm(total=total, desc="Exporting embeddings")
while True:
after_clause = f'after: "{cursor}"' if cursor else ""
query = f"""
{{
Get {{
{COLLECTION}(
limit: {PAGE_SIZE}
{after_clause}
) {{
chunk_id
_additional {{
id
vector
}}
}}
}}
}}
"""
res = gql(query)
objs = res["Get"][COLLECTION]
if not objs:
break
for obj in objs:
all_ids.append(obj["_additional"]["id"])
all_chunk_ids.append(obj.get("chunk_id"))
all_vectors.append(obj["_additional"]["vector"])
cursor = objs[-1]["_additional"]["id"]
fetched += len(objs)
pbar.update(len(objs))
if fetched >= total:
break
pbar.close()
vectors_np = np.array(all_vectors, dtype=np.float32)
np.save(OUT_DIR / "embeddings.npy", vectors_np)
np.save(OUT_DIR / "chunk_ids.npy", np.array(all_chunk_ids))
np.save(OUT_DIR / "uuids.npy", np.array(all_ids, dtype=object))
df_meta = pd.DataFrame({"uuid": all_ids, "chunk_id": all_chunk_ids})
df_vec = pd.DataFrame(
vectors_np,
columns=[f"dim_{i}" for i in range(vectors_np.shape[1])]
)
df_full = pd.concat([df_meta, df_vec], axis=1)
df_full.to_csv(OUT_DIR / "embeddings.csv", index=False)
df_full.to_parquet(OUT_DIR / "embeddings.parquet", index=False)
print("Vectors successfully exported!")The script creates the export directory with one of the files being embeddings.npy. This file contains the raw numerical representations of the document chunks, which we'll attempt to invert back to text. The methods we can use depend on time, resources, knowledge of the embedding model, and access to the embedding model's weights.
Identifying the embedding model
Since each embedding model has its own characteristic dimensionality, this may give us insight into the model being used.
A option is to perform fingerprinting via the dimensionality. Let's use the following script to obtain the dimensionality of the embeddings:
Unfortunately, many models share the same dimensions. Therefore, we can only use this as a first indication for common dimensions, such as 384 and 768. The 384-dimensional space narrows our possibilities significantly, but we still need more information to be certain.
Inference Probing
Manually querying a RAG system and eyeballing which model might match would be tedious and error-prone, so the script automates the entire workflow with several key features.
Example output:
Triaging Chunks
A production vector database may contain thousands or tens of thousands of chunks. Running a full inversion attack against every chunk is not practical. Each chunk takes minutes to attack, and most chunks in a typical enterprise knowledge base contain mundane content: HR policies, meeting notes, product documentation. The chunks worth attacking are the small fraction that hold credentials, API keys, or other secrets. We need a way to find those chunks before committing resources.
Chunk triage solves this by working entirely in embedding space, without needing to invert anything first. The core idea is simple: if a chunk contains a password reset message, its embedding will be similar to other texts about password resets. We define banks of sensitivity probes, short texts that describe or resemble sensitive content like credentials, API keys, SSH keys, PII, and financial data. Each probe is embedded with the same model, and we compute cosine similarity against every chunk. High scores against credential probes mean the chunk likely contains credentials.
The chunk_triage_pipe.py script chains three stages into a cascading pipeline, each narrowing the candidate pool before the next one runs.
https://gist.github.com/1ncendium/9e73f9aeda9a4ba6a75d8b00388d55eb
The output shows that from 31 chunks, the top 3 fused scores cluster above 0.071, while the fourth drops to 0.068, a gap that marks a natural cutoff. Each of these three chunks likely contains a credential such as a password. In a real-world engagement, this score gap would guide us as red teamers to focus embedding inversion on just these three targets rather than the full database.
To extract the chunk we can use the following one-liner:
Zero-shot embedding inversion
embedding inversion fundamentally struggles with high-entropy tokens like passwords, API keys, and random strings. These values have no semantic predictability, so neither an LLM nor a gradient-based optimizer can reconstruct them. The embedding space captures "this is a password field" but cannot distinguish between Spring2026! and GovPass123! because both are semantically equivalent as passwords.
The practical solution is to separate the problem into two stages. First, we need to recover the semantic structure of the text (the sentence patterns and context surrounding the secrets) while replacing high-entropy values with placeholders like {PASSWORD} or {URL}. Second, we'll use membership inference to fill in those placeholders: for each slot, iterate through a wordlist, embed the template with each candidate value, and measure which candidate produces the highest cosine similarity against the target embedding. The correct value will match the exact token sequence in the original chunk, producing a measurably higher similarity than any incorrect candidate.
With the template bank ready, let's also download a password wordlist for the membership inference stage:
Now we run emb_fin.py, the unified pipeline that scores the template bank, selects diverse high-scoring templates, and runs membership inference with consensus voting. A straightforward implementation of this idea would score all templates, take the top 20, and run membership inference against each, but this produces unreliable results because many top-scoring templates are near-duplicates that inflate consensus artificially.
The output shows that the password N0=Acc3ss was identified, with 18 out of 20 templates agreeing. The --default-URL flag provides the known URL value so the script can fill that slot with accurate context rather than a generic placeholder, improving signal quality for the PASSWORD slot.
Last updated