When a company's confidential data is processed by an AI system, what risks does that data face at every stop along the pipeline, and how can those risks be mitigated?

Compliance for RAG is thornier than for traditional databases because RAG inherently lacks an audit trail. A single query reads, retrieves, and summarizes across multiple sources within seconds, while regulations like GDPR and HIPAA require companies to account for where personal data is stored and how it's used. Once the system surfaces personal data in an answer where it shouldn't appear, demonstrating compliance after the fact is often extremely difficult. The following breakdown examines risks by the four "states" of data — these four states (at rest, in transit, in use, and residency) reflect generally accepted security concepts, though organizing them as a four-item checklist is this guide's framing, not an official classification from any compliance standard (such as ISO 27001 or SOC 2).

Three points in the table are worth calling out separately, because they are the most counterintuitive and the most likely to become buried risks:

"Deletion" is often an illusion. For performance reasons, vector databases default to "soft deletion" at the metadata layer — an API returning no error is taken as successful deletion, but the data may still physically exist. This is the same pain point as "weights are difficult to selectively delete," manifesting at a different storage layer — and GDPR Article 17 requires true, complete deletion.

Redaction isn't as simple as blacking things out. Naive redaction destroys semantics: if you simply strip out names or account numbers, the model also loses the information it needs to produce a useful answer. For example, "Customer ___ 's account ___ was double-charged" leaves the model with a broken sentence it can't properly process. The correct approach is context-preserving tokenization rather than brute-force removal: replace "John Smith" with [NAME_1] and the specific account number with [ACCOUNT_1], producing "Customer [NAME_1]'s account [ACCOUNT_1] was double-charged." The sensitive values are hidden from the model, but the role information — "there is a name here," "there is an account number here" — is preserved, so the model can still follow the sentence structure and produce a useful answer, instead of staring at a bunch of blanks.

Permissions are lost by default. Documents from Confluence, SharePoint, or an internal wiki — once converted into vectors, their original access controls no longer travel with them. This is precisely why "binding ACLs" must be done as an explicit additional step — it does not happen automatically.

None of this is alarmist. The next chapter switches to the attacker's perspective and walks through these risks again.

Two conclusions most worth remembering:

1. The single most sensitive action across the entire pipeline is the moment confidential content is assembled into a complete prompt and sent to a cloud-hosted model — this is the point at which data truly leaves the company's control.
2. The single most easily overlooked risk point is the logging system — all the attention goes to "is the database secure," while forgetting that the logs also contain a complete plaintext copy of every conversation.

Further reading: what does a managed cloud model (MaaS) actually retain?

MaaS (Model-as-a-Service) refers to cloud-hosted models accessed via network calls, as opposed to self-hosted deployments. What it actually retains is a key compliance question:

• Weights are read-only: the content of your queries does not get written into the model's parameters or change the model itself. "The model has its own storage" is a common misconception — unless the provider deliberately uses conversations for subsequent training, which is a separate matter.
• Prompt caching is an easily overlooked short-term landing point: if the provider enables cross-request prompt caching (see Section 3.3), a portion of the request content resides briefly in their infrastructure in the form of KV cache. This is not the same as training or long-term retention, but strictly speaking, inference is not entirely stateless.
• What is actually retained long-term is logs (complete Q&A plaintext), abuse-detection records, and (when the contract allows) data used for training — whether it's retained, and for how long, depends on contract terms and technical settings.
• ZDR (Zero Data Retention): a contract clause that, once signed, disables the persistent retention described above. But you should verify whether it covers infrastructure-level short-term residency like prompt caching, to avoid leaving a gap. OpenAI's documentation notes that, at least for some models, organizations with ZDR enabled default to keeping the cache in memory only, without the extended 24-hour retention option.³

Take OpenAI as an example: its API has defaulted to not training on customer data since March 2023, but every API call still generates "abuse-monitoring logs" that may include prompts and responses, retained for up to 30 days; excluding this requires applying for Zero Data Retention and getting OpenAI's advance approval.⁴ So "won't train on your data" and "won't retain your data" are two separate questions — ask about them separately when evaluating a provider.