Multi-tenant RAG: keeping one customer's data out of another's answers

Three isolation models

There are broadly three ways to keep tenants apart, in decreasing order of isolation and cost.

• —A database per tenant — strongest isolation, and an operational burden that grows linearly with customers. Migrations become a fleet operation.
• —A schema or namespace per tenant — middle ground. Cleaner than shared tables, but per-tenant index management and thousands of namespaces bring their own problems.
• —Shared tables with a tenant column — one schema, one migration, one index. Isolation depends entirely on every query being correctly filtered, which is why it needs enforcement rather than discipline.

Shared tables, done so the filter cannot be forgotten

We use shared tables, because the operational simplicity is substantial and the isolation weakness is fixable. The fix has two parts.

First, row-level security in the database. Policies mean Postgres itself refuses to return rows belonging to another tenant, so a query missing its filter returns nothing rather than everything. Application code becomes the second line of defence rather than the only one.

Second, retrieval lives inside a function that takes the tenant ID as a required parameter. There is no callable path that performs a similarity search without scoping, because the function signature will not permit it.

create or replace function match_chunks(
  p_workspace_id uuid,
  p_query_embedding vector(1536),
  p_match_count int default 6,
  p_min_similarity float default 0.0
)
returns table (id uuid, content text, metadata jsonb, similarity float)
language sql stable as $$
  select c.id, c.content, c.metadata,
         1 - (c.embedding <=> p_query_embedding) as similarity
  from chunks c
  where c.workspace_id = p_workspace_id
    and c.embedding is not null
    and 1 - (c.embedding <=> p_query_embedding) >= p_min_similarity
  order by c.embedding <=> p_query_embedding
  limit p_match_count;
$$;

The dangerous path: service-role clients

A public widget endpoint has no logged-in user, so row-level security has no session to evaluate. These endpoints necessarily run with a service-role connection that bypasses RLS — which removes exactly the safety net described above.

This is the highest-risk code in a multi-tenant RAG product and it should be treated that way. The tenant must be resolved from the request's own credential — for us, the widget's public key maps to a workspace — and then every subsequent query scoped to that resolved ID explicitly.

It is worth marking those files clearly, keeping them few, and reviewing them differently from the rest of the codebase. Every query in them is load-bearing.

Isolation is not just the vector table

The chunks table gets the attention, but a leak from any adjacent table is the same incident.

• —Conversations and messages — scope by tenant and by visitor, or one visitor's conversation ID lets them read or append to another's transcript
• —Unanswered questions — these contain verbatim customer text
• —Usage counters and billing state — cross-tenant reads here are a commercial leak rather than a personal one
• —Tool configuration — endpoints and encrypted credentials belonging to one customer
• —Cached embeddings or retrieval results, if you add a cache, keyed per tenant

One index, many tenants

A single HNSW index over all tenants' vectors with a tenant filter applied at query time performs well and avoids per-tenant index management entirely. New workspaces need no setup, which matters when they arrive continuously.

This is also a practical argument against IVFFlat in a multi-tenant product: its clustering wants representative data at build time, which sits awkwardly with a corpus that grows tenant by tenant. HNSW has no such dependency.

Test the boundary explicitly

Isolation is not something to verify by inspection. Write tests that would fail loudly if it broke.

• —Seed two tenants with deliberately distinctive content, query one, assert nothing from the other appears
• —Call the public endpoint with tenant A's key and tenant B's conversation ID; assert it is rejected
• —Exercise the service-role paths in tests, not just the RLS-protected ones — the unprotected paths are the ones that matter
• —Assert the retrieval function rejects a null tenant rather than defaulting to unscoped