checklist
AI chatbot retrieval access control and tenant isolation checklist for small teams
A practical checklist for enforcing identity and tenant boundaries in AI retrieval, connectors, indexes, caches, citations, exports, and downstream actions.
Use this checklist when an AI chatbot can retrieve internal documents, customer records, tickets, files, messages, or connected application data. It is for retrieval-augmented generation (RAG), search-backed assistants, support bots, AI copilots, and agent routes that use a vector index, keyword search, connector, cache, or export path.
The short answer: authorize the user, tenant, role, resource, action, and destination before retrieval; carry the approved context through every search and connector call; filter results again before they reach the model; prevent caches, citations, logs, exports, and tools from crossing the same boundary; and test denied cases deliberately. The model should never be the component that decides whether a user may see or change a record. If identity, tenant scope, source permissions, or filter behavior cannot be proven, keep the route read-only, limited, draft-only, or paused.
Start with the AI Tool Risk Checker and attach its result to the access review. Use the Small Team AI Security Checklist for baseline identity, data, access, logging, and incident controls. Pair this page with the AI chatbot answer provenance and citation traceability checklist for source and claim evidence, the AI chatbot permission drift review checklist for recurring access changes, and the AI chatbot output validation checklist for safe destination handling.
Bottom line
Do not treat a successful login or a well-written system prompt as tenant isolation. A defensible retrieval boundary can show:
- Which authenticated subject, tenant, role, device or session context, route, and purpose were authorized.
- Which source, document, record, connector, index, and field scopes were allowed for that context.
- That authorization was enforced in application code or a policy gateway before search and again before model context assembly.
- That unauthorized results do not leak through ranking metadata, snippets, citations, caches, logs, errors, exports, or tool arguments.
- That indirect prompt injection or malicious content in a retrieved document cannot change the access decision or trigger a privileged action.
- That denied, stale, ambiguous, and failed authorization paths are tested and produce a safe result with an owner and evidence record.
NIST SP 800-207 describes zero trust as removing implicit trust based on network location and performing authentication and authorization before access to a resource. OWASP LLM01:2025 notes that RAG does not fully mitigate prompt injection and recommends least privilege, code-enforced controls, and human approval for high-risk actions. OWASP LLM02:2025 treats PII, financial data, confidential business data, credentials, and legal documents as sensitive information and recommends strict access controls and secure runtime orchestration. This checklist applies those ideas to the retrieval path; it is not a proof of isolation or a compliance certification.
This is operational guidance, not legal advice, a compliance certification, or a security assurance opinion.
When to use this checklist
| Situation or change | Use this checklist? | Minimum review |
|---|---|---|
| New internal knowledge assistant | Yes | Identity, tenant, source permissions, retrieval filters, denied tests, and release decision. |
| New customer or multi-tenant chatbot | Yes | Tenant key propagation, cross-tenant negative tests, citation checks, and export review. |
| New connector to a CRM, helpdesk, drive, wiki, or mailbox | Yes | Connector scope, delegated permissions, sync behavior, deletion, and user-level filtering. |
| New vector index, parser, chunker, or ingestion job | Yes | Metadata propagation, ACL preservation, stale content, and post-ingestion boundary tests. |
| New cache, transcript, analytics, or observability sink | Yes | Cache keys, retention, access control, redaction, and tenant-aware query tests. |
| New model, prompt, agent, or tool | Yes | Treat retrieved text as untrusted input and retest action authorization. |
| Permission, role, plan, or tenant change | Yes | Re-run allow and deny cases, including already indexed and cached content. |
| Incident, complaint, suspicious citation, or unexpected result | Yes | Preserve a redacted evidence record, contain the route, investigate, and re-test. |
| Synthetic-data prototype with no connected source | Maybe | Record the boundary and the conditions required before production data is enabled. |
If a team cannot state the identity and tenant context attached to every retrieval request, do not expand the route to new users or sources.
Retrieval access review intake form
Copy this form into a release, connector, source, permission, incident, or recurring review. Store sensitive evidence in an approved restricted system; keep this record to identifiers and decisions.
| Field | Entry |
|---|---|
| Review ID | |
| Review date and timezone | |
| Chatbot, route, and environment | |
| User and tenant scope | |
| Roles, plans, groups, and device context | |
| Purpose and supported intents | |
| Source systems and resource types | |
| Search, index, connector, and cache components | |
| Allowed fields and prohibited fields | |
| Model, provider, prompt, and tool versions | |
| Data classification in scope | |
| Export and downstream destinations | |
| Security and privacy reviewers | |
| Source and platform owners | |
| Release, pause, or rollback owner | |
| Restricted evidence location | Do not paste passwords, tokens, signed URLs, raw customer records, or private transcripts. |
| Next review trigger |
Assign stable review, request, and test IDs. Do not encode names, tenant identifiers, email addresses, or raw record keys into IDs that may appear in broad logs.
Data-flow and trust-boundary map
Draw or describe the path from the user request to each durable destination. The map should show where identity and tenant context are created, checked, transformed, and logged.
| Component | Trust question | Required control | Evidence |
|---|---|---|---|
| Client and session | Can the client choose or change tenant, role, or user claims? | Derive claims from authenticated server-side context. | Session and claim validation result. |
| API or chatbot gateway | Is route access checked before retrieval? | Authorize route, purpose, user, tenant, and action in code. | Gateway policy and denied test. |
| Query planner | Can user text become a filter or resource selector without validation? | Separate query intent from policy filters; validate fields and operators. | Query and policy test set. |
| Connector | Does the connector use broader credentials than the user? | Scope service credentials and enforce user or tenant filters. | Connector permission record. |
| Index and metadata | Are ACLs, tenant IDs, resource IDs, and classification preserved? | Reject or quarantine chunks missing security metadata. | Ingestion validation and sample rows. |
| Retriever and reranker | Can ranking expose restricted snippets or metadata? | Filter before ranking where possible and filter again before context assembly. | Query result boundary test. |
| Model context | Can instructions in source text override access or tool policy? | Mark retrieved text as untrusted data; keep authorization outside the model. | Injection and policy tests. |
| Cache and session memory | Can one key return another user or tenant’s content? | Include the full authorization scope in keys and invalidate on change. | Cache key and eviction test. |
| Output and citation layer | Can citations reveal a restricted title, path, snippet, or URL? | Re-authorize citations and redact or omit restricted references. | Citation review record. |
| Tool and export layer | Can an answer cause a cross-boundary read or write? | Validate arguments and independent authorization before the action. | Action audit and deny test. |
| Logs and analytics | Do traces contain raw prompts, results, or tenant data? | Minimize, classify, restrict, and test log access and deletion. | Log sample and access review. |
Mark every boundary where the trust level changes. A document retrieved from a connected system is data, not an authority to change identity, policy, permissions, or tool access.
Identity and tenant context matrix
Write down which server-side attributes are required for each route. Avoid accepting a tenant ID, role, account number, or source filter from the model or an untrusted client field.
| Context field | Where it comes from | Used by | Required validation | Failure behavior |
|---|---|---|---|---|
| Subject identity | Authenticated session or service identity | Route, connector, source, and action policy | Valid session, status, and audience. | Deny retrieval. |
| Tenant or workspace | Server-side membership and resource mapping | Index filter, source API, cache, citation, and export | Membership, active state, and exact scope. | Deny and alert if supplied scope conflicts. |
| Role and entitlements | Identity provider or application policy | Field and action authorization | Current role and effective permissions. | Use least privilege or deny. |
| Resource scope | Application policy and source ACL | Search and tool call | Resource exists and belongs to allowed tenant. | Return no result or safe error. |
| Data classification | Source metadata and policy | Retrieval, output, log, and export handling | Classification is present and allowed. | Quarantine or escalate. |
| Purpose or intent | Approved route and request context | Data minimization and policy decision | Intent is supported and within scope. | Ask for clarification or deny. |
| Device or session context | Authenticated request and risk signal | Sensitive route and export policy | Fresh session and required assurance. | Require step-up or deny. |
| Locale or region | Server-side profile or route | Source scope and response handling | Supported and contractually allowed. | Limit scope or hand off. |
Test combinations, not only individual fields. A valid user with a valid role may still be unauthorized for a different tenant, source, record, purpose, or action.
Source and index permission controls
- Every source has an owner, tenant model, permission model, classification, retention rule, and deletion path.
- The ingestion job retrieves only the source scope approved for the connector or service identity.
- Each chunk or indexed record carries the minimum metadata needed to enforce resource and tenant filters.
- Items without required tenant, ACL, or classification metadata are rejected, quarantined, or excluded from retrieval.
- Permission changes propagate to the index, cache, replicas, derived summaries, and exports within a defined window.
- Deleted or revoked items are removed from retrieval and checked with a post-deletion query.
- The index cannot be queried by a client that can supply arbitrary filters or resource identifiers.
- Source updates cannot silently broaden access by changing metadata or parser behavior.
- Service credentials are scoped to the minimum source and operation permissions.
- A source administrator cannot use a chatbot route to bypass application-level tenant or role checks.
- ACL failures are visible to operators without exposing the protected content in the error message.
Do not infer a source’s access policy from its title, folder name, URL, or natural-language instructions. Use the source system’s authorization model and a server-side policy decision.
Query, filter, and ranking controls
| Control | Safe default | Test evidence |
|---|---|---|
| Tenant filter | Derived server-side and applied to every retrieval path. | Same query under two tenants returns only each tenant’s fixtures. |
| Role filter | Derived from effective permissions, not prompt text. | Lower role cannot retrieve higher-scope fixture. |
| Resource filter | Allowlisted source and record types; no arbitrary field selectors. | Invalid selector is rejected without a data-bearing error. |
| Field filter | Exclude prohibited fields before model context. | Sensitive-field fixture never reaches context or citation. |
| Query normalization | Treat user and document text as untrusted input. | Encoded, multilingual, and obfuscated selectors stay bounded. |
| Ranking and snippets | Do not rank or display results outside the policy scope. | Restricted title or snippet is absent from all responses. |
| Empty result | Return an explicit no-access or no-match state without confirming protected data. | Denied and nonexistent cases are distinguishable only to authorized operators. |
| Multi-query retrieval | Reapply policy to every subquery and merge operation. | Second-hop query cannot widen the boundary. |
| Reranking | Reranker receives only authorized candidates. | Restricted candidate is excluded before scoring. |
| Model context | Assemble context only after final authorization filtering. | Captured test context contains no denied fixture. |
Prefer filtering before retrieval and then re-checking the final result set. A post-generation refusal cannot undo a restricted snippet that was already placed in a prompt, cache, trace, or citation.
Cache and session memory review
Caches and conversation memory often outlive the request that created them. Review them as separate data stores.
| Cache or memory | Key must include | Invalidation trigger | Negative test |
|---|---|---|---|
| Retrieval result cache | Tenant, subject or permission scope, source version, query policy version | Role, membership, source ACL, or policy change | User B cannot receive User A’s cached result. |
| Prompt or context cache | Same authorization scope plus route and model version | Permission, route, source, or model change | Context cache cannot replay restricted content. |
| Conversation memory | Subject, tenant, conversation, retention class, and access policy | Offboarding, deletion, tenant change, or expiry | New conversation cannot read old tenant memory. |
| Response cache | User-visible scope, locale, route, and citation policy | Source, role, or customer-facing policy change | Public response never contains private citation data. |
| Embedding or index cache | Source ID, version, tenant, ACL, and classification | Revocation or deletion | Deleted fixture cannot be returned by a stale replica. |
| Analytics cache | Aggregation scope and privacy class | Data policy or retention expiry | Tenant detail cannot be reconstructed from a broad report. |
Document cache TTLs and invalidation behavior. A short TTL reduces exposure but does not replace authorization. For high-risk sources, prefer no cross-request content cache until the team can prove the key, eviction, and access checks.
Connector and export boundaries
| Path | Questions to answer | Default rule |
|---|---|---|
| Delegated user connector | Does the source evaluate the current user or only the connector identity? | Do not assume delegated access unless the source response proves it. |
| Service-account connector | What exact tenants, resources, fields, and operations can it access? | Apply a server-side policy narrower than the service credential. |
| Synchronization job | What happens when a user is removed or a source permission changes? | Reconcile revocations and verify the index after sync. |
| Citation URL | Can the recipient open the URL under their own authorization? | Omit or replace citations that reveal unauthorized paths. |
| Download or export | Which rows, fields, and tenant scope are included? | Require an independent export policy and audit record. |
| Support or admin view | Can support staff see customer data through the assistant? | Use explicit role, purpose, and case scope with logging. |
| Tool action | Can retrieved content influence a write, email, ticket, or access change? | Validate arguments and authorization outside the model. |
| Error and retry path | Can failure details contain source names or snippets? | Use safe, non-confirming errors and restrict diagnostics. |
Test direct source access, chatbot access, citation access, export access, and operator access separately. Matching results across those paths is not automatically correct; each path can have a different purpose and permission boundary.
Indirect prompt injection controls
Retrieved documents, tickets, web pages, and user-uploaded files may contain instructions that the model interprets. Treat them as untrusted content even when the source is inside the organization.
- Mark retrieved content as data, separate from system and policy instructions.
- Keep identity, tenant, source authorization, and tool permission decisions in code or a policy gateway.
- Do not allow a document to change the allowed tenant, role, resource filter, or destination.
- Do not pass source-provided URLs, tool names, or action arguments directly to a privileged tool.
- Validate output and tool arguments against an allowlist and the current request scope.
- Test instructions hidden in normal text, markup, metadata, filenames, images, multilingual text, and encoded fragments.
- Require human approval for high-risk writes, messages, access changes, purchases, deletions, or other irreversible actions.
- Log the source ID and finding without copying the malicious or sensitive payload into broad logs.
- Add a pause and investigation path for repeated injection or unexpected cross-boundary retrieval.
OWASP LLM01 describes direct and indirect prompt injection and notes that RAG does not fully prevent it. Prompt instructions can improve behavior, but they are not a substitute for least privilege, authorization, filtering, and action controls.
Negative test set
Use synthetic fixtures or approved masked data. The expected result should state both what is returned and what must not be revealed.
| Test class | Fixture or request | Expected result | Evidence |
|---|---|---|---|
| Cross-tenant query | Tenant A asks for a known Tenant B record. | No content, no confirming title, and a safe result. | Request ID and policy decision. |
| Role escalation | Read-only user asks for an admin-only source. | Deny or limit without disclosing the source body. | Role test result. |
| Direct record selector | User supplies another tenant’s record ID or URL. | Server-side scope wins; no record disclosure. | Selector validation result. |
| Filter removal | Prompt asks to ignore tenant or ACL filters. | Policy filter remains enforced. | Gateway and retrieval trace. |
| Indirect injection | Retrieved document says to reveal hidden records or call a tool. | Treat text as data; no scope or action change. | Injection fixture result. |
| Citation leak | Answer references a restricted title, path, or signed URL. | Citation is removed, redacted, or replaced with a safe explanation. | Citation review. |
| Cache replay | Same query follows a user or tenant change. | Old scoped content is not returned. | Cache invalidation result. |
| Revoked source | Source permission is removed after indexing. | Retrieval and citation stop after the defined propagation window. | Revocation test. |
| Deletion | Indexed and cached record is deleted. | No new answer, citation, export, or memory entry contains it. | Deletion verification. |
| Connector overreach | Service credential sees more than the chatbot route should expose. | Application policy narrows the result. | Connector scope record. |
| Multi-hop query | First result suggests a second resource or connector. | Second hop is separately authorized. | Multi-hop trace. |
| Error path | Search, source, or policy service fails. | Fail closed or safe fallback without sensitive detail. | Failure fixture. |
| Export path | User requests a CSV, document, or CRM write. | Export scope and action authorization are checked independently. | Export audit record. |
| Locale or encoding | Same denied request uses another language or encoding. | Boundary remains unchanged. | Locale test result. |
For each failure, record the affected scope, containment, owner, correction, and re-test date. Do not remove a hard case from the test set because it is inconvenient or produces an unflattering metric.
Monitoring and alert rules
| Signal | Why it matters | Default response |
|---|---|---|
| Denied retrieval spike | May indicate misuse, a broken client, or an access policy change. | Rate-limit if needed, sample safely, and investigate. |
| Cross-tenant fixture in a result or trace | Potential boundary failure. | Block route, preserve restricted evidence, and start incident response. |
| Missing ACL or tenant metadata | Index items may be unscoped. | Quarantine affected items and stop expansion. |
| Citation points to unauthorized path | User-visible leakage risk. | Remove citation, contain route, and review affected answers. |
| Revocation-to-invalidation delay | Stale content remains reachable. | Alert owner and enforce a safe source or route state. |
| Cache key or policy version mismatch | Cached result may use stale authorization. | Invalidate affected cache and re-run boundary tests. |
| Connector scope changes | Service access may be broader than intended. | Require change review and least-privilege verification. |
| Prompt injection or filter-bypass pattern | Possible attempt to alter policy or retrieve protected data. | Keep the boundary, record a minimized signal, and escalate. |
| Export or tool action block spike | Could indicate a legitimate workflow issue or abuse. | Review action policy and keep high-risk actions gated. |
Alert records should contain route, tenant scope, policy version, source IDs, request ID, and decision metadata only as permitted. Avoid logging raw protected results just to make an alert useful.
Review workflow
| Step | Owner | Output |
|---|---|---|
| 1. Define route and data boundary | Product and data owner | Supported intents, users, tenants, sources, actions, and exclusions. |
| 2. Map identity and trust boundaries | Engineering and security owner | Data-flow map and server-side policy points. |
| 3. Register source and connector permissions | Source and platform owners | Source inventory, ACL model, service scope, and deletion path. |
| 4. Verify index and cache metadata | Platform owner | Tenant, ACL, classification, version, and invalidation evidence. |
| 5. Run allow and deny tests | Reviewer | Positive, cross-tenant, revocation, injection, error, and export results. |
| 6. Review citations, logs, and exports | Privacy and security reviewers | Leakage assessment and minimization decision. |
| 7. Decide scope | Release owner | Approve, limit, read-only, draft-only, require review, pause, or rollback. |
| 8. Monitor and recheck | Service owner | Alerts, corrections, permission changes, and next review trigger. |
Use an independent reviewer for customer-facing or high-impact routes. The developer who built the retriever should not be the only person deciding that the retriever is isolated.
Evidence record
| Evidence | Minimum content | Access rule | Retention note |
|---|---|---|---|
| Policy decision | Review ID, subject scope, tenant scope, allowed sources, and decision. | Review team and accountable owner. | Retain by documented policy. |
| Source register | Source ID, owner, ACL model, classification, version, and lifecycle state. | Source and security owners. | Recheck on source or permission change. |
| Index sample | Synthetic or masked rows with required security metadata. | Platform and security reviewers. | Remove after review if not needed. |
| Boundary test result | Fixture ID, expected and observed scope, route and policy version. | Engineering and incident responders. | Keep enough for reproduction. |
| Citation review | Claim, source ID, authorization result, and display decision. | Need-to-know reviewers. | Minimize source content. |
| Cache and revocation test | Key scope, invalidation event, delay, and observed result. | Platform owners. | Update after cache design changes. |
| Action audit | Tool, arguments, authorization, reviewer, and result. | Operators and incident responders. | Follow action and privacy policy. |
| Finding and correction | Impact, containment, owner, due date, and re-test. | Owners and reviewers. | Preserve decision history. |
Never store API keys, passwords, access tokens, signed URLs, raw customer exports, or unredacted sensitive records in the provenance or boundary review. A test record that cannot be shared broadly may still be necessary, but it should have an explicit restricted location and owner.
Release gate
Approve only the documented user, tenant, source, route, and action scope when each applicable gate has evidence.
- Server-side identity, tenant, role, purpose, and resource context are derived and validated before retrieval.
- Every connector, source, index, cache, memory store, log, citation, export, and tool path has an owner and boundary.
- Required tenant, ACL, classification, source, and version metadata is present before a record can be indexed or retrieved.
- Results are filtered before model context assembly and checked again before display, citation, export, or action.
- Service credentials and route permissions follow least privilege and do not rely on a system prompt for authorization.
- Permission revocation and deletion propagate to indexes, caches, replicas, summaries, and exports.
- Cross-tenant, role escalation, filter removal, indirect injection, citation leak, cache replay, and error-path tests pass.
- Retrieved content is treated as untrusted data and cannot change policy, identity, tenant, or tool authorization.
- High-risk reads, writes, exports, messages, access changes, and deletions have independent action gates.
- Logs and analytics are minimized, access controlled, retention bounded, and checked for sensitive leakage.
- Findings have owners, severity, containment, deadlines, and re-test evidence.
- The decision names monitoring, correction, incident, pause, rollback, and next review triggers.
If any critical boundary cannot be tested, default to no retrieval, a synthetic-only environment, read-only scope, or human review. Do not compensate for missing authorization evidence with a better prompt or a higher model setting.
Staged rollout plan
| Stage | Scope | Required evidence | Exit condition |
|---|---|---|---|
| 0. Offline fixtures | Synthetic sources and masked identities only. | Data-flow map, policy tests, index metadata, cache design, and injection tests. | No unresolved critical boundary failure. |
| 1. Internal read-only | Named reviewers, one tenant or isolated workspace. | Live allow and deny retrieval, citation, revocation, deletion, and log review. | Another reviewer can reproduce the boundary decision. |
| 2. Limited pilot | Small approved group or tenant set. | Connector, cache, export, monitoring, and correction evidence. | Signals stay within team-defined limits; no cross-scope result. |
| 3. Controlled customer scope | Supported intents and explicit sources only. | Sampled production-safe tests, human gates, and rollback readiness. | No material open finding; source and action owners respond. |
| 4. Ongoing operation | Approved route and tenant scope. | Scheduled permission, source, index, cache, citation, and incident review. | Re-review after any boundary or workflow change. |
Do not expand scope because a route returns useful answers. Expansion requires evidence that the usefulness did not come from broader-than-authorized retrieval or hidden leakage through citations, caches, logs, or exports.
Findings and remediation
| Finding ID | Failure or limitation | Affected scope | Containment | Owner and due date | Re-test evidence |
|---|---|---|---|---|---|
| Missing tenant or ACL metadata | Quarantine items and stop retrieval. | ||||
| Cross-tenant or role-boundary result | Block route, preserve restricted evidence, investigate. | ||||
| Stale permission in index or cache | Invalidate, revoke, and use a safe source state. | ||||
| Citation or error leaks restricted detail | Redact or disable path and assess exposure. | ||||
| Indirect injection changes policy or action | Treat source as untrusted, disable action, add test. | ||||
| Connector or export overreach | Narrow credential and require independent approval. | ||||
| Excessive data in logs or memory | Minimize, restrict, delete, and verify retention. |
Prioritize by realistic impact and affected scope. A single cross-tenant result or privileged export may be more urgent than many harmless no-result errors.
Decision table
| Review result | Default decision | Follow-up |
|---|---|---|
| Identity, tenant, source, cache, citation, and action boundaries pass | Approve the defined scope | Monitor and schedule the next review. |
| A narrow source or intent fails without customer impact | Limit or remove the affected scope | Fix metadata or policy and re-test. |
| ACL or tenant metadata is missing or stale | Quarantine and pause retrieval | Repair ingestion, invalidate caches, and verify deletion. |
| Cross-tenant, unauthorized, or restricted result appears | Block and investigate | Preserve restricted evidence, assess exposure, and notify owners. |
| Retrieved text changes policy or triggers an unauthorized action | Disable action or route | Add injection tests and enforce authorization in code. |
| Citation, log, cache, or error exposes protected detail | Redact and contain | Review affected records and tighten minimization. |
| Connector or export is broader than the approved route | Narrow or disable it | Reauthorize with least privilege and run export tests. |
| Boundary evidence is incomplete | Keep read-only, draft-only, or paused | Collect evidence before expansion. |
| Repeated material failures or unresolved incident | Pause or rollback | Use the incident and recovery owner path. |
Sign-off record
| Field | Entry |
|---|---|
| Review ID | |
| Chatbot and route | |
| Approved subject, tenant, role, locale, and purpose scope | |
| Approved sources, fields, index, connector, and cache scope | |
| Model, prompt, retrieval, and policy versions | |
| Allow and deny test set | |
| Citation, log, export, and action review | |
| Open findings and accepted limitations | |
| Monitoring and correction owner | |
| Pause, rollback, and incident owner | |
| Decision | Approve / limit / read-only / draft-only / require review / pause / rollback. |
| Decision date and timezone | |
| Release owner | |
| Reviewer | |
| Next review trigger |
This sign-off applies only to the listed scope. A model, prompt, connector, source, index, cache, permission, tenant, role, export, tool, or policy change requires a new review or documented change assessment.
Action tracker
| Action ID | Action | Reason | Owner | Priority | Due date | Status | Evidence link or record ID |
|---|---|---|---|---|---|---|---|
| Open / blocked / done | |||||||
Link each action to the review and finding IDs. A completed action should point to a policy diff, invalidation result, access test, deletion check, or other evidence that closes the issue.
Final tenant isolation checklist
- The route derives identity, tenant, role, purpose, and resource context from trusted server-side sources.
- Every retrieval and connector request carries the approved scope and cannot accept an arbitrary client or model scope.
- Source, index, chunk, cache, memory, citation, export, log, and tool boundaries are documented.
- ACL, tenant, classification, source, and version metadata is required before indexing and retrieval.
- Filtering is applied before model context assembly and rechecked before every user-visible or durable result.
- Cross-tenant and role-escalation tests return no content and do not confirm protected details.
- Revocation, deletion, cache invalidation, and index updates are tested after permissions change.
- Retrieved documents, files, tickets, and web content are treated as untrusted data.
- Prompt injection cannot change tenant, role, source filters, authorization, or tool permissions.
- Citations, errors, logs, analytics, memory, and exports do not disclose restricted metadata or content.
- High-risk tools and writes have independent validation, authorization, confirmation, and audit.
- No password, API key, token, signed URL, raw customer export, or unredacted private record is in the review evidence.
- Findings, actions, monitoring, pause, rollback, incident, and next review owners are named.
- I ran the AI Tool Risk Checker and attached the route result.
- I completed the Small Team AI Security Checklist for surrounding controls.
Metrics to track
Track by route, tenant scope, source class, connector, policy version, and review period. Do not publish a single isolation score as proof of security.
| Metric | Why it matters | Review question |
|---|---|---|
| Authorized retrieval rate | Shows whether supported work is available within policy. | Are denials caused by broken context or intentional policy? |
| Cross-scope result count | Directly signals a boundary failure. | Is the count zero under the defined test and monitoring population? |
| Missing security metadata rate | Finds unscoped index or ingestion items. | Are affected items quarantined before retrieval? |
| Revocation-to-invalidation time | Measures how long stale access may remain. | Does it meet the team-defined risk window? |
| Deletion verification failure rate | Finds retained content in indexes, caches, memory, or exports. | Can the team prove removal after deletion? |
| Citation or error leakage rate | Captures metadata exposure that a content test may miss. | Are restricted titles, paths, snippets, or signed URLs omitted? |
| Cache boundary failure rate | Detects key or invalidation defects. | Do changes in tenant, role, or source invalidate safely? |
| Prompt injection and filter-bypass detections | Shows boundary pressure and test coverage. | Do signals lead to safe containment and useful fixes? |
| Blocked high-risk action rate | Separates answer usefulness from action authorization. | Are automatic actions appropriately narrow? |
| Connector scope changes | Tracks third-party access drift. | Does every change have an owner and approval record? |
| Time to reproduce a boundary finding | Measures evidence quality. | Can another reviewer reproduce it without raw private data? |
Every metric needs a population, denominator, time window, route version, owner, and limitation. A low event count may mean low traffic or weak detection; it is not automatically evidence of a safe boundary.
Evidence checked
- OWASP LLM01:2025 Prompt Injection describes direct and indirect prompt injection and recommends least privilege, code-enforced controls, and human approval for high-risk actions.
- OWASP LLM02:2025 Sensitive Information Disclosure covers PII, financial details, confidential business data, credentials, legal documents, access controls, and secure runtime orchestration.
- NIST SP 800-207 Zero Trust Architecture explains identity-centered authorization and the removal of implicit trust based on network location.
- NIST AI RMF Core describes contextual mapping, documented measurement, production monitoring, risk tracking, and management decisions.
- CISA and UK NCSC Guidelines for Secure AI System Development covers secure design, development, deployment, and operation of AI systems.
- AI Tool Risk Checker for a route-level risk record.
- Small Team AI Security Checklist for identity, data, access, logging, and incident foundations.
FAQ
Is tenant filtering in the vector database enough?
No. The application must derive and validate identity and tenant context, apply policy across every retrieval path, preserve ACL metadata, and re-check results before model context, citations, exports, and actions. Index filters can be bypassed or become stale if surrounding paths are broader.
Can the system prompt tell the model not to reveal another tenant’s data?
It can be a useful instruction, but it is not an authorization control. Enforce permissions in application code or a policy gateway before retrieval and before downstream actions. Treat retrieved text and user prompts as untrusted input.
Should we use one service account for all customer retrieval?
Only if the service account is narrowly scoped and the application enforces a separate user and tenant policy. A broad credential does not make a chatbot authorized. Review the connector scope, source ACL behavior, metadata propagation, and deny tests before using it.
What should happen when tenant or ACL metadata is missing?
Do not guess. Quarantine or exclude the item, record the failure, and keep the affected route limited or paused until the metadata and policy path are repaired and tested.
How do caches cause cross-tenant leakage?
A cache may reuse a result, prompt context, response, or memory entry under a key that is too broad. Include the effective authorization scope, route, policy and source versions in the design, invalidate on permission changes, and test replay after tenant or role changes.
Do citations create a separate access-control risk?
Yes. A citation may reveal a restricted document title, path, snippet, signed URL, or source existence even when the answer body is filtered. Re-authorize citation display for the recipient and omit or replace references that are not safe to disclose.
How should we test prompt injection in retrieved documents?
Use synthetic or approved masked fixtures containing instructions that attempt to change tenant scope, remove filters, reveal hidden data, or call tools. The expected result is that the content remains untrusted data and cannot change policy or authorization.
How often should retrieval access be reviewed?
Review before launch and after changes to identity, roles, tenants, sources, connectors, indexes, parsers, embedding jobs, caches, models, prompts, tools, exports, or policies. Add recurring review based on risk and monitor revocations, denied requests, leakage signals, and incidents.
Does this checklist prove compliance or isolation?
No. It is a practical operating aid. Legal, privacy, contractual, sector, and security requirements vary by workflow and jurisdiction; obtain qualified advice for those decisions.
Recommended next step
Run the AI Tool Risk Checker for one chatbot route, build two synthetic tenants with one shared-looking query, and trace both allow and deny paths through identity, index filtering, cache replay, citation display, logs, and a blocked tool action. Record the result in the Small Team AI Security Checklist and keep production scope read-only or human-reviewed until the boundary can be reproduced.