Key takeaways
- Identify the tenant before searching: derive tenant and user context from verified application identity.
- Test access and denial together: pair one tenant's successful lookup with another tenant's denial.
- Include lifecycle events: repeat the test after a role change, withdrawal, and cache refresh.
A shared memory service must keep each customer's knowledge within its permitted users and agents. For the same record, query, and operation, pair an authorized user's success with another tenant's denial. Each tenant represents a separate customer or organization in the test.
Define the boundary in business terms
Create two synthetic tenants, two ordinary users, and an administrator with explicitly approved duties. Use the same project name in both tenants to catch lookups that rely on a label rather than a verified tenant identifier.
Write down the expected read and write rights before running the test. Administrative access should have its own named purpose and audit path. A model-generated tenant name or a tenant identifier embedded in retrieved text is input data, not authentication.
The OWASP RAG Security Cheat Sheet treats authorization during retrieval as an important control. Your test must cover the actual application's route to the store, including any filtering performed outside it.
Build a paired test matrix
| Case | Authorized side | Out-of-scope side | Evidence to retain |
|---|---|---|---|
| Read one project note | Intended user receives the accepted version | Other tenant receives no protected content | Request identity, record ID, allow/deny result |
| Update a convention | Authorized owner creates a reviewed change | Other tenant's write is refused | Version and decision identifiers |
| Search an ambiguous name | Results stay within the current tenant | Same name reveals no cross-tenant excerpt | Query scope and returned IDs |
| Revoke a role | An unaffected authorized user retains permitted access | Revoked user no longer receives protected content | Revocation and test timestamps |
| Withdraw a source | Remaining valid evidence stays usable | Withdrawn material leaves the answer path | Source, index, and cache results |
Treat denial messages and snippets as outputs to inspect. A search that returns no document but includes its protected title or excerpt has a different result from a clean denial.
Check copies as well as the original record
Follow the record into any search index, cached answer, conversation context, export, or diagnostic record. Each copy needs its own access and retention policy; a database filter alone may not cover them all.
After a permission change, repeat the query through the user-facing application and any permitted API client. Record when each path applies the new decision. Content freshness may have a stated delay, but a successfully revoked identity must not continue receiving protected content through stale indexes or caches. Test the authorization outcome separately from freshness. Withdrawal from the answer path does not necessarily delete restricted audit or retention copies.
Evaluate AKB as one component
AKB's public repository documents agent-facing organizational knowledge through MCP. Evaluate the selected release and deployment against this matrix. Keep source permission translation, collection, curation, and application action approval assigned to their actual owners.
If production use or distribution is being considered, review the applicable AKB LICENSE for that release. Hosting location and licensing do not, by themselves, establish the authorization result your test needs.
Make failures reproducible
For each failure, keep a small synthetic example, expected permissions, returned identifiers, and the point where access went wrong. Fix it and rerun both the allowed and denied cases. Keep the pair as a regression test.