Identity & workload identity
Know which person, service or agent is acting and how trust is established.
- Separate human and agent identities
- Use strong authentication
- Expire temporary access
Secure AI / 2026
Secure the whole AI system—not only the prompt.
Modern AI security depends on the application around the model: identities, data, retrieval, tools, agents, connectors, suppliers and the decisions outputs are allowed to trigger.
IdentityBounded
DataClassified
ToolsLeast privilege
AuthorityHuman
AI Security Matrix
Ten control domains around every AI use case.
Threat-model the complete workflow and assign owners before deployment.
Data & classification
Define what data may enter prompts, files, retrieval stores, logs and feedback.
- Minimise sensitive data
- Protect provenance and integrity
- Define retention and deletion
Models & behaviour
Treat model output as probabilistic and potentially manipulable.
- Test unsafe behaviours
- Control model changes
- Do not treat system prompts as secrets
Context, RAG & retrieval
Treat retrieved documents, webpages and messages as untrusted input.
- Authorise retrieval per user
- Track provenance
- Separate interpretation from execution
Application & output
Validate AI output before rendering, executing or passing it downstream.
- Encode and validate output
- Use deterministic policy checks
- Fail closed for high impact
Tools & capability
Every tool converts text into possible real-world impact.
- Allowlist tools
- Validate parameters
- Rate-limit and scope actions
Agents, memory & autonomy
Long-running agents can compound mistakes, manipulation and stale memory.
- Bound goals and duration
- Review memory sources
- Require approval checkpoints
MCP & connectors
A protocol connection is not a trust decision.
- Validate authorization
- Restrict servers and tools
- Log sensitive invocations
Supply chain & provenance
Models, datasets, packages and providers can change beneath the application.
- Record source and version
- Review supplier controls
- Plan exit and rollback
Human authority & recovery
People need enough evidence to approve, pause and recover.
- Log prompts and tool calls
- Design kill switches
- Test recovery
2026 risk baseline
Risk language is a map—not a substitute for architecture-specific threat modelling.
Use current OWASP GenAI, agentic AI, ASD/ACSC and MCP guidance to structure review.
Prompt injection
Untrusted content can redirect model behaviour or tool use.
- Assume indirect injection
- Keep privileged actions outside model authority
Excessive agency
Broad permissions turn a wrong answer into an operational incident.
- Minimum privilege
- Approval before consequence
Sensitive information exposure
Prompts, retrieval and outputs can disclose information outside intended boundaries.
- Classify inputs
- Authorise retrieval
- Validate outputs
Supply-chain change
A model, connector, library or provider can materially change risk.
- Version and review
- Monitor material changes
- Maintain rollback
Operating model
Discover → classify → bound → verify → observe → recover.
Apply one repeatable sequence to every AI use case.
Discover
Map owner, model, data, identities, tools, connectors and intended outcome.
Classify
Classify data and consequences before access is granted.
Bound
Apply minimum privilege, narrow tools and explicit prohibited actions.
Verify
Validate identity, source, output and approval before material action.
Observe
Capture useful logs, approvals and unusual behaviour.
Recover
Provide credential revocation, rollback and evidence-preserving recovery.
Human-controlled AI defence
Build capability without losing authority.
Start with the real workflow, data, identities, tools and consequences. Then place controls where they can stop harm.