02 · AI & LLM Security
Can your AI
be made to misbehave?
Evaluation asks how often your AI gets things wrong on its own. Security asks whether someone can make it go wrong on purpose. With AI, that can take nothing more than a well-written sentence.
Why it's different
Why AI needs its own security testing
Traditional security testing looks for unauthorised access, flaws in code and known vulnerabilities. AI systems need all of that, and more. They take instructions in plain language, behave slightly differently every time, and can be persuaded. That opens up risks a conventional penetration test was never designed to find.
The best known is prompt injection. Someone writes a message, or hides one in a document the AI reads, designed to trick it into doing something it was never meant to: leaking another customer's data, revealing its hidden instructions, or taking an action on your systems that it shouldn't.
Finding these takes adversarial testing: deliberately trying to make the system misbehave the way an attacker would, and showing exactly how it happened. A scanner won't find them, and neither will a list of functional tests.
Scope
Three things we test together
Can it be made to misbehave? Prompt injection, guardrail bypass, agents acting beyond their brief, and the APIs they act through.
Can it leak what it shouldn't? Customer data, other customers' records in a shared system, hidden instructions and training data.
Does it behave as you'd want in front of a customer? Inconsistent treatment across customer groups, harmful or misleading answers, and whether it tells people they're talking to AI.
We test them together because in practice they fail together. One injected instruction can be a security breach, a privacy breach and a headline at the same time.
What we test
What we test
- Prompt injection, direct and indirect, including hidden instructions in documents, web pages and tool results.OWASP LLM01
- Data leakage, including one customer reaching another customer's records, and training data extraction.OWASP LLM02
- System prompt leakage.OWASP LLM07
- Excessive agency: agents calling tools they shouldn't, or acting without the checks they need.OWASP LLM06
- Unsafe output handling: AI output passed into a browser, database or another system without being checked.OWASP LLM05
- Retrieval (RAG) weaknesses: documents surfacing to people who shouldn't see them, and poisoned content steering answers.OWASP LLM08 · LLM04
- Runaway usage: requests designed to run up cost or take the service down.OWASP LLM10
- Guardrail bypass.OWASP LLM01
Wrong or misleading answers (OWASP LLM09) are measured in AI Evaluation →
MethodThe OWASP Top 10 for LLM Applications and for agentic applications is our starting point, applied to what your system actually does. Standard application security checks run on established tools, so our time goes on the logic and AI-specific risks that scanners cannot see.
You getReproducible findings, each with business impact and a prioritised fix. Walked through with your engineers, not left in a PDF.
Typical length1 to 3 weeks for a first engagement.
Related
A secure AI can still give wrong answers. Evaluation measures how often, on your data, with your experts.
AI agents act through your APIs. If an agent can be talked into a request, your API decides what happens next.