LLM security: defending generative AI with the OWASP LLM Top 10

In short: LLM security protects applications built on large language models against attacks such as prompt injection, data leakage and data poisoning. The authoritative reference is the OWASP LLM Top 10, 2025 version. Effective protection comes not from a single tool but from layered defenses, embedded in a management system aligned with ISO 42001 and ISO 27001.

LLM security at a glance

• Definition: LLM security covers all measures that protect applications built on large language models (LLMs) and generative AI from misuse, manipulation and data leakage.
• Reference framework: the OWASP Top 10 for LLM Applications, 2025 version (LLM01 to LLM10).
• Top risk: prompt injection (LLM01:2025), direct via user input or indirect via external content.
• Threat modeling: MITRE ATLAS documents real-world adversary tactics against AI systems.
• Governance: anchored in an AI management system under ISO 42001 and an ISMS under ISO 27001; NIST AI RMF as the risk framework.
• Regulation: the EU AI Act requires accuracy, robustness and cybersecurity for high-risk AI in Article 15.
• Important: there is no single complete defense against prompt injection; layered measures are required.

What is LLM security?

LLM security is the discipline of protecting applications built on large language models across their entire lifecycle against attacks and misbehavior. It spans the model itself, the upstream data and embeddings, the interfaces, and the downstream systems that process model output.

The decisive difference from traditional application security lies in the attack surface. A language model does not reliably separate instruction from content: input that looks like harmless text can redirect the model's behavior. This creates risks that conventional web application firewalls and simple input filters do not fully cover.

LLM security is therefore not a pure tooling topic. It combines technical safeguards with governance: clear accountability, risk assessment, and a management system that sustains secure operation over time.

LLM security vs traditional application security

LLM security does not replace traditional application security; it extends it with AI-specific attack surfaces:

• Attack vector: traditionally via code and configuration; with LLMs additionally via natural language and external content (prompt injection).
• Trust boundary: traditionally clear between code and data; a language model blends instruction and content and blurs that boundary.
• Determinism: traditional systems respond deterministically; LLMs are probabilistic, so the same input can yield different output.
• Data risk: beyond classic leaks, training and embedding data become attack targets (poisoning, extraction).

Why LLM security matters now

Generative AI has moved from experiment to production in record time, in customer service, knowledge management and software development. With every model connected to internal data, tools or agents, the attack surface grows. A successful attack can expose confidential data, produce false answers, or trigger unwanted actions through connected tools.

At the same time, regulatory pressure is rising. The EU AI Act explicitly requires robustness and cybersecurity for high-risk AI, and customers increasingly expect evidence that AI is operated securely. LLM security is therefore both a technical and a compliance requirement.

A related real-world risk: Control Shadow AI Instead of Banning It: How an AI Governance Framework Really Protects

The OWASP LLM Top 10 for LLM Applications 2025

The most important reference for LLM security is the OWASP Top 10 for LLM Applications. The current version dates from 2024 and replaces the older 2023 list. The ten risks at a glance, each with its central mitigation:

• LLM01:2025 Prompt Injection: input alters model behavior in unintended ways. Mitigation: filter input and output, restrict privileges, require human approval for critical actions.
• LLM02:2025 Sensitive Information Disclosure: the model reveals confidential data. Mitigation: data minimization, access control and output review.
• LLM03:2025 Supply Chain: risks from third-party models, libraries and training data. Mitigation: provenance checks, supplier controls and an AI bill of materials.
• LLM04:2025 Data and Model Poisoning: manipulated training or fine-tuning data implants backdoors or bias. Mitigation: secure data provenance, validation and monitoring.
• LLM05:2025 Improper Output Handling: model output is processed unchecked. Mitigation: validate, encode and treat output as untrusted input.
• LLM06:2025 Excessive Agency: the model is granted overly broad permissions or tools. Mitigation: least privilege, narrow tool permissions, human approval.
• LLM07:2025 System Prompt Leakage: the system prompt with sensitive instructions is exposed. Mitigation: keep no secrets in the system prompt, enforce protection at the architecture level.
• LLM08:2025 Vector and Embedding Weaknesses: weaknesses in vector and embedding stores, for example in RAG. Mitigation: access control plus segregation and labeling of external content.
• LLM09:2025 Misinformation: the model produces false but plausible content. Mitigation: source grounding via RAG, fact-checking and human oversight.
• LLM10:2025 Unbounded Consumption: uncontrolled resource consumption up to denial of service. Mitigation: rate limiting, quotas and monitoring.

Prompt injection: the biggest LLM risk

Prompt injection sits at the top of the OWASP list for good reason. The term was coined in 2022 by analogy to SQL injection and describes input that makes the model disregard its intended instructions. There are two forms:

• Direct prompt injection: the attacker enters the manipulating instruction immediately, for example in the chat, to make the model ignore its safety rules.
• Indirect prompt injection: the instruction hides in external content the model processes, such as a web page, document or email. This is especially dangerous for RAG systems and agents that retrieve external content.

It is important to understand that no filter yet prevents prompt injection completely. OWASP and leading security researchers stress that filtering alone is not enough. Only the combination of least privilege, strict output handling and human approval for critical actions is effective.

Common attack scenarios

Three scenarios show how LLM risks combine in practice:

• Data exfiltration via a connected tool: an attacker uses prompt injection to make an assistant retrieve internal documents and copy them into its answer. Here LLM01, LLM02 and LLM06 interlock.
• Poisoned RAG source: hidden instructions are planted in a knowledge-base document. As soon as the system retrieves it, an indirect prompt injection fires (LLM01 and LLM08).
• Manipulated model from the supply chain: a fine-tuned model pulled from a public source contains a backdoor. Without provenance checks it reaches production unnoticed (LLM03 and LLM04).

All three share one lesson: a single safeguard is not enough. Only the combination of technical protection and clear governance breaks the attack chain.

Mapping the OWASP LLM Top 10 to ISO 42001, ISO 27001 and NIST AI RMF

Most articles on LLM security treat the OWASP risks as a pure tooling question. For organisations, however, the decisive step is embedding them in existing management systems. That turns a list of individual measures into an auditable, durably owned discipline. The following mapping shows which framework addresses which risks:

• ISO/IEC 42001 (AI management system): governs the AI lifecycle. It primarily addresses Supply Chain (LLM03), Data and Model Poisoning (LLM04), Excessive Agency (LLM06) and Misinformation (LLM09) through controls on data, third parties, impact assessment and human oversight.
• ISO/IEC 27001 (information security): secures the underlying infrastructure. It addresses Sensitive Information Disclosure (LLM02), Vector and Embedding Weaknesses (LLM08) and Unbounded Consumption (LLM10) through access control, data classification and availability.
• NIST AI RMF: provides a risk framework through its Govern, Map, Measure and Manage functions that orders all ten risks across the lifecycle. The Generative AI Profile (NIST AI 600-1) extends it for generative AI.
• EU AI Act, Article 15: makes accuracy, robustness and cybersecurity legally binding for high-risk AI. ISO 42001 and ISO 27001 provide the organisational basis to meet these obligations in a structured way.

This creates one continuous arc: OWASP names the technical risks, MITRE ATLAS models the attacks, ISO 42001 and ISO 27001 anchor the governance, and the EU AI Act sets the legal frame.

Important: this mapping is an analytical aid, not an official standards mapping. In real organisations the frameworks overlap, and which control covers which risk is more the result of a negotiation than a deterministic formula. The mapping gives orientation but does not replace your own risk assessment.

Modeling attacks with MITRE ATLAS

While the OWASP LLM Top 10 names the risk classes, MITRE ATLAS (Adversarial Threat Landscape for Artificial-Intelligence Systems) provides the attacker's perspective. ATLAS is a freely accessible, continuously maintained knowledge base of real-world tactics and techniques against AI systems, modeled on MITRE ATT&CK. It contains roughly 16 tactics, more than 170 techniques, plus documented case studies and mitigations.

In practice this means you can walk through concrete attack paths, such as extracting a model (model extraction) or bypassing safeguards via manipulated input. Unlike the 2023 OWASP list, model theft is no longer a standalone top-10 entry in the 2025 version; as an attack technique it remains covered by MITRE ATLAS. In spring 2025, ATLAS was expanded specifically with generative-AI techniques, including RAG Poisoning, False RAG Entry Injection and LLM Prompt Crafting.

Agentic AI: the next escalation

Many articles treat LLM applications as if they were plain chatbots. As soon as language models act as agents, calling tools on their own, using memory and planning multi-step tasks, the attack surface grows considerably.

The OWASP Gen AI Security Project has launched a dedicated Agentic Security Initiative for this. Its agentic threat taxonomy (early 2025) and the Top 10 for Agentic Applications, published in December 2025, name new attack surfaces: agent memory, tool integration, identity and permissions, and multi-agent coordination.

Typical agentic risks are tool misuse, multi-step prompt-injection chains, and lateral movement across connected tools. The countermeasures follow the same principles, applied more strictly: least privilege per tool, human approval at critical points, and a clear limit on autonomy.

More on securing autonomous agents: Security concept for autonomous AI agents: Use specialized security agents as monitoring instances

LLM security as a governance discipline

Point tools fall short when no one is accountable and no process sustains secure operation. This is exactly where the governance approach comes in. An AI management system under ISO 42001 creates the structure: it requires risk assessment, impact assessment, clear roles and control across the entire AI lifecycle, including the security of AI systems as part of risk treatment.

In addition, an ISMS under ISO 27001 secures the information-security layer the AI sits on. A forthcoming standard, ISO/IEC 27090, provides guidance on cybersecurity for AI (a non-certifiable guidance document) and is still in development as of 2026; a companion standard, ISO/IEC 27091, addresses privacy. Until these are published, ISO 42001 and ISO 27001 together form the dependable foundation.

Beyond the ISO world, internationally aligned baselines have emerged. The "Guidelines for Secure AI System Development" by CISA and the UK NCSC (November 2023, co-sealed by agencies from 18 countries) describe a secure AI lifecycle on Secure by Design principles. NIST adds the draft NIST IR 8596 (Cyber AI Profile, December 2025), bridging cybersecurity and AI on the basis of CSF 2.0. These frameworks complement ISO 42001 and ISO 27001 rather than replacing them.

Defenses: layered protection

Effective LLM security follows the principle of layered defense. No single measure is enough; only their interplay reduces risk meaningfully:

• Filter and validate input, but do not rely on it as the sole defense against prompt injection.
• Handle output strictly: validate, encode and never pass it to downstream systems unchecked.
• Least privilege for models, tools and agents to limit excessive agency.
• Human-in-the-loop approval for critical or irreversible actions.
• Sandboxing and isolation of tool and code execution.
• Rate limiting and quotas against uncontrolled resource consumption.
• Label and segregate external content for RAG, and source it only from trusted origins.
• Adversarial testing and regular red teaming, supported by monitoring.

Common misconceptions about LLM security

• "A good filter prevents prompt injection." No. Filtering helps but is not a complete defense and must be combined with least privilege and strict output handling.
• "Security is solely the AI provider's job." Only partly. The organisation remains responsible for its own application, its data and the connected tools.
• "RAG makes the model secure." No. RAG improves factual accuracy but opens new paths for indirect prompt injection through external content.
• "LLM security is a developer-only topic." No. Without governance, clear accountability and a management system it stays piecemeal.
• "ISO 27001 is enough for AI." No. ISO 27001 secures the infrastructure but does not fully cover AI-specific risks; ISO 42001 complements it.

First steps toward secure generative AI

A pragmatic start orders the measures rather than buying individual tools:

1. Inventory: which LLM applications, data sources, tools and agents are in use?
2. Build a threat model based on the OWASP LLM Top 10 and MITRE ATLAS.
3. Map the risks to ISO 42001 and ISO 27001 and name the owners.
4. Implement layered defenses and establish red teaming.
5. Move secure operation into the AI management system and audit it regularly.

Regulatory context: EU AI Act High Risk: What companies must implement by August 2026

Frequently asked questions about LLM security

What is LLM security?

LLM security is the discipline of protecting applications built on large language models across their entire lifecycle against attacks and misbehavior. It combines technical safeguards against risks such as prompt injection with governance, meaning accountability, risk assessment and a management system for secure operation.

What are the OWASP LLM Top 10?

The OWASP Top 10 for LLM Applications are the authoritative reference for the most common security risks of generative AI. The current 2025 version runs from LLM01 Prompt Injection to LLM10 Unbounded Consumption and names a matching mitigation for each risk.

What is prompt injection?

Prompt injection is an attack in which input makes a language model disregard its intended instructions. The term was coined in 2022 by analogy to SQL injection and sits at the top of the OWASP list as LLM01:2025.

What is the difference between direct and indirect prompt injection?

In direct prompt injection the attacker enters the manipulating instruction immediately, for example in the chat. In indirect prompt injection it hides in external content the model processes, such as web pages or documents. The indirect form is especially dangerous for RAG systems and agents.

Can prompt injection be fully prevented?

No. No filter yet prevents prompt injection completely. Only a combination of measures is effective: least privilege, strict output handling and human approval for critical actions.

Does ISO 42001 cover LLM security?

ISO 42001 is an AI management system, not a dedicated security standard. It addresses security as part of risk treatment across the AI lifecycle, for example on data, third parties and oversight. Purely technical LLM security is complemented by the OWASP LLM Top 10, ISO 27001 and the forthcoming ISO/IEC 27090.

Is ISO 27001 enough for generative AI?

No. ISO 27001 secures the information security of the infrastructure but does not fully cover AI-specific risks such as prompt injection or data poisoning. For generative AI you combine ISO 27001 with ISO 42001 and the OWASP LLM Top 10.

How does the EU AI Act relate to LLM security?

The EU AI Act requires an appropriate level of accuracy, robustness and cybersecurity for high-risk AI across the lifecycle in Article 15. LLM security provides the technical measures, while ISO 42001 and ISO 27001 provide the organisational basis to meet that obligation.

What is MITRE ATLAS?

MITRE ATLAS (Adversarial Threat Landscape for Artificial-Intelligence Systems) is a freely accessible knowledge base of real-world tactics and techniques for attacks on AI systems, modeled on MITRE ATT&CK. It helps model attack paths and derive mitigations.

How do I get started with LLM security?

Start with an inventory of your LLM applications, a threat model based on the OWASP LLM Top 10 and MITRE ATLAS, a mapping of the risks to ISO 42001 and ISO 27001, the implementation of layered defenses, and regular red teaming and auditing within the AI management system.