Misinformation and hallucinations in LLM applications (OWASP LLM07)

What misinformation means for an LLM

The OWASP Top 10 for Large Language Model (LLM) Applications uses the term misinformation for false or misleading output that someone takes as true. It ranks LLM07 in the 2026 edition; in 2025 it was LLM09.

The best-known cause is hallucination: invented answers that sound plausible. Outdated data, training bias and incomplete answers that leave out an important condition add to it. The risk is not just the error itself, but the overreliance of whoever uses it without checking.

Why language models hallucinate

An LLM generates text by predicting, token by token, the most likely continuation based on what it learned during training. It does not look facts up in a database, and it cannot tell what it knows from what it does not. When information is missing, the most likely continuation is still a well-formed sentence, even if its content is false.

Some situations make it worse: questions about events after the training cut-off, highly specialised topics with little training data, ambiguous questions and prompts built on a false premise. In those cases the model tends to go along with the premise rather than correct it.

When misinformation becomes a security issue

A false answer stops being a curiosity when someone acts on it. There are well-known public cases: lawyers who filed court submissions citing rulings invented by a chatbot, and an airline that a tribunal held to a refund policy its assistant had described incorrectly.

In software development the risk is more direct. Coding assistants sometimes recommend libraries that do not exist. If an attacker registers a package under that invented name on a public registry, anyone who installs what the assistant suggested runs the attacker's code. This technique is known as slopsquatting.

Other examples include vulnerable code accepted without review, insecure configurations presented as best practice, and medical, legal or financial advice given without nuance in an application the user trusts.

# The assistant suggests a package that never existed:
pip install <name-the-assistant-made-up>
# If someone has since registered it with malicious code,
# it runs during installation on the developer's machine.

How it connects to other risks on the list

Misinformation can also be caused on purpose. An attacker who manages to slip false documents into the index of a retrieval-augmented generation (RAG) application, or who poisons the data a model is fine-tuned on, gets the assistant to repeat that information with the authority of an internal source. The outcome is the same as a hallucination, but targeted.

That is why the RAG poisoning and supply chain guides complement this one: the first explains how false content gets in, the second how a malicious dependency reaches a project.

How to reduce the risk of misinformation

No model is free of mistakes, so the defence is to make sure its errors do not reach important decisions without a check.

• Ground answers in verified sources through RAG, with citations the user can open and check.
• Ask the model to say when it does not know something, and measure that behaviour in testing instead of always rewarding an answer.
• Check that every suggested package exists, has a history and comes from the expected author before installing it, and pin dependencies in a lock file.
• Mandatory human review in sensitive areas: health, law, finance and code headed for production.
• Clear notices in the interface about the assistant's scope and limits.
• Regular evaluations with questions whose correct answer is known, to catch regressions when the model or prompt changes.