Why AI Risk Starts Before the Model Runs

Why the risk starts earlier

Most AI risk discussions start too late. They start with bad answers, hallucinations, or unsafe output. Those risks are real. But in enterprise environments, many of the highest-consequence decisions happen before a model produces anything at all.

They happen when an application decides what user text, business records, documents, retrieval results, system instructions, and tool permissions to assemble into a single AI execution path.

That earlier boundary is now central to enterprise AI security guidance. NIST’s Generative AI Profile points to risk across data inputs, processing, deployment, third-party dependencies, privacy, and cybersecurity. OWASP’s 2025 LLM Top 10 includes prompt injection, sensitive information disclosure, supply-chain risk, vector and embedding weaknesses, and excessive agency.

The UK NCSC has also warned that prompt injection is not like traditional SQL injection because large language models do not maintain a reliable internal boundary between instructions and data. [1][2][3]

The practical lesson is simple: AI risk is not only about what the model says. It is also about what the system was allowed to send, retrieve, trust, route, and execute before the model ever ran.

Recent disclosures show the same pattern. Public records and reporting around Microsoft 365 Copilot, Perplexity Comet, and Meta support-chatbot incidents describe risk where context, authority, and action met before the final AI response. [4][5][6]

The request path before execution

For a simple chatbot, the request boundary may be narrow: a user types a prompt, the model responds, and the interaction ends.

Enterprise workflows are different. A single AI request may include a support ticket, CRM note, internal document, retrieved knowledge-base passage, account metadata, policy context, connector result, prior conversation, or tool permission. In a retrieval-augmented generation system, the application may retrieve passages from internal sources before the model runs. In an agentic workflow, the system may also decide whether the AI can call a tool, browse a page, update a record, or trigger a connector action.

The question is not only “Did the model produce a safe answer?” It is also: what data was selected, who assembled the context, which content was trusted, which policy checks ran, which route was allowed, what happened when checks failed, and what evidence exists afterward?

Five moments before the model runs

1. Data is selected

Sensitive information exposure often begins when too much raw material is selected for the AI request. That may include support transcripts, customer records, HR notes, contract text, screenshots, code, account identifiers, or internal strategy documents.

OWASP treats sensitive information disclosure as a major LLM application risk. The NSA, CISA, FBI, and international partners have also framed data used to train and operate AI systems as part of the AI supply chain that requires lifecycle-wide protection. [2][7]

2. Context is assembled

AI workflows rarely send only the user’s text. They assemble context: retrieved documents, search results, conversation memory, metadata, system instructions, and sometimes third-party content. Once assembled, the model sees a combined execution context.

This is where indirect prompt injection becomes relevant. External content in emails, documents, webpages, retrieved snippets, or plugins may contain instructions that are not intended for the model to follow. The NCSC’s core warning is that LLMs do not inherently preserve the same command/data separation that security teams expect from traditional systems. [3]

3. Policy is applied before execution

Provider-native controls are increasingly moving earlier in the execution path. AWS Bedrock’s ApplyGuardrail API can assess text using configured guardrails without invoking a foundation model. Google Cloud’s Model Armor can screen prompts, responses, and supported document types for prompt injection, jailbreak attempts, sensitive data, and malicious URLs.

Microsoft’s Prompt Shields can detect user-prompt attacks and document attacks involving third-party content such as documents and emails. [8][9][10]

4. The provider, model, tool, or connector route is chosen

Enterprise AI rarely lives inside one model endpoint. A request may be routed to different providers, internal models, SaaS copilots, tools, or connectors depending on task, data class, cost, geography, user role, or product feature.

The risk changes when the AI can act. OWASP calls this “excessive agency”: too much functionality, too many permissions, or too much autonomy for the task. Anthropic’s computer-use guidance recommends additional confirmation and safeguards when browser or computer actions are in scope, especially because prompt injection can appear in the content the model is viewing. [2][7][11]

5. Evidence is created or missing

After execution, security and privacy teams often need to reconstruct what happened. But many AI workflows do not preserve enough evidence to answer basic review questions.

Reviewers may need to know what data was selected, whether it was transformed, which provider route was used, whether tool access was granted, whether the request failed closed, whether the provider executed, what policy decision was made, and whether the decision is reviewable. [3][8][10]

Why provider-native controls still matter

It would be a mistake to dismiss provider-native controls. AWS Bedrock Guardrails, Microsoft Prompt Shields, Google Model Armor, Anthropic’s computer-use protections, and other provider controls reflect a clear market direction: AI safety and security controls are moving closer to the point of execution. [8][9][10][11]

But those controls are part of a broader responsibility model. AI workflows often span multiple applications, providers, SaaS tools, internal systems, retrieval layers, and connector paths. A provider-native control may be strong inside one stack, while the organization still needs a way to reason about the full request path.

The Cloud Security Alliance’s AI Controls Matrix is a useful signal here. It defines control objectives across domains and maps to frameworks such as NIST AI RMF and ISO 42001, because AI risk is distributed across data, infrastructure, identity, application, model, and governance layers. [12]

What teams should ask

Before approving a sensitive AI workflow, security, privacy, platform, product, and procurement teams should ask:

• What data is being sent?
• Who or what assembled it?
• Was sensitive content transformed, masked, minimized, or excluded before execution?
• Which provider, model, tool, or connector route is eligible?
• What permissions does that route carry?
• What happens if checks fail?
• Does the request fail closed, or can it still proceed through another path?
• What evidence exists afterward for review?

Where Clariva fits

This is the type of problem Clariva is focused on: proof-bound AI request control before provider execution. Sensitive AI workflows need an explicit control decision before data, tools, or connector actions enter a provider or agent execution path.

Clariva’s perspective is not that provider-native controls are unnecessary. They are useful and will continue to improve. The gap is that enterprise teams often need a vendor-neutral request boundary that can verify, transform, route, reject, and record sensitive AI requests before execution across workflows and providers.

Practical takeaway

Enterprises should evaluate the AI request path before execution, not only the model output after execution. Output safety still matters. But if sensitive data, hidden instructions, over-broad permissions, or unreviewable routing decisions are already inside the execution path, the risk has already begun.

Sources

1. NIST — Artificial Intelligence Risk Management Framework: Generative Artificial Intelligence Profile PDF
2. OWASP — Top 10 for LLM Applications and GenAI Security Project LLM01 prompt injection LLM06 sensitive information disclosure
3. UK NCSC — Prompt injection is not SQL injection
4. NVD — CVE-2025-32711
5. Brave — Indirect Prompt Injection in Perplexity Comet
6. Reuters — Meta AI support chatbot / Instagram account incident reporting
7. CISA / NSA and partner agencies — AI data security and careful adoption of agentic AI services Agentic AI guidance NSA AI data security
8. AWS — Amazon Bedrock ApplyGuardrail API and Bedrock Guardrails documentation Sensitive filters Prompt injection
9. Google Cloud — Model Armor overview
10. Microsoft — Prompt Shields and indirect prompt injection guidance Indirect prompt injection
11. Anthropic — Claude computer-use and prompt-injection mitigation guidance Guardrails
12. Cloud Security Alliance — AI Controls Matrix