Runtime Request Pipeline

The UnieAI LLM Runtime exposes OpenAI-compatible APIs and handles authentication, model access, feature configuration, upstream providers, billing, and observability within each request. The stages vary by API key, model type, Agent Core, streaming mode, and custom-model configuration; requests do not all pass through one fixed linear chain.

1. Authenticate the request

The runtime checks the bearer token, platform license, key status, expiry, rate limits, and whether the requested model supports the endpoint.

An API key does not bypass model, custom-model, or resource permissions. For an organization key, access to a custom model is resolved against the key's organization rather than the creator's personal memberships in other organizations.

Access and model resolution are currently cached for about three seconds, but rate limits are checked on every request and cannot be bypassed through the cache.

2. Resolve the model and execution path

After authentication, the runtime resolves the base or custom model named by the API model field and confirms support for an endpoint such as /v1/chat/completions.

Chat requests use one of three main paths.

Agent Core path

When a custom model selects Agent Core 1.0, the request enters the native function-calling loop. The core loads custom-model settings, core memory, knowledge bases, SQL, APIs, MCP, skills, and other tools allowed for the request, then alternates model steps and tool results.

Agent Core handles its own guardrails, context compaction, tool timeouts, step limits, and stream events. See Agent Core 1.0 and Agent Orchestration.

Plain model path

When a custom model selects Plain model, the request enters a dedicated path with five stages: input guardrail → Kernel Prompt and memory injection → one base-model call → stream → output guardrail.

This path builds no tools, loads no data map or knowledge base catalogue, performs no context compaction, and does not override the base model's thinking setting. The system prompt consists only of an identity-and-date line, the Kernel Prompt, the memory block, and two safety rules. Billing, request logs, usage, and evaluation behave exactly as on the other paths. See Kernel Prompt for how that prompt is composed.

Base and non-Agent custom-model path

Without Agent Core, the runtime prepares the request according to model configuration. Preparation can include:

• Base-model or custom-model system prompts
• Base-model web-search context
• Custom-model knowledge retrieval and memory context
• Configured API, remote MCP, SQL, or skill flows
• Thinking settings supported by Qwen-based models

Each feature runs only when enabled, available, and authorized. RAG, memory, MCP, and skills are not mandatory stages on every request, and there is no universal standalone LLM router for them.

3. Input guardrail

When guardrails are enabled for a custom model, the runtime checks input text according to its configuration. If blocked, it immediately returns the configured refusal and skips the base-model and tool calls.

Platform guardrail settings are currently cached for about three seconds. Whether a judge-service failure blocks or allows a request depends on the fail-open configuration, so only a completed check should be treated as a definitive verdict.

4. Reserve billing capacity

Before a billable upstream call, the runtime reserves wallet capacity from estimated input tokens, maximum output tokens, and relevant additional usage:

• Personal requests reserve from the personal wallet
• Non-personal organization requests reserve from the organization wallet
• Insufficient balance fails before the upstream call

A reservation is not the final charge. Success is settled from actual usage; on failure the runtime attempts to release any unsettled reservation.

5. Select an upstream connection

A model can have one or more upstream connections. The runtime selects an attempt order from current routing and weights and translates provider formats such as OpenAI and Anthropic into the common runtime shape.

Before stream headers or content reach the client, an eligible connection error or upstream status can trigger an attempt on the next candidate. After SSE output starts, the runtime cannot switch upstreams and replay the answer without duplicating content.

6. Normalize response and streaming

Non-streaming calls return OpenAI-compatible JSON. Streaming calls use SSE and end with [DONE]. The runtime normalizes model names, choices, usage, and provider-specific differences.

A custom model can also include UnieAI extension metadata such as:

• rag_metadata
• memory_metadata
• websearch_metadata
• api_metadata
• mcp_metadata
• mysql_metadata
• skills_metadata
• agent_metadata or guardrail_metadata

Third-party SDKs that ignore unknown fields can still read the normal answer. Applications that display a tool timeline must parse the relevant extensions.

7. Output guardrail

Assistant output is checked only when guardrails are enabled and configured to Apply to output:

• Non-streaming: check the complete answer and replace blocked content with the refusal
• Streaming: use the path's output gate to check windows of text that have not yet been released

Text already sent to the client cannot be recalled. Do not claim that a streaming guardrail can retroactively revoke every earlier chunk.

8. Settlement, logs, and observability

After completion, the runtime settles the reservation from actual usage and writes available data to Request Logs and usage summaries. Records can include:

The service also emits structured JSON logs, but stdout is not a replacement for the Request Logs database or wallet ledger. If log persistence fails, the runtime records a warning; not every external failure is guaranteed to leave a complete trace.

Caching

Common caches currently include:

These are implementation defaults and can change between deployments. They are not API compatibility guarantees. Key rate limits are still enforced per request.

Related documentation