Skip to main content

LLM

Charivo's LLM layer is built from two pieces:

  • @charivo/llm for conversation state
  • an LLMClient implementation for transport

For production browser apps, pair @charivo/llm with @charivo/llm/remote and a server route backed by a provider package.

@charivo/llm
@charivo/llm/remote
your /api/chat route
@charivo/server/openai

This keeps the browser client simple and vendor credentials on the server.

Basic Setup​

import { createCharivo } from "@charivo/core";
import { createLLMManager } from "@charivo/llm";
import { createRemoteLLMClient } from "@charivo/llm/remote";

const charivo = createCharivo({
  llm: createLLMManager(createRemoteLLMClient({ apiEndpoint: "/api/chat" })),
  character: {
    id: "hiyori",
    name: "Hiyori",
    personality: "Cheerful and helpful assistant",
  },
});

createCharivo applies the character after attaching, so it reaches the LLM, rendering, and realtime managers together. When you attach by hand instead, call charivo.setCharacter(...) after the attach* calls to keep that character state aligned.

Client Choices​

Remote​

  • @charivo/llm/remote
  • best default for production browser apps
  • expects your route to receive messages and return { success, message }

Direct OpenAI​

  • @charivo/llm/openai
  • useful for local development and testing
  • exposes credentials to the browser

Direct Gemini​

  • @charivo/llm/gemini
  • useful for local development and testing
  • exposes credentials to the browser

Direct OpenClaw​

  • @charivo/llm/openclaw
  • useful when your app targets an OpenClaw deployment directly
  • best treated as a development or trusted-environment option unless browser access is intentional

Stub​

  • @charivo/llm/stub
  • useful for UI work, deterministic demos, and tests

Provider Choices​

Remote clients pair with provider packages on the server:

  • @charivo/server/openai
  • @charivo/server/openclaw
  • @charivo/server/gemini — Gemini 3 rejects a resent tool-call turn that lacks a thought signature, so this provider sends Google's documented skip_thought_signature_validator placeholder on the first tool call of each turn, and reasoning continuity across tool rounds is lost

Minimal OpenAI route shape:

const provider = createOpenAILLMProvider({
  apiKey: process.env.OPENAI_API_KEY!,
  model: "gpt-4.1-nano",
});

const text = await provider.generateResponse(messages);

What @charivo/llm Owns​

  • message history
  • character-aware prompt building
  • response generation through an injected client

The client is replaceable. The manager remains the stable place for conversation state.

Avatar Tool Calling​

LLMManager can drive @charivo/avatar's tools the same way RealtimeManager does, on top of the recommended remote stack. For what belongs in the catalog below — and how to establish what an expression id actually means — see Avatar Control. The tool loop turns on only when both a tool is registered and the client implements callWithTools — @charivo/llm/remote does, provided your route forwards tools to a provider's generateResponseWithTools.

import { createLLMManager } from "@charivo/llm";
import { createRemoteLLMClient } from "@charivo/llm/remote";
import {
  buildAvatarControlInstructions,
  createAvatarControlTools,
  createAvatarResultProjector,
} from "@charivo/avatar";

const catalog = {
  expressions: ["Smile", "Sad"],
  motions: { Idle: 2, TapBody: 2 },
  expressionDescriptions: { Smile: "happy or amused", Sad: "downcast or disappointed" },
  motionDescriptions: { Idle: ["resting", "shifting weight"], TapBody: ["waves hello", "folds her arms"] },
};

const manager = createLLMManager(
  createRemoteLLMClient({ apiEndpoint: "/api/chat" }),
  {
    tools: createAvatarControlTools(catalog),
    resultProjectors: [createAvatarResultProjector()],
    toolInstructions: buildAvatarControlInstructions(catalog),
  },
);

charivo.attachLLM(manager);

attachLLM(...) wires the event emitter resultProjectors need to turn successful tool calls into avatar:expression / avatar:motion / avatar:gaze events; a RenderManager on the same Charivo instance already listens for them. The same emitter publishes tool:call / tool:result / tool:error around every tool execution — the events RealtimeManager emits too, so tool activity stays observable from one place.

On the server side, your route needs to accept an optional tools array and call the tool-calling variant of your provider whenever the request needs it — that includes both a tools-carrying request (even tools: [], the terminal round) and a plain request whose messages already contain a tool-call or tool-result turn:

const { messages, tools } = parsedBody;
const needsTools = requiresToolCallingPath(parsedBody);

const result = needsTools
  ? await provider.generateResponseWithTools(messages, tools ?? [])
  : { content: await provider.generateResponse(messages) };

return NextResponse.json({
  success: true,
  message: result.content,
  toolCalls: "toolCalls" in result ? result.toolCalls : undefined,
});

See examples/web/src/app/api/chat-request.ts for the full request-parsing and validation this demo uses (parseChatRequest and requiresToolCallingPath).

The tool loop runs at most 3 rounds before a final tools: [] call forces a text-only reply, and only the final assistant text is added to LLMManager's history — see Tool Calling in the package README for the full round-cap and remote protocol details.

History Retention​

LLMManager keeps the latest 40 turns by default. A turn is one user message plus one character response, so getHistory() and LLM client calls are bounded to the latest 80 stored messages. This keeps long-running chat sessions from growing memory and context cost without additional app code.

Override the limit with createLLMManager(client, { maxHistoryTurns }), or use maxHistoryTurns: null for unbounded history.

Eviction trims to the exact bound and takes any reply it strands at the head with it, so overlapping generateResponse(...) calls at a tight limit cannot leave a transcript of replies that answer nothing.

Realtime sessions maintain conversation state on the provider side and are not affected by maxHistoryTurns.

Under Charivo Orchestration​

The turn's history writes belong to Charivo, not to the manager: userSay(text) places the user message through addToHistory(...), calls generateResponse(..., { callerOwnsHistory: true }) so the manager writes nothing for that call, and commits the reply once the turn reaches presentation. That is what keeps a superseded turn's user message in history while its unspoken reply never enters it — see the latest-wins turn contract in the core README. maxHistoryTurns applies to those messages like any other.

Charivo also passes a per-turn signal alongside isCancelled, so a superseded turn's in-flight request is aborted when the client honors it. @charivo/llm/remote does; the direct OpenAI and Gemini dev clients ignore it and run to completion. A custom LLMManager should forward signal to its client — the built-in one already does.

Alternatives​

  • Use OpenClaw when your backend or testing flow targets OpenClaw instead of OpenAI.
  • Use Gemini when you already hold a Gemini API key for Gemini Live realtime voice and want one vendor for both text chat and voice.
  • Use the stub client when you want UI behavior without network or model variability.
  • Use direct browser clients only when development speed matters more than credential isolation.

References​