AI gateway and assistant

Blouppy.Ai is a module: one gateway that every call to a model goes through, and two features built on it — a workspace description drafted from the workspace's name, and an assistant that answers questions about the organization's data and proposes changes. The model provider sits behind a driven port, ILlmClient, with Anthropic as the one adapter. The Starter builds and runs without a provider key.

What it does

For the people using the app

• Built-in AI is an organization setting, on by default, that owners and administrators change under Organization → AI. It is checked on the server before every call, so turning it off holds for every way in — a button, the assistant, a feature added later.
• A monthly allowance per person. Each member has their own allowance for the calendar month (UTC), under a ceiling for the whole organization. One member's use never takes from another's. Members never see a number: when their allowance is used, the app says when it comes back.
• What owners and administrators see, on the billing page: this month's use against the ceiling, a breakdown per feature, and the five members who used the most — never what anyone asked.
• The organization's own key. On a plan that allows it, an administrator saves the organization's own provider key. Its calls are used instead of the server's key, recorded, and counted against nobody, since the organization pays the provider directly. The key is stored encrypted; only its last four characters are shown again.
• The two features: Draft with AI on a workspace's general settings, which suggests a description the person edits before saving; and the assistant, a panel over the whole shell (the app bar's button, or Cmd+K / Ctrl+K).

Plans

The allowances are Billing's entitlements (BillingCatalogOptions), measured in credits:

One credit is US$0.01 of provider cost at list price (AiCreditPricing.CreditInUsd). A call's cost is its four token counts — input, output, cache read, cache write — at the served model's rate, rounded up, at least one credit; a call that consumed no token costs nothing. The pool is for automatic work nobody asked for at that moment; both features shipped are interactive, so nothing draws on it yet. See Billing for where entitlements come from.

For the developer

• One gateway. A feature names an AiFeature, builds a prompt and calls IAiGateway. The switch, the key, the allowance and the ledger are the gateway's.
• A ledger of every call: ai_usage_records holds the organization, the user, the feature, the provider, the model, the tokens, the credits and the outcome — never what was sent or answered. Rows are purged after 13 months (AiRules.UsageRetentionInMonths) by AiUsageRecordPurgeJob, a daily Quartz.NET job.
• Refusals with a code.GlobalExceptionHandler (Blouppy.Web/ExceptionHandlers/) turns each refusal or failure into problem details with a code, so the front can say what the person can do about it.

How it works

The port is deliberately not in Blouppy.Ai.Abstractions: that is the contract the Web consumes, and a controller that could inject ILlmClient would skip the switch, the allowance and the ledger. LlmBoundaryTests (Blouppy.Architecture.UnitTests) fails the build if any type other than Blouppy.Ai.Gateway.AiGateway injects it.

A call

• A refusal is not a call: nothing is written and nothing is charged.
• The model and the effort are configuration. An AiPrompt carries the system prompt, the messages, an optional JSON schema and an output ceiling — no model. The gateway applies Ai:Features:{feature}, so no feature picks its own.
• The ledger never fails a request. By the time the row is written the provider has answered; an error writing it is logged. Only Completed and Cancelled outcomes are charged; Refused, Failed and Unusable are recorded and counted against nobody. A call that started with credits left is charged at most what was left.
• Who asked is sent to the provider as an opaque hash (EndUserIds), keyed by Ai:EndUserIdSalt; none is sent when the salt is empty.
• The front never offers a call the gateway would refuse.GetAvailabilityAsync runs the three checks as questions; GET /api/v1/organizations/{id}/ai returns them to the SPA's AiAvailabilityService.

The organization's own key

OrganizationAiKeyService encrypts the key with ISecretProtector (purpose Blouppy.Ai.OrganizationAiKey.v1) and stores the ciphertext and the last four characters in ai_organization_keys. OrganizationLlmClientResolver hands the gateway a client on that key when the plan has AllowsOwnAiKey and a key is saved, the server's client otherwise; OrganizationLlmClients caches one client per organization. The endpoints are under /api/v1/organizations/{id}/ai/key (GET, PUT, DELETE, and POST test, which asks the provider through ILlmKeyVerifier whether the key works, sending nothing of the organization's). Saving and removing the key, and turning Built-in AI on or off, are recorded in the audit log.

The assistant

POST /api/v1/organizations/{id}/ai/assistant/turns answers with server-sent events in the response to that POST: conversation, text, tool, proposal, usage and done, with a keep-alive comment every 15 seconds. A refusal comes before the stream opens, as problem details.

• Each round is one call through the gateway (AiFeature.Assistant, via IAssistantGateway), checked and charged like any other, and grouped on the ledger by the turn's id. A turn is bounded by Ai:Assistant:MaxRounds (10) and Ai:Assistant:MaxOutputTokensPerRound (16,000).
• The tools are the same catalog as the MCP server, confined to the organization the panel is in. Anyone who may read the organization may ask; only someone who may contribute is offered the write tools.
• Browser only. A turn made with a token is refused (403).
• Writes are proposed, not run. A write tool call ends the turn on a proposal of up to 10 changes (AssistantProposal.MaxActions), kept for 30 minutes. Apply runs the chosen ones with the person's rights at that moment, stamped as made through the assistant; a proposal is decided once (409 ai.proposalDecided) and expires (410 ai.proposalExpired). Applying calls no model.
• Conversations are kept per person and organization, the last 24 exchanges replayed to the model on the next turn (AssistantTurnService.MaxReplayedExchanges).

Configuration

Set on the API.

AiOptionsValidator refuses to start on a feature without a model, or a model without a credit rate. In development, give the AppHost the key, which it passes to the API:

dotnet user-secrets set "Parameters:anthropic-apiKey" "sk-ant-…" --project back/Blouppy.Aspire.AppHost

Extending it

The workspace description draft (back/Blouppy.Ai/Writing/) is the worked example.

Step 7, in the example service (Writing/WorkspaceDescriptionDraftService.cs):

var draft = await gateway.CompleteAsAsync<WorkspaceDescriptionAnswer, WorkspaceDescriptionDraft>(
    new AiCall(organizationId, request.UserId, AiFeature.WorkspaceDescriptionDraft),
    WorkspaceDescriptionPrompt.For(request.Name),
    ToDraft,
    logger,
    cancellationToken);

CompleteAsAsync (Answers/StructuredAnswers.cs) reads the JSON answer against the schema and hands it to ToDraft, which checks what a schema cannot carry — here the 500-character limit — and throws AiAnswerException(AiAnswerFailure.Unusable) otherwise. That call is then recorded Unusable and counted against nobody. Test the service against FakeLlmClient (Blouppy.Shared.TestHelpers/Llm/).