Pydantic AI Server inference

Configure the Server’s independent generation and embedding providers, then understand their lifecycle and failure boundaries.

Pydantic AI is used here inside the PowerContext Server. It supplies schema-bound generation and an independent embedding provider. It does not install the Pydantic AI Agent adapter, add tools to an Agent, or share that adapter’s lifecycle.

Status: available in the built-in runtime at the upstream revision used for this documentation. Install powercontext[builtin] for in-process composition or powercontext[server] for the Server. Python 3.11 or later is required. Calls to real providers, credentials, and production latency were not validated in this verification.

Choose one of four runtime states

Generation and embedding are optional and independent. A model-free Server remains useful: explicit Memory and FTS do not need either provider.

State	Configuration	What becomes available
No model	Neither model configured	Explicit Memory, FTS, typed proposals and Review, and typed Handoff operations
Generation only	generation_model	Source extraction, Experience incubation and generation, Skill generation, Handoff generation, and optional Memory reranking
Embedding only	Complete embedding profile	Vector and hybrid indexing/search; explicit Memory writes remain available
Both	Generation plus complete embedding profile	All of the above; the two providers still fail independently

Generation does not enable the Scheduler. You may call flush_memory manually, or configure POWERCONTEXT_SERVER_RUNTIME_SCHEDULE_SECONDS separately.

Configure the Server

Install one of the supported roles. This pin is the same commit as the LangGraph and LangChain pages, written as a full hash:

uv add "powercontext[builtin] @ git+https://github.com/oceanbase/powercontext.git@3e629eb163ba0ddf2935ce3b3fbaced0f9371977"
uv add "powercontext[server] @ git+https://github.com/oceanbase/powercontext.git@3e629eb163ba0ddf2935ce3b3fbaced0f9371977"

The Server settings use Pydantic AI model strings. Provider-specific environment variables, such as credentials and base URLs, are read by the selected provider. POWERCONTEXT_SERVER_INFERENCE_GENERATION_MODEL_SETTINGS accepts a JSON object shared by generation pipelines, reranking, and the readiness probe. Put provider request fields under extra_body; keep credentials and static headers in provider configuration. Non-empty settings without a generation model, or settings containing extra_headers, fail configuration validation.

export POWERCONTEXT_SERVER_INFERENCE_GENERATION_MODEL="openai:gpt-5.2"
export POWERCONTEXT_SERVER_INFERENCE_GENERATION_MODEL_SETTINGS='{}'
export POWERCONTEXT_SERVER_INFERENCE_GENERATION_TIMEOUT_SECONDS="30"
export POWERCONTEXT_SERVER_INFERENCE_GENERATION_MAX_REQUESTS="2"

export POWERCONTEXT_SERVER_INFERENCE_EMBEDDING_MODEL="openai:text-embedding-3-small"
export POWERCONTEXT_SERVER_INFERENCE_EMBEDDING_PROFILE_ID="openai-text-embedding-3-small-v1"
export POWERCONTEXT_SERVER_INFERENCE_EMBEDDING_DIMENSION="1536"
export POWERCONTEXT_SERVER_INFERENCE_EMBEDDING_NORMALIZATION="unit"
export POWERCONTEXT_SERVER_INFERENCE_EMBEDDING_TIMEOUT_SECONDS="30"
export POWERCONTEXT_SERVER_INFERENCE_EMBEDDING_BATCH_SIZE="10"

uv run powercontext server run

EMBEDDING_MODEL, EMBEDDING_PROFILE_ID, and EMBEDDING_DIMENSION are all-or-none. Partial configuration is rejected before startup. The profile is part of the persisted vector contract, so choose its ID, dimension, and normalization deliberately rather than changing them in place.

A model string is configuration, not evidence that a provider works. Readiness sends a small live request, but the documented check does not certify a provider account, credential, quota, regional endpoint, model dimension, or acceptable latency.

Use the structured generator directly

The low-level generator accepts an already constructed and opened Pydantic AI Model; it deliberately rejects a model string. The caller owns that model’s lifecycle.

from pydantic import BaseModel
from pydantic_ai.models import infer_model

from powercontext.builtin.inference.pydantic_ai import PydanticAIStructuredGenerator


class ExtractionInput(BaseModel):
    text: str


class ExtractionOutput(BaseModel):
    summary: str


async def generate_summary() -> ExtractionOutput:
    async with infer_model("openai:gpt-5.2") as model:
        generator = PydanticAIStructuredGenerator(
            model=model,
            instructions="Return a concise factual summary.",
            input_type=ExtractionInput,
            output_type=ExtractionOutput,
        )
        result = await generator.generate(ExtractionInput(text="The client must be regenerated."))
        return result.output

Server composition is different: InferenceConfig.generation_model accepts a model string, opens the provider model while the runtime is active, reuses it across the generation pipelines, and closes it with the runtime.

Follow the execution flow

Generation flow

model string
→ Server opens one provider Model
→ schema-bound PydanticAIStructuredGenerator
→ extraction / incubation / generation / Handoff / optional rerank pipeline
→ domain validation and evidence checks
→ explicit durable write or Candidate Review boundary

Pydantic AI validates the declared output schema. That is only the first check. PowerContext still validates domain rules, evidence references, current versions, and other invariants before generated data can become authoritative state. Generated Experience and Skill content still goes through Candidate Review; structured output is not approval.

Embedding flow

embedding model + profile ID + dimension
→ Server opens the provider and Embedder
→ bounded document batches
→ verify input type, order, count, dimension, normalization, and finite values
→ best-effort write projection or vector/hybrid query

Generation and embedding do not fall back to each other, and there is no cross-provider fallback. One may be healthy while the other is degraded.

Understand search and write failures

A provider outage affects only the paths that depend on that provider. The table separates those paths from durable state that remains authoritative in storage.

Path	Provider failure behavior	Authoritative data
Search with mode="auto"	A temporary query-embedding failure may fall back to FTS when FTS is supported	No durable state changes
Search with mode="vector" or mode="hybrid"	Returns an error; it does not silently switch to FTS	No durable state changes
Write-time projection embedding	Best effort; projection failure does not discard the committed Memory	Memory remains authoritative and can still be found through FTS
Generation-backed operation	Timeout or unavailability is surfaced to the operation	Invalid or missing generated output is not treated as a successful write

Provider timeouts and temporary unavailability map to PowerContext inference errors and Server HTTP 503 responses. Invalid output, including reordered embeddings, a wrong vector count or dimension, and non-finite values, is rejected rather than indexed.

Do not interpret a successful FTS fallback as proof that vector search worked. To verify embedding, make an explicit vector request and require a source-linked hit whose matched_by includes vector.

Check readiness without overstating it

Configured generation and embedding each add a non-blocking readiness probe. A failed inference probe reports degraded with HTTP 200, not not_ready. Runtime or database failure is the blocking case and returns not_ready with HTTP 503.

powercontext ready
powercontext capabilities

A degraded Server can still perform database-backed operations that do not need the failed provider. An operation that needs that provider may return 503.

Readiness checks availability at that moment. It does not validate production quotas, every schema used by the pipelines, sustained concurrency, data quality, or end-to-end latency.

Set privacy and Scope boundaries

The provider receives the serialized input for the operation. Depending on the pipeline, that can include Source content, Memory candidates, a query, Handoff evidence, or Candidate-generation evidence. Provider usage records request and token counts; optional instrumentation may emit inference spans. Configure provider data handling and tracing before sending sensitive project material.

Scope still selects PowerContext data. It does not stop selected content from leaving the Server for the configured model, prove user identity, grant model permissions, or approve generated content. Keep secrets out of Source and Memory when the provider must not receive them.

Currently unsupported or not validated

Item	Status
Pydantic AI Agent tools or lifecycle hooks	Unsupported here; use the separate Agent adapter
Generation-to-embedding failover or cross-provider failover	Unsupported
Silent fallback for explicit vector or hybrid search	Unsupported
Schema validation as a substitute for domain invariants or Review	Unsupported
Real provider credentials, quotas, regional endpoints, and latency	Not validated in this verification
Automatic Scheduler activation from generation configuration	Unsupported; configure scheduling separately

Completion check

Before relying on inference, confirm that:

• you chose one of the four states intentionally;
• the three required embedding fields are either all set or all absent;
• powercontext ready distinguishes ready, degraded, and not_ready correctly;
• powercontext capabilities lists only the search and generation capabilities you configured;
• explicit vector verification fails visibly rather than being mistaken for FTS fallback;
• your provider privacy, tracing, and retention settings fit the data sent by each pipeline;
• downstream domain validation and Candidate Review remain in place after schema validation.

For a first integration, go back to LangGraph or LangChain. The Pydantic AI Agent adapter is still Preview and currently has no working install command, so read it only when you need it.