> ## Documentation Index
> Fetch the complete documentation index at: https://powercontext.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Pydantic AI Server inference

> Configure the Server's independent generation and embedding providers, then understand their lifecycle and failure boundaries.

Pydantic AI is used here inside the PowerContext Server. It supplies schema-bound generation and an independent embedding provider. It does not install the [Pydantic AI Agent adapter](https://powercontext.mintlify.site/en/frameworks/pydantic-ai.md), add tools to an Agent, or share that adapter's lifecycle.

> [!NOTE]
>
> Status: available in the built-in runtime at the upstream revision used for this documentation. Install `powercontext[builtin]` for in-process composition or `powercontext[server]` for the Server. Python 3.11 or later is required. Calls to real providers, credentials, and production latency were not validated in this verification.

## Choose one of four runtime states

Generation and embedding are optional and independent. A model-free Server remains useful: explicit Memory and FTS do not need either provider.

| State | Configuration | What becomes available |
| - | - | - |
| No model | Neither model configured | Explicit Memory, FTS, typed proposals and Review, and typed Handoff operations |
| Generation only | `generation_model` | Source extraction, Experience incubation and generation, Skill generation, Handoff generation, and optional Memory reranking |
| Embedding only | Complete embedding profile | Vector and hybrid indexing/search; explicit Memory writes remain available |
| Both | Generation plus complete embedding profile | All of the above; the two providers still fail independently |

Generation does not enable the Scheduler. You may call `flush_memory` manually, or configure `POWERCONTEXT_SERVER_RUNTIME_SCHEDULE_SECONDS` separately.

## Configure the Server

Install one of the supported roles. This pin is the same commit as the [LangGraph](https://powercontext.mintlify.site/en/frameworks/langgraph.md) and [LangChain](https://powercontext.mintlify.site/en/frameworks/langchain.md) pages, written as a full hash:

```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
uv add "powercontext[builtin] @ git+https://github.com/oceanbase/powercontext.git@3e629eb163ba0ddf2935ce3b3fbaced0f9371977"
uv add "powercontext[server] @ git+https://github.com/oceanbase/powercontext.git@3e629eb163ba0ddf2935ce3b3fbaced0f9371977"
```

The Server settings use Pydantic AI model strings. Provider-specific environment variables, such as credentials and base URLs, are read by the selected provider. `POWERCONTEXT_SERVER_INFERENCE_GENERATION_MODEL_SETTINGS` accepts a JSON object shared by generation pipelines, reranking, and the readiness probe. Put provider request fields under `extra_body`; keep credentials and static headers in provider configuration. Non-empty settings without a generation model, or settings containing `extra_headers`, fail configuration validation.

```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
export POWERCONTEXT_SERVER_INFERENCE_GENERATION_MODEL="openai:gpt-5.2"
export POWERCONTEXT_SERVER_INFERENCE_GENERATION_MODEL_SETTINGS='{}'
export POWERCONTEXT_SERVER_INFERENCE_GENERATION_TIMEOUT_SECONDS="30"
export POWERCONTEXT_SERVER_INFERENCE_GENERATION_MAX_REQUESTS="2"

export POWERCONTEXT_SERVER_INFERENCE_EMBEDDING_MODEL="openai:text-embedding-3-small"
export POWERCONTEXT_SERVER_INFERENCE_EMBEDDING_PROFILE_ID="openai-text-embedding-3-small-v1"
export POWERCONTEXT_SERVER_INFERENCE_EMBEDDING_DIMENSION="1536"
export POWERCONTEXT_SERVER_INFERENCE_EMBEDDING_NORMALIZATION="unit"
export POWERCONTEXT_SERVER_INFERENCE_EMBEDDING_TIMEOUT_SECONDS="30"
export POWERCONTEXT_SERVER_INFERENCE_EMBEDDING_BATCH_SIZE="10"

uv run powercontext server run
```

`EMBEDDING_MODEL`, `EMBEDDING_PROFILE_ID`, and `EMBEDDING_DIMENSION` are all-or-none. Partial configuration is rejected before startup. The profile is part of the persisted vector contract, so choose its ID, dimension, and normalization deliberately rather than changing them in place.

> [!WARNING]
>
> A model string is configuration, not evidence that a provider works. Readiness sends a small live request, but the documented check does not certify a provider account, credential, quota, regional endpoint, model dimension, or acceptable latency.

## Use the structured generator directly

The low-level generator accepts an already constructed and opened Pydantic AI `Model`; it deliberately rejects a model string. The caller owns that model's lifecycle.

```python theme={"theme":{"light":"github-light","dark":"github-dark"}}
from pydantic import BaseModel
from pydantic_ai.models import infer_model

from powercontext.builtin.inference.pydantic_ai import PydanticAIStructuredGenerator


class ExtractionInput(BaseModel):
    text: str


class ExtractionOutput(BaseModel):
    summary: str


async def generate_summary() -> ExtractionOutput:
    async with infer_model("openai:gpt-5.2") as model:
        generator = PydanticAIStructuredGenerator(
            model=model,
            instructions="Return a concise factual summary.",
            input_type=ExtractionInput,
            output_type=ExtractionOutput,
        )
        result = await generator.generate(ExtractionInput(text="The client must be regenerated."))
        return result.output
```

Server composition is different: `InferenceConfig.generation_model` accepts a model string, opens the provider model while the runtime is active, reuses it across the generation pipelines, and closes it with the runtime.

## Follow the execution flow

### Generation flow

```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
model string
→ Server opens one provider Model
→ schema-bound PydanticAIStructuredGenerator
→ extraction / incubation / generation / Handoff / optional rerank pipeline
→ domain validation and evidence checks
→ explicit durable write or Candidate Review boundary
```

Pydantic AI validates the declared output schema. That is only the first check. PowerContext still validates domain rules, evidence references, current versions, and other invariants before generated data can become authoritative state. Generated Experience and Skill content still goes through Candidate Review; structured output is not approval.

### Embedding flow

```text theme={"theme":{"light":"github-light","dark":"github-dark"}}
embedding model + profile ID + dimension
→ Server opens the provider and Embedder
→ bounded document batches
→ verify input type, order, count, dimension, normalization, and finite values
→ best-effort write projection or vector/hybrid query
```

Generation and embedding do not fall back to each other, and there is no cross-provider fallback. One may be healthy while the other is degraded.

## Understand search and write failures

A provider outage affects only the paths that depend on that provider. The table separates those paths from durable state that remains authoritative in storage.

| Path | Provider failure behavior | Authoritative data |
| - | - | - |
| Search with `mode="auto"` | A temporary query-embedding failure may fall back to FTS when FTS is supported | No durable state changes |
| Search with `mode="vector"` or `mode="hybrid"` | Returns an error; it does not silently switch to FTS | No durable state changes |
| Write-time projection embedding | Best effort; projection failure does not discard the committed Memory | Memory remains authoritative and can still be found through FTS |
| Generation-backed operation | Timeout or unavailability is surfaced to the operation | Invalid or missing generated output is not treated as a successful write |

Provider timeouts and temporary unavailability map to PowerContext inference errors and Server HTTP 503 responses. Invalid output, including reordered embeddings, a wrong vector count or dimension, and non-finite values, is rejected rather than indexed.

> [!WARNING]
>
> Do not interpret a successful FTS fallback as proof that vector search worked. To verify embedding, make an explicit vector request and require a source-linked hit whose `matched_by` includes `vector`.

## Check readiness without overstating it

Configured generation and embedding each add a non-blocking readiness probe. A failed inference probe reports `degraded` with HTTP 200, not `not_ready`. Runtime or database failure is the blocking case and returns `not_ready` with HTTP 503.

```bash theme={"theme":{"light":"github-light","dark":"github-dark"}}
powercontext ready
powercontext capabilities
```

A degraded Server can still perform database-backed operations that do not need the failed provider. An operation that needs that provider may return 503.

Readiness checks availability at that moment. It does not validate production quotas, every schema used by the pipelines, sustained concurrency, data quality, or end-to-end latency.

## Set privacy and Scope boundaries

The provider receives the serialized input for the operation. Depending on the pipeline, that can include Source content, Memory candidates, a query, Handoff evidence, or Candidate-generation evidence. Provider usage records request and token counts; optional instrumentation may emit inference spans. Configure provider data handling and tracing before sending sensitive project material.

Scope still selects PowerContext data. It does not stop selected content from leaving the Server for the configured model, prove user identity, grant model permissions, or approve generated content. Keep secrets out of Source and Memory when the provider must not receive them.

## Currently unsupported or not validated

| Item | Status |
| - | - |
| Pydantic AI Agent tools or lifecycle hooks | **Unsupported** here; use the separate Agent adapter |
| Generation-to-embedding failover or cross-provider failover | **Unsupported** |
| Silent fallback for explicit vector or hybrid search | **Unsupported** |
| Schema validation as a substitute for domain invariants or Review | **Unsupported** |
| Real provider credentials, quotas, regional endpoints, and latency | **Not validated** in this verification |
| Automatic Scheduler activation from generation configuration | **Unsupported**; configure scheduling separately |

## Completion check

Before relying on inference, confirm that:

* you chose one of the four states intentionally;
* the three required embedding fields are either all set or all absent;
* `powercontext ready` distinguishes `ready`, `degraded`, and `not_ready` correctly;
* `powercontext capabilities` lists only the search and generation capabilities you configured;
* explicit vector verification fails visibly rather than being mistaken for FTS fallback;
* your provider privacy, tracing, and retention settings fit the data sent by each pipeline;
* downstream domain validation and Candidate Review remain in place after schema validation.

For a first integration, go back to [LangGraph](https://powercontext.mintlify.site/en/frameworks/langgraph.md) or [LangChain](https://powercontext.mintlify.site/en/frameworks/langchain.md). The [Pydantic AI Agent adapter](https://powercontext.mintlify.site/en/frameworks/pydantic-ai.md) is still Preview and currently has no working install command, so read it only when you need it.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.