fastrag.query.llm.base

The ChatModel contract: messages to a generated answer.

Classes

ChatModel()	Turns a conversation into a generated answer.
Message(role, content)	One turn in a chat conversation.

class fastrag.query.llm.base.Message(role, content)

Bases: object

One turn in a chat conversation.

Parameters:

• role (Literal['system', 'user', 'assistant'])

• content (str)

role

Who sent the turn.

Type:

Literal[‘system’, ‘user’, ‘assistant’]

content

The turn’s text.

Type:

str

class fastrag.query.llm.base.ChatModel

Bases: ABC

Turns a conversation into a generated answer.

invoke and stream are separate methods, not one method with a flag, because a streaming caller and a one-shot caller want different return shapes — tokens as they land versus one finished string — and forcing both through one signature would make one of the two callers reconstruct what it actually wanted from the other’s shape.

abstractmethod async invoke(messages)

Generate the whole answer at once, as plain text.

Parameters:

messages (Sequence[Message])

Return type:

str

abstractmethod stream(messages)

Generate the answer as text deltas, nothing else.

Yields plain text tokens only, so a provider that cannot report usage mid-stream never has to fabricate it.

Parameters:

messages (Sequence[Message])

Return type:

AsyncIterator[str]

abstract property model_id: str

Stable identifier of the model, for logging and events.