fastrag.serve

Serving: a FastAPI app that persists chats and streams grounded answers.

The public surface is exactly create_app(), ServeConfig, cleanup_expired(), CleanupReport, and CLI. Everything else is private.

Importing this package does not require FastAPI or Typer: only create_app() needs FastAPI and only CLI needs Typer, and a missing serving extra surfaces as a ConfigurationError naming the install command — never a bare ImportError.

class fastrag.serve.CLI(backend_url, *, token_file=None, token=None, transport=None)

Bases: object

An interactive terminal client for the serving API, built on Typer.

CLI(backend_url=...) returns an object; calling it (cli()) runs the interactive loop. backend_url is the base URL of a serving app started from create_app(). token_file points at where the session token is read from and written to (default ~/.fastrag/session); token short-circuits all token discovery by supplying the credential directly.

Typer is imported lazily inside the constructor, so merely importing this class needs no serving extra. Only constructing a CLI does.

Parameters:

• backend_url (str)

• token_file (Path | str | None)

• token (str | None)

• transport (httpx.AsyncBaseTransport | None)

property token: str | None

The current session token (bare credential — handle with care).

class fastrag.serve.CleanupReport(sessions, chats)

Bases: object

How many expired rows were deleted.

Parameters:

• sessions (int)

• chats (int)

sessions: int

chats: int

class fastrag.serve.ServeConfig(*, allowed_origins, cookie_secure=True, cookie_domain=None, session_ttl_days=30, expiry_bump_min_days=1, max_request_bytes=32768, max_content_chars=4000, max_messages_per_chat=200, max_chats_per_session=100, generation_timeout_s=120.0, keepalive_interval_s=15.0, log_level='INFO', bearer_tokens=False, metrics_enabled=True, metrics_token=None, trust_proxy_headers=False, rate_limiting_enabled=False, rate_limit_generation='10/minute', rate_limit_read='60/minute', rate_limit_new_session='20/hour', geoip_database_path=None)

Bases: object

Configuration for the serving app.

Frozen and keyword-only. Nothing under src/fastrag/ reads os.environ — the caller reads the environment and passes the result in, exactly as 08_sources.py already does for the chat model. Fail fast rather than start in a degraded state: every field is validated in __post_init__, raising ConfigurationError earlier than startup, at the line that got it wrong.

Parameters:

• allowed_origins (list[str]) – Origins allowed by CORS. The one required field, with no default. A "*" wildcard is rejected: credentialed requests cannot use Access-Control-Allow-Origin: *. The app configures CORS from this list itself, so create_app() callers never add their own CORSMiddleware.

• cookie_secure (bool) – Whether the session cookie carries the Secure flag.

• cookie_domain (str | None) – Optional Domain for the session cookie.

• session_ttl_days (int) – Days a session (and its chats) live without activity.

• expiry_bump_min_days (int) – Minimum headroom before an expiry bump is written.

• max_request_bytes (int) – Reject a request body larger than this with 413.

• max_content_chars (int) – Reject a content longer than this with 422.

• max_messages_per_chat (int) – Cap on messages per chat (422 beyond).

• max_chats_per_session (int) – Cap on chats per session (422 beyond).

• generation_timeout_s (float) – End-to-end budget (retrieval + generation) per generation; a timeout lands as TIMEOUT.

• keepalive_interval_s (float) – Seconds between SSE keep-alive comments, and between pings on the chat’s lock connection while it answers. Keep it below the idle timeout of any proxy in front of the app or of Postgres.

• log_level (str) – Root logger level for the JSON log output.

• bearer_tokens (bool) – Enable the bearer-token path and POST /v1/sessions. Off by default — see “Bearer tokens, in full” in the serving design notes.

• metrics_enabled (bool) – Whether GET /metrics is mounted at all. On by default; every series is still collected when this is False, only the route that exposes them is withheld.

• metrics_token (str | None) – Optional bearer token guarding GET /metrics. None (the default) leaves the endpoint unguarded — reasonable for a scrape target reachable only from inside a private network. When set, a request without a matching Authorization: Bearer header gets 401 and no metric text.

• trust_proxy_headers (bool) – Whether the leftmost entry of an incoming X-Forwarded-For header is trusted as the real client IP, instead of request.client.host. Off by default: this stack usually runs behind a reverse proxy, but trusting a client-settable header unconditionally is a trivial rate-limit and audit-log bypass. Turn this on only when a proxy in front of this app is known to set (and not merely pass through) the header.

• rate_limiting_enabled (bool) – Whether the three limits below are enforced at all. Off by default, so an existing caller or the test suite is unaffected until it opts in.

• rate_limit_generation (str) – Limit on POST /v1/chats, POST /v1/chats/{id}/messages, and POST /v1/chats/{id}/messages/{mid}, as a "10/minute"-style string (<count>/<second|minute|hour|day>), per client IP.

• rate_limit_read (str) – Limit on GET /v1/chats and GET /v1/chats/{id}, same format, per client IP.

• rate_limit_new_session (str) – Limit on the branch of a request that mints a new session (POST /v1/chats and POST /v1/sessions when neither presents a live session), same format, per client IP — not a limit on those endpoints as a whole, which rate_limit_generation already covers for POST /v1/chats.

• geoip_database_path (Path | None) – Path to a MaxMind GeoLite2-Country .mmdb file, for resolving a session’s country. None (the default) disables country resolution entirely; nothing is downloaded or vendored by this package — see the serving guide for where to get one. Requires the geoip extra (geoip2); without it, country resolution is silently unavailable even when a path is set.

allowed_origins: list[str]

cookie_secure: bool = True

cookie_domain: str | None = None

session_ttl_days: int = 30

expiry_bump_min_days: int = 1

max_request_bytes: int = 32768

max_content_chars: int = 4000

max_messages_per_chat: int = 200

max_chats_per_session: int = 100

generation_timeout_s: float = 120.0

keepalive_interval_s: float = 15.0

log_level: str = 'INFO'

bearer_tokens: bool = False

metrics_enabled: bool = True

metrics_token: str | None = None

trust_proxy_headers: bool = False

rate_limiting_enabled: bool = False

rate_limit_generation: str = '10/minute'

rate_limit_read: str = '60/minute'

rate_limit_new_session: str = '20/hour'

geoip_database_path: Path | None = None

async fastrag.serve.cleanup_expired(engine, *, target='all', batch_size=1000)

Delete expired sessions and chats.

Deleting a session cascades to its chats and messages; chats are swept separately because a chat can go idle while its session stays active.

Deletes in bounded batches — one transaction per batch — for the same reason a single unbounded DELETE would be wrong: cascading through chats and messages in one very large transaction would hold locks and bloat WAL for as long as it ran. Sessions are swept before chats, mirroring the cascade.

This function carries the deletion logic but never runs it itself — nothing in the request path calls it. It is meant to be invoked on a daily cadence by an external scheduler (cron, a Kubernetes CronJob, …); a short script under cron calls it.

Parameters:

• engine (AsyncEngine) – The async engine to run the deletes on. The caller typically passes a PostgresChatStore’s engine. The SQL targets the plain table names (sessions/chats) on the engine’s search path — a store pointed at a non-default schema requires a schema-qualified variant (see this task’s result file).

• target (str) – "sessions", "chats", or "all".

• batch_size (int) – Rows deleted per transaction.

Returns:

A CleanupReport of rows deleted per table.

Return type:

CleanupReport

fastrag.serve.create_app(*, engine, chat_store, config=None)

Build the serving app.

A thin adapter over its two collaborators, laid out in the example the SDK ships: a QueryEngine and a ChatStore. The app itself owns no database engine and no ORM session — chatting goes through the store’s methods, generation through engine.stream, and the advisory lock and /healthz connection check borrow the store’s engine via its engine property. Both collaborators are required: there is no half-configured mode.

Parameters:

• engine (QueryEngine) – The QueryEngine the API exposes — the single place retrieval and generation happen.

• chat_store (ChatStore) – The ChatStore all session/chat/message persistence goes through. Already entered by the caller (async with ... as chat_store).

• config (ServeConfig | None) – ServeConfig. Defaults to a config with an empty allowed_origins list.

Returns:

A FastAPI app serving /healthz and the full /v1 chats API.

Return type:

FastAPI

Modules

app	The FastAPI app: create_app(), lifespan, middleware, handlers.
cleanup	Expired-row cleanup, callable from an external scheduler.
cli	An interactive terminal client for the serving API, built on Typer.
config	Frozen configuration for fastrag.serve.create_app().
geolocalization	Optional GeoIP country resolution for the serving app.