# Decisions Recorded so a future change cannot silently reverse one without someone noticing. Append-only: when a decision changes, add a new entry that supersedes the old one and says so, rather than editing the original away. ## Non-goals Stated explicitly, because they bound every decision below: - **Not a LangChain replacement.** `fastrag` uses LangChain internally where it is the best available implementation and adapts at the edges ({doc}`conventions`). It does not aim to cover LangChain's surface. - **Not a CLI.** See decision 1. - **Not a serving platform.** `fastrag.serve` ships the chats API as a thin adapter over a `QueryEngine` and a `ChatStore`, with no mandatory infrastructure — the app owns no database engine and no ORM session; it borrows the store's. Serving does not reimplement retrieval or prompting. - **No global configuration.** No settings singleton, no implicit `.env` loading, no ambient state. Credentials are arguments — see decision 14. ## The SDK | # | Decision | Rationale | | --- | ---------------------------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | 1 | **No CLI.** SDK only. | The value is typed, composable pipelines. A CLI re-flattens them into strings and doubles the surface to document and test. | | 2 | **Async at process boundaries; sync for computation.** One name per operation, one `run_sync` door. | Duplicate `run`/`arun` pairs made the contract unreadable. See {doc}`conventions` for the two narrow exceptions this settled into in practice (`Metric.score`, `BenchmarkResult.save`/`.load`). | | 3 | **Own types; LangChain adapted at the edges.** | Delivers the "easy wrapper" pitch without inheriting upstream API churn into the public surface. `fastrag.models.Document.content` is deliberately not named `page_content`, so a future LangChain-object adapter can rename in both directions at zero cost when one is built. | | 4 | **Step cache and record manager are separate.** | They answer different questions: "must I recompute?" and "what is in the collection that should not be?" Conflating them is why purely time-based caching cannot detect change. | | 5 | **Metrics are objects.** | Strings cannot carry a judge model, cannot be type-checked, and cannot be extended without a registry. `metrics=["faithfulness"]` silently returning bag-of-words overlap is the worst failure an evaluation tool can have. | | 6 | **Collection name = human prefix + config fingerprint.** | The prefix keeps it declarative and readable at the call site; the fingerprint makes mixing incompatible vectors impossible. Excluding fetcher configuration from the fingerprint keeps a live site indexing into one collection as it grows. | | 7 | **Deterministic content-derived document ids.** | The precondition for `VectorStore.upsert`. Random ids per run are what made a second run duplicate the corpus. | | 8 | **`incremental` cleanup by default; `full` refuses after fetch errors.** | A failed fetch must never be read as "this content was deleted." | | 9 | **No global configuration.** | A settings singleton hard-coding one vector DB and one endpoint contradicts a swappable-services SDK. | | 10 | **Benchmark results carry a name, not a config.** | The benchmark script is already the readable, version-controlled record. Serializing the pipeline configuration too would duplicate state that will drift. | | 11 | **Metrics aggregate to a distribution, not a mean.**[^guidellm-metrics] | A mean latency is close to useless and a mean faithfulness hides the tail. Percentiles are what a service level is written against. | | 12 | **Serving ships the chats API.** | Originally kept serving WIP — `/healthz` only, no mandatory infrastructure. That requirement survives as `create_app`'s layering (neither argument → `/healthz` only), and serving now ships the full `/v1` chats API as a thin adapter over a `QueryEngine`. | | 13 | **The step cache is a database when one is configured — PostgreSQL today.** | A directory of files could not evict, size itself, or clear a prefix without walking the tree, and could hand a parser a truncated entry after a crash. In the current tree the cache is explicit and per-step: each step takes `cache=` (a `CacheStore`, or `None` to inherit, or `False` to opt out) — see {doc}`/guide/caching`. | | 14 | **No settings module.** `fastrag.settings` is deleted; every credential is an argument, with no exception. | The module existed only for `alembic`'s `env.py`, which now reads `FASTRAG_DATABASE_URL` directly — appropriate for a CLI entry point invoked by the `alembic` command, not code a caller imports. Keeping the module around "for Alembic" while telling users the SDK never reads the environment was a contradiction the documentation could not describe truthfully on one site; deleting it was cheaper than narrowing the rule, and the rule is load-bearing for a swappable-services SDK. | ## Serving Appended to the log above with its own numbering, `serving`-tagged, ported from the design this package's serving layer was built against. | # | Decision | Rationale | | --- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | S1 | **Streaming is stateless.** No streaming state is persisted; a message is written only once its generation finishes. | Any replica can serve any request — no load balancer session affinity is required, and the database is the only shared state. A disconnect mid-generation is stored with `finish_reason: "interrupted"` rather than resumed. | | S2 | **The server assigns every id.** Clients never choose a session, chat, or message id. | A client-supplied id is untrusted input that would let a caller overwrite or probe rows it does not own. This is why `message_start` carries ids at all — it is the only way a client learns the id of the message it just sent. | | S3 | **IDs are ULIDs.** | Lexicographically sortable by creation time, so `ORDER BY id` is chronological and cursor pagination needs no secondary timestamp column. | | S4 | **Chats are linear, not trees.** Re-running a message truncates everything after it rather than branching. | The UI shows a single thread; a tree would need a "current leaf" pointer and merge rules for no product gain. | | S5 | **Generations are serialized per chat**, with a session-level Postgres advisory lock. | See {doc}`serving-internals` for the full lock design and why it is session-level rather than transaction-scoped. | | S6 | **Sessions and chats expire after 30 days of inactivity**, swept by an externally-scheduled job rather than on request. | Cleanup cadence stays independent of API traffic; see {doc}`serving-internals`'s Retention section for why not `pg_cron`. | | S7 | **Expiry bumps are write-throttled.** An expiry is extended on activity only when the new value is materially later than the stored one. | Keeps active conversations alive without a write on every single request. | | S8 | **The session id is the bearer secret.** There is no username, password, or OAuth layer. | The cookie _is_ the credential; a bearer token is the same credential carried in a header instead — see {doc}`serving-internals`'s "Bearer tokens, in full." | | S9 | **The chat-store schema is self-managed, the same way the ingestion cache's is.** `PostgresChatStore.__aenter__` creates its schema, tables, and `ENUM` types if absent (`create_all(checkfirst=True)`); nothing in `fastrag` touches Alembic or a shared declarative `Base`, though `alembic` is still a declared dependency from the design this was ported from. | Matches decision 13's reasoning for the ingestion cache: a self-managing store needs no separate migration step, and a swappable `ChatStore` cannot assume the caller runs one. The ported design's original "migrations only" rule was not carried into this implementation and never got its own decision entry recording the change — this row corrects that gap rather than adding a new one. | | S10 | **Every generation carries the chat's full history**, unwindowed and unsummarized. | The alternative — condensing history into a standalone query before retrieval — needs an extra LLM round-trip before the first token; deferred (`BACKLOG.md`). Accepted trade: a follow-up pronoun resolves in generation but not in retrieval. | | S11 | **History replay includes a row if and only if its `content` is not `NULL`.** | The `content_required_unless_error` constraint permits a `NULL` body on a failed assistant row; replaying unconditionally would construct an empty message and fail the third turn of any chat that ever errored. | ## References - Custom datasets — — column aliasing and multi-format loading, adapted into `QADataset` and `ColumnMapping` ({doc}`/guide/benchmarking`). [^guidellm-metrics]: — `DistributionSummary`, percentile reporting, and status breakdown, adapted into `DistributionSummary` and `StatusCounts`.