fastrag.ingestion.steps.fetching.renderers#

Pluggable raw-content renderers, owned by the fetching stage.

Functions

summarize_html(content, limit)

Return visible text (truncated to limit) plus control counts.

Classes

`ChatRenderer`(*, model, api_key, base_url[, ...])	Reveal dynamic content using an OpenAI-compatible chat model.
`DecisionsRenderer`(*[, api_format])	Reveal dynamic content using typed Decisions/System One model choices.
`HttpRenderer`()	Keep the HTTP response unchanged; perform no browser work.
`PlaywrightRenderer`(*[, timeout, ...])	Execute JavaScript and hydrate HTML without model-driven interaction.
`Renderer`()	Turn an HTTP response into raw content before hashing and caching.

class fastrag.ingestion.steps.fetching.renderers.Renderer[source]#

Bases: ABC

Turn an HTTP response into raw content before hashing and caching.

abstractmethod config()[source]#

Return settings that affect rendered bytes, excluding credentials.

Return type:: Mapping[str, Any]

abstractmethod async render(page)[source]#

Return raw rendered content with response provenance preserved.

Parameters:: page (FetchedPage)
Return type:: FetchedPage

async aclose()[source]#

Release any long-lived resources. Idempotent; a no-op by default.

Return type:: None

class fastrag.ingestion.steps.fetching.renderers.HttpRenderer[source]#

Bases: Renderer

Keep the HTTP response unchanged; perform no browser work.

config()[source]#

Return the static rendering identity.

Return type:: Mapping[str, Any]

async render(page)[source]#

Return the response unchanged.

Parameters:: page (FetchedPage)
Return type:: FetchedPage

async aclose()#

Release any long-lived resources. Idempotent; a no-op by default.

Return type:: None

class fastrag.ingestion.steps.fetching.renderers.PlaywrightRenderer(*, timeout=30, wait_selector=None, browser=None, wait_until='networkidle')[source]#

Bases: Renderer

Execute JavaScript and hydrate HTML without model-driven interaction.

Parameters:

timeout (float)
wait_selector (str | None)
browser (SharedBrowser | None)
wait_until (str)

config()[source]#

Return browser rendering settings.

Return type:: Mapping[str, Any]

async render(page)[source]#

Hydrate HTML; leave non-HTML responses untouched.

Parameters:: page (FetchedPage)
Return type:: FetchedPage

async aclose()#

Release any long-lived resources. Idempotent; a no-op by default.

Return type:: None

fastrag.ingestion.steps.fetching.renderers.summarize_html(content, limit)[source]#

Return visible text (truncated to limit) plus control counts.

Parameters:

content (bytes)
limit (int)

Return type:

dict[str, Any]

class fastrag.ingestion.steps.fetching.renderers.ChatRenderer(*, model, api_key, base_url, goal='Extract all useful page content for a searchable knowledge base.', max_steps=32, max_snapshot_chars=24000, timeout=600, request_timeout=120, render_timeout=30, settle_seconds=0.5, wait_selector=None, extra_body=None, fallback=None, http_renderer=None, dynamic_renderer=None)[source]#

Bases: Renderer

Reveal dynamic content using an OpenAI-compatible chat model.

Parameters:

model (str)
api_key (str)
base_url (str)
goal (str)
max_steps (int)
max_snapshot_chars (int)
timeout (float)
request_timeout (float)
render_timeout (float)
settle_seconds (float)
wait_selector (str | None)
extra_body (dict[str, Any])
fallback (_AIRenderer | None)
http_renderer (Renderer | None)
dynamic_renderer (Renderer | None)

async aclose()#

Close the shared model client, browser and any fallback.

Return type:: None

api_format: Literal['decisions', 'systemone'] = 'decisions'#

config()#

Return extraction settings without the API key.

Return type:: Mapping[str, Any]

protocol: str = 'openai'#

async render(page)#

Reveal same-page content, preserving HTML across replacement states.

Parameters:: page (FetchedPage)
Return type:: FetchedPage

base_url: str#

extra_body: dict[str, Any]#

model: str#

api_key: str#

class fastrag.ingestion.steps.fetching.renderers.DecisionsRenderer(*, api_format='decisions', **kwargs)[source]#

Bases: Renderer

Reveal dynamic content using typed Decisions/System One model choices.

api_format picks the wire format: "decisions" (OpenAI Decisions, /decisions) or "systemone" (TypeSafe System One, /systemone).

Parameters:

api_format (Literal['decisions', 'systemone'])
kwargs (Any)

protocol: str = 'decisions'#

api_format: Literal['decisions', 'systemone'] = 'decisions'#

async aclose()#

Close the shared model client, browser and any fallback.

Return type:: None

config()#

Return extraction settings without the API key.

Return type:: Mapping[str, Any]

async render(page)#

Reveal same-page content, preserving HTML across replacement states.

Parameters:: page (FetchedPage)
Return type:: FetchedPage

base_url: str#

extra_body: dict[str, Any]#

model: str#

api_key: str#