fastrag.ingestion.filters#

fastrag.ingestion.filters — composable predicates over sources and documents.

Re-exports Filter, the composable predicate base, and its three concrete filters, MediaTypeFilter, URLFilter, and ContentFormatFilter.

class fastrag.ingestion.filters.ContentFormatFilter(include_formats=None, exclude_formats=None)[source]#

Bases: Filter[Document]

Filters documents by metadata["content_format"].

Unlike MediaTypeFilter, a document carrying no recorded format is not treated as unmatchable — it is treated as TEXT, since every document that existed before content_format was introduced is plain text by definition.

Parameters:

include_formats (Sequence[ContentFormat | str] | None)
exclude_formats (Sequence[ContentFormat | str] | None)

property header_safe: bool#: This predicate uses identity/metadata only.

matches(item)[source]#

Return True when item’s content format passes the filter.

Reads item.metadata["content_format"], defaulting to TEXT when the key is absent.

Parameters:: item (Document)
Return type:: bool

class fastrag.ingestion.filters.Filter[source]#

Bases: ABC, Generic[T_contra]

Base abstract class for composable entity filtering.

property header_safe: bool#

Whether matching uses identity/metadata only, never content/vectors.

Custom filters default to full records. Override only for predicates that remain correct on the index-only cache projection.

abstractmethod matches(item)[source]#

Evaluate whether an entity passes the filter criteria.

Parameters:: item (T_contra) – Item to evaluate (e.g. a Source or Document).
Returns:: True if the item satisfies the condition, False otherwise.
Return type:: bool

class fastrag.ingestion.filters.MediaTypeFilter(include_types=None, exclude_types=None)[source]#

Bases: Filter[Source | Document]

Filters sources or documents by MIME media type.

Parameters:

include_types (Sequence[str] | None)
exclude_types (Sequence[str] | None)

property header_safe: bool#: This predicate uses identity/metadata only.

matches(item)[source]#

Return True when item’s content type passes the filter.

Reads Source.content_type directly, and Document from metadata["content_type"], comparing the bare media type (no charset or other parameters). A source sent as application/octet-stream is matched by the type its URI’s extension implies, when it implies one. A Document with no recorded content type matches only when include_types is empty.

Parameters:: item (Source | Document)
Return type:: bool

class fastrag.ingestion.filters.URLFilter(include_patterns=None, exclude_patterns=None)[source]#

Bases: Filter[Source | Document]

Filters sources or documents by URI regex patterns.

Parameters:

include_patterns (Sequence[str] | None)
exclude_patterns (Sequence[str] | None)

property header_safe: bool#: This predicate uses identity/metadata only.

matches(item)[source]#

Return True when item’s URI passes the filter.

Reads Source.uri directly, and a Document’s metadata["source_uri"], falling back to source_id when absent.

Parameters:: item (Source | Document)
Return type:: bool

Modules

`base`	`Filter`: the composable predicate base every concrete filter builds on.
`contentformat`	`ContentFormatFilter`: filters documents by their content format.
`mediatype`	`MediaTypeFilter`: filters sources or documents by MIME media type.
`url`	`URLFilter`: filters sources or documents by URI regex patterns.