fastrag.ingestion.logger.source_tree#

Final URL tree summaries grouped by domain and path.

Classes

SourceTreeLogger(pipeline, *[, max_depth, ...])

Print a final tree of unique source URLs and their subtree totals.
class fastrag.ingestion.logger.source_tree.SourceTreeLogger(pipeline, *, max_depth=None, stream=None, color=None, enrich_youtube=False)[source]#

Bases: Logger

Print a final tree of unique source URLs and their subtree totals.

Domain nodes have depth 1; each path segment adds one level. Schemes, queries, fragments and trailing slashes distinguish URLs for counting, but share the same domain/path branch. A YouTube /watch URL adds its ?v= video ID as one more level. Every node includes URLs at that path and below it, even when descendants are hidden by max_depth. Path labels below domain roots start with a slash. Each line shows nonzero colored status numbers; a final summary names and totals them.

Status rules, enrich_youtube titles (shown in parentheses after the video’s label) and failure handling are those of the shared source summary base. No optional logging dependency is required.

Parameters:

  • pipeline (Pipeline)

  • max_depth (int | None)

  • stream (TextIO | None)

  • color (bool | None)

  • enrich_youtube (bool)

async run()#

Consume events, then print the complete or interrupted summary.

Return type:

None

pipeline: Pipeline#