fastrag.ingestion.policy
Execution policy controlling operational constraints for pipeline steps.
Classes
-
class fastrag.ingestion.policy.Policy(retries=UNSET, backoff=UNSET, max_backoff=UNSET, timeout=UNSET, on_error=UNSET, concurrency=UNSET, batch_size=UNSET, cache_batch_size=UNSET, recreate_cache=UNSET)[source]
Bases: object
Execution policy controlling operational constraints for pipeline steps.
- Parameters:
-
-
retries
Maximum number of execution retries on failure.
- Type:
int
-
backoff
Initial exponential backoff delay in seconds.
- Type:
float
-
max_backoff
Ceiling delay for exponential backoff in seconds.
- Type:
float
-
timeout
Maximum execution duration per item or batch in seconds.
- Type:
float
-
on_error
Failure handling strategy ("skip", "raise",
"retry").
- Type:
Literal[‘skip’, ‘raise’, ‘retry’]
-
concurrency
Max concurrent worker tasks.
- Type:
int
-
batch_size
Processing batch size for batched transforms.
- Type:
int
-
cache_batch_size
Cache lookup window, independent of transform concurrency.
- Type:
int
-
recreate_cache
Skip cached results and overwrite them with freshly
recomputed values.
- Type:
bool
A step’s policy is layered over its parent’s (the pipeline’s, or an
enclosing router/aggregator’s) with merge(): only the fields the
caller explicitly passed override the parent; the rest are inherited.
-
merge(base)[source]
Return this policy layered over base.
Fields this policy set explicitly win; every other field is taken
from base. The result’s explicit set is the union of both, so a
chain of merges keeps resolving correctly.
- Parameters:
base (Policy | None) – The inherited (parent) policy, or None for no parent.
- Return type:
Policy
-
delay_for(attempt)[source]
Return the capped exponential backoff delay before attempt.
- Parameters:
attempt (int) – The retry attempt number, 1 for the first retry.
- Returns:
backoff * 2 ** (attempt - 1), capped at max_backoff.
- Return type:
float