erlab.interactive.imagetool.provenance

Describe, compose, and replay ImageTool data provenance.

Changed in version 3.25.0: This module now focuses on provenance models, source constructors, composition, grouping, parsing, and replay. Concrete operation models live in the evolving erlab.interactive.imagetool._provenance._operations catalog.

Functions

compose_display_provenance(parent, ...[, ...])	Compose streamlined display provenance from a live source spec.
compose_full_provenance(parent, local, *[, ...])	Compose canonical full provenance from parent and local provenance.
file_load(*, start_label, seed_code, ...[, ...])	Build structured file-backed provenance for runtime reload.
full_data(*operations)	Build a spec that starts from the parent's full current data.
operation_group_range(operations, ...[, kind])	Return the complete contiguous group range containing an operation.
parse_tool_provenance_operation(value)	Parse one serialized operation payload.
parse_tool_provenance_spec(value)	Parse a serialized provenance payload into a validated spec instance.
public_data(*operations)	Build a spec that starts from the parent's restored public data.
replay_file_provenance(spec, *[, cache])	Replay structured file provenance without executing generated Python.
replay_script_provenance(spec, inputs, *[, ...])	Execute script provenance from already resolved input arrays.
script(*operations, start_label[, ...])	Build script provenance from code, structured steps, and named inputs.
selection(*operations)	Build a spec that starts from the parent's public selection model.
stamp_operation_group(operations, *, kind[, ...])	Return operations stamped as one complete contiguous edit group.
strip_operation_groups(operations)	Return operations with all edit-group metadata removed.

Classes

DerivationEntry(label, code[, copyable])	One user-visible step in a provenance derivation listing.
FileDataSelection(**data)	Serializable selection of one displayable array from a loaded file object.
FileLoadSource(**data)	Serializable file origin used by saved file-backed provenance.
FileReplayCall(**data)	Serializable call information used to reload file-backed provenance.
ReplayStage(**data)	Legacy schema-v2 transformation stage.
ReplayStep(**data)	One operation in the canonical ordered durable replay pipeline.
ScriptInput(**data)	Named input captured by script or multi-tool provenance.
ToolProvenanceOperation(**data)	Base class for typed operations stored in ToolProvenanceSpec.
ToolProvenanceSpec(**data)	Saved provenance recipe for ImageTool data.

class erlab.interactive.imagetool.provenance.DerivationEntry(label, code, copyable=False)

Bases: object

One user-visible step in a provenance derivation listing.

copyable: bool = False

label: str

code: str | None

class erlab.interactive.imagetool.provenance.FileDataSelection(**data)

Bases: BaseModel

Serializable selection of one displayable array from a loaded file object.

New provenance stores stable Dataset variable names and DataTree node/variable pairs instead of the positional parsed-array index used by older workspaces.

model_config: ClassVar[ConfigDict] = {'arbitrary_types_allowed': True, 'extra': 'forbid', 'frozen': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

kind: typing.Literal['dataarray', 'dataset_variable', 'datatree_variable', 'sequence_index', 'parsed_index']

value: typing.Any

class erlab.interactive.imagetool.provenance.FileLoadSource(**data)

Bases: BaseModel

Serializable file origin used by saved file-backed provenance.

model_config: ClassVar[ConfigDict] = {'arbitrary_types_allowed': True, 'extra': 'forbid', 'frozen': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

path: str

loader_label: str

loader_text: str

kwargs_text: str

replay_call: FileReplayCall | None

load_code: str | None

class erlab.interactive.imagetool.provenance.FileReplayCall(**data)

Bases: BaseModel

Serializable call information used to reload file-backed provenance.

model_config: ClassVar[ConfigDict] = {'arbitrary_types_allowed': True, 'extra': 'forbid', 'frozen': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

kind: typing.Literal['erlab_loader', 'callable']

target: str

kwargs: dict[str, typing.Any]

selection: FileDataSelection

cast_float64: bool

class erlab.interactive.imagetool.provenance.ReplayStage(**data)

Bases: BaseModel

Legacy schema-v2 transformation stage.

New provenance must use ReplayStep. This model remains public only so saved schema-v2 workspaces and callers constructing their old payloads continue to deserialize. The compatibility conversion is centralized in ToolProvenanceSpec._migrate_legacy_replay_shape(); remove this class together with that converter when schema-v2 workspace support is retired.

classmethod from_source_spec(source)

model_config: ClassVar[ConfigDict] = {'arbitrary_types_allowed': True, 'extra': 'forbid', 'frozen': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

source_kind: _SourceKind

operations: tuple[pydantic.SerializeAsAny[ToolProvenanceOperation], ...]

class erlab.interactive.imagetool.provenance.ReplayStep(**data)

Bases: BaseModel

One operation in the canonical ordered durable replay pipeline.

classmethod from_source_spec(source)

Lower one live source into independently movable durable steps.

model_config: ClassVar[ConfigDict] = {'arbitrary_types_allowed': True, 'extra': 'forbid', 'frozen': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

operation: pydantic.SerializeAsAny[ToolProvenanceOperation]

input_policy: _ReplayInputPolicy | None

context_names: tuple[str, ...]

legacy_context: _LegacyReplayContext | None

class erlab.interactive.imagetool.provenance.ScriptInput(**data)

Bases: BaseModel

Named input captured by script or multi-tool provenance.

name is the immutable replay variable, label is the historical display label, node_uid and node_snapshot_token identify the live manager input that was used, data_role selects its durable source or displayed view, and provenance_spec stores the historical replay source used when that live input is unavailable. When omitted, label defaults to name.

model_config: ClassVar[ConfigDict] = {'arbitrary_types_allowed': True, 'extra': 'forbid', 'frozen': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

parsed_provenance_spec()

name: str

label: str

node_uid: str | None

node_snapshot_token: str | None

data_role: ScriptInputDataRole

provenance_spec: dict[str, typing.Any] | None

class erlab.interactive.imagetool.provenance.ToolProvenanceOperation(**data)

Bases: BaseModel

Base class for typed operations stored in ToolProvenanceSpec.

New operations should keep runtime fields in their decoded Python form, prefer the annotated provenance field aliases in this module for lossless JSON serialization, implement apply() to replay the transformation, implement expression_code() to emit the public Python expression for the same transformation, and implement derivation_label() to describe the step in manager UI.

Operation instances store the exact arguments used by live refresh, replay graph execution, copied code, and derivation display. If a public console call maps exactly to an operation, expose it with console_patterns or from_console_call(); ambiguous calls should return None so script provenance records the original code instead.

apply(data)

Apply this operation to the current derived array.

Subclasses that participate in live refresh or executable provenance replay should reimplement this method. The implementation must be deterministic for the operation’s stored model fields and must not mutate data in place.

Parameters:

data (DataArray) – Array produced by the preceding replay step.

Returns:

xarray.DataArray – Array after this operation has been applied.

Return type:

DataArray

Notes

Operations that only emit generated code should set live_applicable = False and raise from this method.

batch_available: typing.ClassVar[bool] = False

console_applies_to_receiver: typing.ClassVar[bool] = False

console_patterns: typing.ClassVar[tuple[ConsoleOperationPattern, ...]] = ()

derivation_entry()

Return the user-visible derivation entry for this operation.

Structured transform subclasses should normally reimplement derivation_label() and expression_code() instead of overriding this method. Override this method only for operations whose display entry cannot be represented as a label plus parameterized operation code, such as stored free-form script code.

Returns:

DerivationEntry – Label, replay code, and copyability flag shown in derivation UI and used by legacy derivation-code paths.

Return type:

DerivationEntry

Notes

The base implementation keeps the legacy derived replay contract while letting concrete operations emit expression code for any input variable.

Use DerivationEntry(..., code=None) instead when the step should remain visible in the derivation list but code generation should stop and return None.

derivation_label()

Return the derivation-list label for this operation.

Subclasses should reimplement this for every structured operation that uses the base derivation_entry() implementation.

Returns:

str – Human-readable operation label for derivation/history UI. Include the operation’s meaningful parameters when they help explain the produced data.

Return type:

str

expression_code(input_name, *, source_name=None)

Return a Python expression applying this operation to an input name.

Subclasses should reimplement this for every structured operation that can be represented as ordinary user-facing Python code. The emitted expression should be complete, public-API-based, and free of assignment to hardcoded temporary names.

Parameters:

• input_name (str) – Python expression or identifier for the array produced by the previous replay step. Use this exact value as the operation receiver/input instead of assuming a variable name such as derived.

• source_name (str | None, default: None) – Python expression or identifier for the original public input array for the enclosing replay sequence. Operations that need parent/source context, such as coordinate-order restoration, should use this name for that context. Operations that only transform input_name may ignore it.

Returns:

str – Python expression that evaluates to the transformed DataArray.

Return type:

str

classmethod from_console_call(call)

Build an operation from a normalized manager-console call.

Subclasses should reimplement this when a console call maps cleanly to the operation but cannot be expressed with declarative console_patterns alone. Return None for unsupported, ambiguous, or lossy calls so the original console code remains recorded as script provenance.

Parameters:

call (ConsoleCall) – Normalized descriptor for the function, method, or accessor call observed by the ImageTool manager console, including unwrapped arguments, keyword arguments, display code, and receiver data when available.

Returns:

ToolProvenanceOperation or None – Operation instance when the call is exactly representable by this operation class; otherwise None.

Return type:

ToolProvenanceOperation | None

live_applicable: typing.ClassVar[bool] = True

model_config: ClassVar[ConfigDict] = {'arbitrary_types_allowed': True, 'extra': 'forbid', 'frozen': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

preferred_replay_input_name()

Return a semantic name for an input reused by generated code.

Most operations use their input once and should return None. Operations that refer to the same intermediate several times may provide a descriptive name so replay code does not expose an internal graph temporary.

preferred_replay_output_name()

Return a semantic output name when this operation changes value kind.

Most transformations preserve the meaning of their input name and return None. Operations that produce a categorically different result may provide a name which the replay graph uses when it matches the enclosing script’s active output.

replay_code(input_name, *, output_name=None, source_name=None, reserved_names=())

Return replay code for this operation with caller-selected names.

This method usually should not be reimplemented by subclasses. Implement expression_code() instead so replay graph emission, dialog copy code, and derivation entries all share the same operation expression.

Parameters:

• input_name (str) – Python expression or identifier for the array produced by the previous replay step.

• output_name (str | None, default: None) – Variable name to assign the transformed value to. If None, return only the expression from expression_code().

• source_name (str | None, default: None) – Python expression or identifier for the original public input array for the enclosing replay sequence. Passed through to expression_code().

• reserved_names (Collection[str], default: ()) – Names already used by the enclosing replay sequence. Statement-based operations use these to choose collision-free auxiliary targets.

Returns:

str – Either an assignment statement when output_name is provided, or a bare expression when it is None.

Return type:

str

statement_code(input_name, *, output_name, source_name=None)

Return Python statements applying this operation to an input name.

Most structured operations are expression-like and should implement expression_code(). Operations whose public API is mutating, such as accessor setters, should implement this method instead and assign their result to output_name.

statement_mutates_input: typing.ClassVar[bool] = False

group: OperationGroupMarker | None

class erlab.interactive.imagetool.provenance.ToolProvenanceSpec(**data)

Bases: BaseModel

Saved provenance recipe for ImageTool data.

Live child-tool refresh uses single-parent specs from full_data(), public_data(), or selection(). Durable reload and copied code use replay specs. The kind field selects the replay representation, not the user-visible origin: file_load_source is the file-origin capability, and steps is the single ordered operation sequence used by both file and script replay specs. script specs may also include multi-input script_inputs for console or UI actions that combine several ImageTools. Deserialize saved payloads with parse_tool_provenance_spec().

A spec records exact operation arguments for live refresh, runtime replay, copied code, and derivation display. Manager children opened from ImageTool cursor or bin selections refresh by replaying those explicit arguments; legacy selection bindings are converted to source specs once for compatibility.

append_display_operation(operation)

Append a live display operation without making a final name stale.

append_final_rename(name)

append_operations(*operations)

Append operation instances to the spec.

Runtime code should pass operation instances from this module. Saved mappings should be normalized with parse_tool_provenance_spec() before calling this method.

append_replacement_operations(*operations)

Replace a final rename, if present, then append new operation instances.

append_replay_stage(source)

Compatibility alias for appending a live source as flat replay steps.

apply(parent_data)

derivation_code()

derivation_entries()

display_code(*, parent_data=None)

Return streamlined replay code for UI and clipboard actions.

The display path preserves exact live-source behavior while omitting user-facing no-op and normalization steps from copied provenance code.

display_entries(*, parent_data=None)

Return streamlined derivation entries for UI and copy-code output.

display_rows(*, parent_data=None, scope='display')

Return streamlined derivation rows for UI, editing, and copy-code output.

The display path hides internal ImageTool normalization steps while keeping the recorded replay steps available through derivation_entries().

drop_trailing_rename()

property is_live_source: bool

model_config: ClassVar[ConfigDict] = {'arbitrary_types_allowed': True, 'extra': 'forbid', 'frozen': True, 'serialize_by_alias': True}

Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_copy(*, update=None, deep=False)

Copy while translating the schema-v2 operations update alias.

BaseModel.model_copy does not validate aliases. Older manager extensions may still update the serialized operations name directly, so keep this small compatibility translation until schema-v2 extension support is removed. New runtime code updates source_operations or steps explicitly.

property operations: tuple[ToolProvenanceOperation, ...]

Return the ordered operations for this live source or durable recipe.

property script_context_bindings: tuple[_ScriptContextBinding, ...]

Compatibility projection of explicit replay-step context names.

to_replay_spec()

Return the durable replay form for this spec.

Replay specs are the canonical, composable form used for derivation metadata, copied code, workspace save/load, and manager dependency status. Structured file provenance remains structured so runtime reloads can replay typed operations without exec. Live-source operations are kept structured inside script replay specs so editable manager rows remain typed after composition. Live ImageTool refresh uses the original single-parent spec via require_live_source_spec().

schema_version: typing.Literal[3]

kind: typing.Literal['full_data', 'public_data', 'selection', 'script', 'file']

start_label: str | None

seed_code: str | None

active_name: str | None

source_operations: tuple[pydantic.SerializeAsAny[ToolProvenanceOperation], ...]

steps: tuple[ReplayStep, ...]

file_load_source: FileLoadSource | None

script_inputs: tuple[ScriptInput, ...]

erlab.interactive.imagetool.provenance.compose_display_provenance(parent, source_spec, *, parent_data=None)

Compose streamlined display provenance from a live source spec.

erlab.interactive.imagetool.provenance.compose_full_provenance(parent, local, *, script_context_names=())

Compose canonical full provenance from parent and local provenance.

parent represents the replay provenance for the current input data. local represents the additional steps performed by the current node. File-backed parents remain structured when composed with live-source specs so runtime reloads can avoid executing generated Python.

erlab.interactive.imagetool.provenance.file_load(*, start_label, seed_code, file_load_source, active_name='derived', steps=(), replay_stages=())

Build structured file-backed provenance for runtime reload.

erlab.interactive.imagetool.provenance.full_data(*operations)

Build a spec that starts from the parent’s full current data.

erlab.interactive.imagetool.provenance.operation_group_range(operations, operation_index, *, kind=None)

Return the complete contiguous group range containing an operation.

erlab.interactive.imagetool.provenance.parse_tool_provenance_operation(value)

Parse one serialized operation payload.

This is the deserialize boundary for saved JSON. Runtime call sites should build concrete operation instances directly instead of passing raw mappings around.

erlab.interactive.imagetool.provenance.parse_tool_provenance_spec(value)

Parse a serialized provenance payload into a validated spec instance.

This is the deserialize boundary for saved tool and workspace metadata. Runtime authoring code should pass ToolProvenanceSpec instances directly.

erlab.interactive.imagetool.provenance.public_data(*operations)

Build a spec that starts from the parent’s restored public data.

erlab.interactive.imagetool.provenance.replay_file_provenance(spec, *, cache=None)

Replay structured file provenance without executing generated Python.

erlab.interactive.imagetool.provenance.replay_script_provenance(spec, inputs, *, trusted_user_code=False)

Execute script provenance from already resolved input arrays.

erlab.interactive.imagetool.provenance.script(*operations, start_label, seed_code=None, active_name=None, file_load_source=None, steps=(), replay_stages=(), script_inputs=())

Build script provenance from code, structured steps, and named inputs.

erlab.interactive.imagetool.provenance.selection(*operations)

Build a spec that starts from the parent’s public selection model.

erlab.interactive.imagetool.provenance.stamp_operation_group(operations, *, kind, focuses=(), group_id=None)

Return operations stamped as one complete contiguous edit group.

erlab.interactive.imagetool.provenance.strip_operation_groups(operations)

Return operations with all edit-group metadata removed.