API — nefi.bench¶
Generated by scripts/gen_api_docs.py — do not edit by hand. Index
nefi.bench¶
Benchmarking: scene/data generators, protocols, ablations, reports.
nefi.bench.ablation¶
Component ablations (NeTMY Tab. 3 / App. E.6, NeFTY Tab. 3 / Tab. 8).
class FnModifier(fn: Callable[[Any, Curriculum], Any], name: str | None = None)¶
Wrap fn(problem, curriculum) -> (problem, curriculum) | None (in-place allowed).
apply(self, problem, curriculum)
class Modifier()¶
Changes one component of a method. Override any of the hooks.
apply(self, problem: Any, curriculum: Curriculum) -> tuple[Any, Curriculum]— Apply the problem and curriculum hooks.config(self, cfg: Any) -> dict[str, Any]— Instance-config overrides (applied before the problem is built).curriculum(self, curriculum: Curriculum) -> Curriculum— Return the (possibly new) curriculum; in-place changes may returnNone.problem(self, problem: Any) -> Any— Return the (possibly new) problem; in-place changes may returnNone.
def as_modifier(obj: Any) -> Modifier¶
Coerce a :class:Modifier, a spec string, a config-override dict or a callable.
def cumulative_ablation(instance: Any, variants: Sequence[Any], n_samples: int = 4, seeds: Sequence[int] = (0, 1, 2), *, cumulative: bool = True, include_full: bool = True, full_na...¶
Ablation table with one method per (cumulative) step (NeTMY Tab. 3, NeFTY Tab. 3).
def drop_loss(name: str) -> Modifier¶
Set loss name to weight 0 (also in per-stage overrides).
def grid_field(lr_scale: float = 1.0) -> Modifier¶
Replace the field by a free :class:GridField with the same heads (Grid Opt. / Tikhonov).
def no_annealing() -> Modifier¶
Disable frequency annealing: every stage starts with all Fourier bands on.
def no_gate() -> Modifier¶
Replace every :class:GatedSoftplus head by a plain :class:Softplus (NeTMY "−gate").
def no_positional_encoding() -> Modifier¶
Replace the Fourier-feature encoding of a :class:NeuralField by raw coordinates.
def parse_modifier(spec: str) -> Modifier¶
Parse a modifier string (used by the CLI).
def parse_value(text: str) -> Any¶
Parse a CLI value: int, float (1e-3 too), bool, none, YAML list/dict, or string.
def rebuild_field(field: Any, heads: Heads | None = None, encoding: Any = None) -> Any¶
A fresh field of the same kind/settings with new heads and/or encoding.
def set_config(**overrides: Any) -> Modifier¶
Instance-config overrides for the inversion (must not change data generation).
def set_curriculum(**attrs: Any) -> Modifier¶
Set curriculum attributes (e.g. restarts=3, early_stop_patience=200).
def set_optim(**attrs: Any) -> Modifier¶
Set optimizer settings (e.g. optimizer="adam", grad_clip=None, ema=0.99).
def set_stage(**attrs: Any) -> Modifier¶
Set attributes on every curriculum stage (e.g. anneal_fraction=0.25).
def set_weight(name: str, weight: float) -> Modifier¶
Set loss name to weight everywhere (base weight and per-stage overrides).
def single_stage() -> Modifier¶
Collapse a multiscale curriculum to one stage at the final resolution.
def variant_method(instance: Any, label: str, modifiers: Sequence[Any], base: Any = None)¶
A :class:~nefi.bench.protocol.Method = base method with modifiers applied.
nefi.bench.base¶
Benchmark primitives: ground-truth scene generators and inverse-crime-safe data generators.
class DataGenerator(operator: Operator, noise_std: float = 0.0, relative: bool = True, dtype: torch.dtype = torch.float64, device: str | torch.device = 'cpu', supersample: int ...¶
Turns ground-truth fields into a :class:Measurement with an independent forward model.
add_noise(self, clean: torch.Tensor, rng: np.random.Generator, noise_std: float | None = None) -> tuple[torch.Tensor, float]clean(self, gt_fields: dict[str, torch.Tensor]) -> torch.Tensorgenerate(self, gt_fields: dict[str, torch.Tensor], rng: np.random.Generator, noise_std: float | None = None, target_shape: Sequence[int] | None = None) -> Measurement— Measurement forgt_fields(sampled at the generator's resolution).
class SceneGenerator(domain: Domain)¶
Samples ground-truth fields for a domain, optionally by difficulty class.
check_class(self, cls: str | None) -> strsample(self, rng: np.random.Generator, cls: str | None = None, shape: Sequence[int] | None = None) -> dict[str, torch.Tensor]sample_many(self, n: int, seed: int = 0, cls: str | None = None, shape: Sequence[int] | None = None) -> list[dict[str, torch.Tensor]]
nefi.bench.protocol¶
Benchmark protocol: paired, seeded, inverse-crime-safe comparisons with confidence intervals.
class BenchmarkResult(instance: str, rows: list[dict[str, Any]], methods: list[str], metric_names: list[str], config: dict[str, Any] = <factory>, regime: dict[str, str] = <factor...¶
Per-run rows of a benchmark plus aggregation and report writers.
best(self, metric: str) -> str | None— Method with the best meanmetric(Noneif the direction is unknown).direction(self, metric: str) -> bool | None—Trueif higher is better formetric(Noneif unknown).efficiency_table(self, precision: int = 3) -> str— Training-level efficiency (NeFTY Tab. 9): params, steps, ms/step, wall-clock, memory.header(self, ci: float = 0.95) -> strload(path: str | Path) -> BenchmarkResult— Load from abenchmark.jsonfile or a directory written by :meth:save.merge(paths: str | Path | Sequence[str | Path], strict: bool = True) -> BenchmarkResult— Concatenate benchmark shards into one result (the table of the unsharded run).ok_rows(self) -> list[dict[str, Any]]save(self, directory: str | Path, ci: float = 0.95) -> dict[str, Path]— Writesummary.md,summary.csv,rows.csvandbenchmark.json.save_shard(self, directory: str | Path) -> Path— Write this shard as<directory>/shard-III-of-NNN.json(merge with :meth:merge).shard(property) —(i, n)if this result is shardiofn(seerun_benchmark(shard=...)).shard_filename(self) -> strshard_files(directory: str | Path) -> list[Path]— Shard files (shard-*-of-*.json) indirectory, sorted.strata(self, prefix: str = 'scene/') -> list[str]— Row columns available for stratification (scene/*by default).summary_csv(self, ci: float = 0.95, by_class: bool = False, by: str | None = None) -> str— Aggregated table as CSV (bygroups by any row column, e.g."scene/n_defects").table(self, ci: float = 0.95, by_class: bool = False, metrics: Sequence[str] | None = None, by: str | None = None) -> list[dict[str, Any]]— Aggregated rows: mean and t-distribution CI half-width over samples × seeds.to_csv(self, path: str | Path | None = None) -> str— Raw per-run rows as CSV (written topathwhen given).to_dict(self) -> dict[str, Any]to_json(self, path: str | Path | None = None) -> str— Everything (settings + raw rows) as JSON (written topathwhen given).to_markdown(self, ci: float = 0.95, by_class: bool = False, precision: int = 4, bold_best: bool = True, runtime: bool = True, header: bool = True, by: str | None = None...— Markdown tablemethod | metric ↑/↓ … | time | peak mem | stepswith best in bold.
class InverseCrimeError(...)¶
Raised when data generation and inversion use the same forward model (NeTMY §3.1).
class Method(name: str, build: Callable[[Any, Any], Any] | str | None = None, solver_kwargs: dict[str, Any] = <factory>, description: str = '')¶
A named way of solving an instance's measurement.
prepare(self, instance: Any, measurement: Any) -> PreparedRun— Instantiate the method onmeasurement.
class PreparedRun(problem: Any, curriculum: Curriculum | None = None, solver_kwargs: dict[str, Any] = <factory>, runner: Runner | None = None)¶
A method instantiated on one measurement: problem + curriculum + solver settings.
resolved_curriculum(self) -> Curriculumrun(self, curriculum: Curriculum | None = None, *, device: str | torch.device = 'auto', seed: int | None = 0, callbacks: Sequence[Any] = ()) -> Result— Solve (customrunnerif set, else :func:solve_problem).
def baseline_method(key: str, name: str | None = None) -> Method¶
A baseline from instance.baselines().
def benchmark_units(class_list: Sequence[Any], n_samples: int, seeds: Sequence[int], shard: Any = None) -> list[tuple[Any, int, int]]¶
The (class, sample, seed) run units of a benchmark, restricted to shard i of n.
def check_inverse_crime(data_generator: Any, operator: Any, allow: bool = False, label: str = '') -> str¶
Assert that the data were simulated with a different model than the inversion operator.
def default_method(name: str = 'neural') -> Method¶
The instance's own (neural-field) method.
def evaluate_metrics(instance: Any, result: Result, gt: Mapping[str, torch.Tensor], metrics: Any = None) -> dict[str, float]¶
Instance metrics (instance.evaluate) or an explicit metric set on the primary field.
def operator_fidelity_tag(operator: Any) -> str¶
Fidelity tag of an inversion operator.
def parse_shard(shard: Any) -> tuple[int, int]¶
None → (0, 1); (i, n) or "i/n" → (i, n) with 0 ≤ i < n.
def resolve_classes(instance: Any, classes: Any = None) -> list[str | None]¶
Scene classes to benchmark: None → the instance default, "all" → every class.
def resolve_methods(instance: Any, methods: Any = None) -> list[Method]¶
Methods from names / :class:Method objects / a comma-separated string.
def resolve_metric_fns(metrics: Any) -> dict[str, Callable]¶
{name: fn} from a mapping or a list of registered metric names.
def run_benchmark(instance: Any, methods: Any = None, n_samples: int = 4, seeds: Sequence[int] = (0, 1, 2), classes: Any = None, metrics: Any = None, device: str | torch.devi...¶
Run every method on the same measurements with several seeds and aggregate the metrics.
def scale_budget(curriculum: Curriculum, scale: float = 1.0, max_total_steps: int | None = None, min_stage_steps: int = 1) -> Curriculum¶
Copy of a curriculum with every stage's steps scaled (and the total optionally capped).
def solve_problem(problem: Any, curriculum: Curriculum | None = None, **solver_kw: Any) -> Result¶
Solve a problem with the solver it asks for.
nefi.bench.report¶
Report writers (markdown / CSV / JSON), confidence intervals and operator runtime tables.
class RuntimeTable(rows: list[dict[str, Any]] = <factory>)¶
Per-operator forward/backward timings and peak memory (NeFTY Tab. 4 style).
to_csv(self) -> strto_json(self) -> strto_markdown(self, precision: int = 3) -> str
def format_mean_ci(mean: float, half_width: float, precision: int = 4) -> str¶
mean ± hw (just mean when the half-width is undefined).
def format_value(x: Any, precision: int = 4) -> str¶
Compact number formatting (— for missing, scientific for very small/large).
def markdown_table(headers: Sequence[str], rows: Sequence[Sequence[Any]], align: Sequence[str] | None = None) -> str¶
GitHub-flavoured markdown table (align entries: "l", "r" or "c").
def mean_ci(values: Iterable[float], ci: float = 0.95) -> tuple[float, float, int]¶
Mean and confidence-interval half-width with the Student-t distribution.
def metric_direction(name: str) -> bool | None¶
True if higher is better, False if lower is better, None if unknown.
def rows_to_csv(rows: Sequence[Mapping[str, Any]], columns: Sequence[str] | None = None) -> str¶
CSV text for a list of dict rows (union of keys, in first-seen order).
def runtime_table(problems: Any, shape: Sequence[int] | None = None, *, n_repeat: int = 5, warmup: int = 1, device: str | torch.device | None = None, grad_fields: Sequence[st...¶
Time the forward and backward pass of one or several operators (NeFTY Tab. 4).
def to_jsonable(obj: Any) -> Any¶
Recursively convert tensors / tuples / paths / non-finite floats to JSON-safe values.
def write_csv(path: str | Path, rows: Sequence[Mapping[str, Any]]) -> Path¶
Write dict rows as CSV.
def write_json(path: str | Path, obj: Any) -> Path¶
Write a JSON file (tensors, tuples and non-finite floats converted).
def write_text(path: str | Path, text: str) -> Path¶
Write text (creating parent directories) and return the path.
nefi.bench.sweep¶
One-axis hyperparameter sweeps (NeTMY Tab. 10: learning rate, hidden width, PE octaves).
def axis_modifier(instance: Any, axis: str, value: Any) -> Modifier¶
The :class:~nefi.bench.ablation.Modifier that sets axis to value.
def sweep(instance: Any, axis: str, values: Sequence[Any], n_samples: int = 4, seeds: Sequence[int] = (0, 1, 2), *, base: Any = None, **bench_kw: Any)¶
One-axis sweep: one benchmark method per value of axis (NeTMY Tab. 10).