Skip to content

Batch

Load many cells, then work with the object you get back.

from cellpy import batch

b = batch.load(name="my_experiment", project="my_project")
b.summaries
b.cells["my_cell_01"]
b.plot()
b.result.report()   # per-cell load outcomes

b is a Batch. You do not need a separate "facade" type — cellpy.batch.facade is only the implementation module.

What b can do

Want Use
Journal table b.pages
Cell labels b.cell_names
One cell b.cells[label]
Combined summaries b.summaries
Summary plot b.plot()
Load / reload b.update() / b.load()
Per-cell load errors b.result.report()
Drop a cell b.drop(label)
Persist the journal b.save()

cellpy.utils.batch is a thin re-export of the same entry points.

Entry points

batch

Load and work with a set of cells.

Typical use::

from cellpy import batch
b = batch.load(name="exp", project="proj")
b.summaries
b.cells["cell_01"]
b.plot()
b.result.report()

b is a Batch. cellpy.utils.batch re-exports the same entry points.

Batch

Batch(journal: Journal, policy: LoadPolicy | None = None, _db: dict | None = None)

A loaded experiment: journal, cells, summaries, and a summary plot.

Typical use::

from cellpy import batch
b = batch.load(name="exp", project="proj")
b.summaries
b.cells["cell_01"]
b.plot()
b.result.report()

Attributes:

  • journal

    The Journal (pages + session).

  • policy

    The LoadPolicy used on the last load.

  • pages (DataFrame) –

    Journal table (filename, mass, group, …).

  • cell_names (list[str]) –

    Labels in journal order.

  • cells (CellStore) –

    Lazy CellStore of CellpyCell objects.

  • summaries (DataFrame) –

    Combined per-cycle summaries.

  • result (BatchResult | None) –

    BatchResult from the last update.

cell_names property

cell_names: list[str]

Cell labels in journal order.

cells property

cells: CellStore

Loaded cells, keyed by label (b.cells['cell_01']).

experiment property

experiment: '_LegacyExperimentAdapter'

Backward-compat view for legacy consumers (helpers/collectors).

cellpy.utils.helpers and cellpy.utils.collectors still reach into b.experiment.{cell_names,data,journal.pages,summary_frames}. This adapter keeps them working against the new Batch until they are migrated (Epic B/C); it is not part of the blessed API.

pages property

pages: DataFrame

Journal table (filename, mass, group, raw_file_names, …).

result property

result: BatchResult | None

Per-cell load outcomes from the last update / load.

Use b.result.report() for a tidy frame (outcome, source, error).

summaries property

summaries: DataFrame

Combined per-cycle summary frame across the batch (cached).

tests property

tests: DataFrame

Per-test metadata across the batch (tidy long-format).

One row per (cell, test_id) with the native TestMeta fields plus cell/group/sub_group keys -- the per-test records a merged (campaign) cell carries. Empty frame if no cell exposes test metadata.

combine_summaries

combine_summaries(**_kwargs) -> pl.DataFrame

Rebuild and return the combined summary frame (clears the cache).

create_journal

create_journal(**kwargs) -> Journal

Populate the journal from the configured database (if any).

Mirrors the legacy init() -> create_journal() flow: init stores the db config, create_journal performs the read.

drop

drop(label: str) -> 'Batch'

Remove label from the journal and the store now.

Parameters:

  • label (str) –

    Cell name as in b.cell_names.

drop_cells_marked_bad

drop_cells_marked_bad() -> 'Batch'

Drop every label listed in journal.session["bad_cells"].

export_journal

export_journal(path: Path | str | None = None) -> Path

Alias of save.

export_project

export_project(destination: Path | str, *, journal_path: Path | str | None = None) -> Path

Write a shareable bundle: .cellpy files plus a rewritten journal.

2.x replacement for 1.x duplicate_cellpy_files(location="standard"). Each loaded cell is saved to <destination>/<label>.cellpy, journal cellpy_file_name is rewritten to those paths (cwd-relative posix when possible), and the journal is saved (Batch.save semantics: cellpy_batch_<name>.json in the cwd unless journal_path is given).

Does not copy raw files or clear raw_file_names. Unloaded cells raise ValueError — call update first.

Parameters:

  • destination (Path | str) –

    directory for the .cellpy files (created if needed).

  • journal_path (Path | str | None, default: None ) –

    optional journal JSON path; default is cwd cellpy_batch_<name>.json.

Returns:

  • Path

    Path of the written journal.

from_cells classmethod

from_cells(cells: Mapping[str, Any] | Sequence[Any], *, groups: Mapping[str, Any] | None = None, sub_groups: Mapping[str, Any] | None = None, group_labels: Mapping[Any, str] | None = None, selected: Mapping[str, bool] | None = None, name: str = 'in_memory', project: str = 'in_memory', policy: LoadPolicy | None = None) -> 'Batch'

Build a batch from already-loaded CellpyCell objects.

The pieces a batch needs -- a journal pages frame (polars, keyed by filename) plus a populated cell store -- are constructed here, so a GUI/notebook holding cells in memory can feed them straight to collect_summaries / collect_cycles or batch.plot() without writing a journal to disk.

Parameters:

  • cells (Mapping[str, Any] | Sequence[Any]) –

    {label: CellpyCell} or a sequence of cells (labels are taken from cell.cell_name, falling back to cell_001 ..., de-duplicated).

  • groups / sub_groups

    optional {label: value} maps (defaults: group 1 for all; sub_group 1..N).

  • group_labels (Mapping[Any, str] | None, default: None ) –

    optional {group: display label} map.

  • selected (Mapping[str, bool] | None, default: None ) –

    optional {label: bool} (defaults True) for the journal selected column.

  • name / project

    journal name/project.

from_db classmethod

from_db(name: str, project: str, policy: LoadPolicy | None = None, **db_kwargs) -> 'Batch'

Build a batch by reading a database (Excel or JSON).

link() -> 'Batch'

No-op in batch v3 (the store loads lazily); kept for the surface.

load

load(**overrides) -> BatchResult

Load cells (alias of update, kept for the legacy surface).

Takes the same executor / on_progress / progress / policy overrides as update, e.g. b.load(executor="threads").

make_summaries

make_summaries() -> pl.DataFrame

Alias of combine_summaries.

mark_as_bad

mark_as_bad(label: str) -> None

Flag label in journal.session['bad_cells'] (does not drop).

Parameters:

  • label (str) –

    Cell name as in b.cell_names.

paginate

paginate() -> tuple[Path, ...]

Create the batch project folders and return their paths.

plot

plot(backend: str | None = None, show: bool = False, **kwargs) -> Any

Plot combined summaries for this batch.

Parameters:

  • backend (str | None, default: None ) –

    Plotting backend (None uses the configured default).

  • show (bool, default: False ) –

    If True, display the figure immediately.

  • **kwargs

    Forwarded to the summary plotter (columns, grouping, …).

Returns:

  • Any

    A figure from the plotting layer (backend-dependent).

recalc

recalc(**overrides) -> BatchResult

Reload every cell and remake step tables and summaries.

Same kwargs as update.

report

report(check: bool = True) -> pl.DataFrame

QC-style per-cell status frame (not the same as result.report()).

Parameters:

  • check (bool, default: True ) –

    kept for the legacy surface; the frame is always built.

save

save(path: Path | str | None = None) -> Path

Write the journal JSON (default cellpy_batch_<name>.json in cwd).

Parameters:

  • path (Path | str | None, default: None ) –

    Destination file. None uses the default name from journal.name.

update

update(on_progress=None, executor: str = 'serial', progress=None, **overrides) -> BatchResult

Load every cell, caching them in the store.

executor is "serial" (default), "threads" or "processes". "threads" mainly speeds up reopening cells from local .cellpy files; a first load of remote raw files does not overlap on the wire, and "processes" usually loses to spawn overhead on Windows. Process workers strip live cells; a raw load is written to .cellpy in the worker so save_cellpy=True can persist without None.save. progress is None (auto: TTY or Jupyter), False (off), True (force), or a callable that receives progress events. on_progress(i, n, result) still wins when set (3-arg callback). Known LoadPolicy fields in overrides update the policy; unknown (legacy) kwargs like testing are forwarded to the loader (cellpy.get) via loader_kwargs.

Journal dataclass

Journal(name: str | None = None, project: str | None = None, pages: DataFrame = pl.DataFrame(), session: dict = _empty_session(), meta: dict = dict())

A batch journal: the cells of an experiment plus session/meta state.

Attributes:

  • name (str | None) –

    batch name.

  • project (str | None) –

    project name.

  • pages (DataFrame) –

    one row per cell; filename is a column (keys-in-columns).

  • session (dict) –

    mutable session state (starred/bad_cells/bad_cycles/notes).

  • meta (dict) –

    free-form metadata carried through save/load (name, project, time_stamp, project_dir, ...).

cell_names property

cell_names: list[str]

The cell labels, in page order.

LoadPolicy dataclass

LoadPolicy(source: SourcePreference = SourcePreference.AUTO, recalc: bool = False, max_cycle: int | None = None, accept_errors: bool = True, all_in_memory: bool = False, skip_bad_cells: bool = False, selector: dict | None = None, loader_kwargs: dict = dict(), overrides: dict = dict())

Batch-wide loading options (one object instead of a kwargs tunnel).

BatchResult dataclass

BatchResult(results: list[CellResult] = list())

The outcome of a batch run: one CellResult per cell.

cells

cells() -> dict[str, Any]

Mapping of label -> loaded cell, successful cells only.

raise_if_failed

raise_if_failed() -> 'BatchResult'

Strict mode: raise if any cell failed; otherwise return self.

report

report() -> pl.DataFrame

A tidy per-cell outcome frame (the dataframe errors only hinted at).

load

load(name: str | None = None, project: str | None = None, *, journal: Journal | None = None, journal_file: Path | str | None = None, journal_dir: Path | str | None = None, frame: Any | None = None, db: str | bool | None = None, db_reader: str | None = None, reader: str | None = None, reader_path: str | None = None, batch_col: str | None = None, policy: LoadPolicy | None = None, allow_from_journal: bool = True, force_reload: bool = False, force_raw_file: bool = False, force_cellpy: bool = False, force_recalc: bool = False, drop_bad_cells: bool = True, save_cellpy: bool = True, accept_errors: bool | None = None, max_cycle: int | None = None, **kwargs) -> Batch

Load a batch the notebook-friendly way (v1 orchestration on v3).

Resolves a journal (explicit file, cwd/journal_dir autoload, or database), runs update, optionally drops bad cells, and by default persists .cellpy files plus the journal JSON.

Journal location: journal_dir or cwd (typically the notebook folder when the kernel was started there). Autoload looks for cellpy_batch_{name}.json in that directory when allow_from_journal is True.

Parameters:

  • name / project

    batch identity (required for DB / autoload paths). project must be the exact folder name under rawdatadir when config.batch.auto_use_file_list is on (that dump joins rawdatadir / project — no fuzzy match).

  • journal (Journal | None, default: None ) –

    explicit in-memory journal model.

  • journal_file (Path | str | None, default: None ) –

    path to a cellpy journal, or a BatBase/custom JSON DB file when reader / db_reader is a JSON reader.

  • journal_dir (Path | str | None, default: None ) –

    directory for journal autoload/save (default: cwd).

  • frame (Any | None, default: None ) –

    build journal pages from a dataframe.

  • db / db_reader / reader

    database reader selection (reader aliases db_reader).

  • reader_path (str | None, default: None ) –

    DB file path for non-default readers.

  • batch_col (str | None, default: None ) –

    Excel batch column (default b01 when reading the DB).

  • policy (LoadPolicy | None, default: None ) –

    explicit LoadPolicy (conflicts with force_* raise).

  • allow_from_journal (bool, default: True ) –

    autoload cellpy_batch_{name}.json when present.

  • force_reload (bool, default: False ) –

    kept for API parity; journal hits always update().

  • force_raw_file / force_cellpy

    map to RAW_ONLY / CELLPY_ONLY. Default source is AUTO (existing local .cellpy, else raw). Use policy=LoadPolicy(source=SourcePreference.NEWEST) for the raw-vs-cellpy freshness check.

  • force_recalc (bool, default: False ) –

    remake step table + summary after load (needed when journal meta like nom_cap changed).

  • drop_bad_cells (bool, default: True ) –

    drop session["bad_cells"] before update.

  • save_cellpy (bool, default: True ) –

    write journal JSON and any newly-needed .cellpy files (default True). Skips rewriting cells already loaded from disk.

  • accept_errors / max_cycle

    forwarded into the load policy.

  • **kwargs

    DB engine knobs (column_map, raw_file_dir, …), load knobs forwarded to update (executor, on_progress, progress and LoadPolicy fields) and loader extras (testing, …). progress=None auto-shows tqdm on a TTY or in Jupyter; False disables; True forces; a callable receives progress events. executor="threads" speeds up reopening cells from local .cellpy files; a first load of remote raw files stays serial on the wire. executor="processes" cannot return live cells (pickle); raw loads are saved in the worker and reopened from .cellpy so save_cellpy=True works. export_cycles / export_raw / export_ica are accepted but ignored (warned once).

Note

executor chooses how cells are loaded (forwarded to update). Suggested use:

  • "serial" (default) — first load from remote raw files. SFTP copies do not overlap, so threads buy almost nothing on the download path (keep this for force_raw_file=True / a missing .cellpy).
  • "threads" — reopen from local .cellpy files (the usual second batch.load after save_cellpy=True). Measured ~2–3× on a warm 25-cell batch. Progress shows one child bar per in-flight cell.
  • "processes" — rarely worth it; Windows process spawn usually eats the gain, and workers return outcomes only (no live cells).

config.batch.auto_use_file_list (default False) is a config flag, not a load() kwarg. When True, file search dumps rawdatadir / project once. project must match that folder name exactly or the dump raises. Leave it False unless the raw tree is large enough that a per-cell walk hurts.

Example

First load (leave serial on the wire)::

b = batch.load(name="exp", project="Proj")

Later reopen from saved .cellpy files::

b = batch.load(name="exp", project="Proj", executor="threads")

Returns:

  • Batch

    Populated Batch.

Raises:

  • ValueError

    missing required args, force-flag conflicts, missing journal when journal_file is set but absent.

  • FileNotFoundError

    journal_file path does not exist.

from_journal

from_journal(journal_file: Path | str, policy: LoadPolicy | None = None, **_kwargs) -> Batch

Build a Batch from a cellpy journal file (.json or .xlsx).

For BatBase / custom JSON downloads that need post-read file search, use load with db_reader="batbase_json_reader" or "custom_json_reader" (and column_map for custom JSON).

from_cells

from_cells(cells, **kwargs) -> Batch

Build a Batch from already-loaded cells (see from_cells) -- feed it to collect_summaries / collect_cycles or call batch.plot().

Batch

Batch

Batch(journal: Journal, policy: LoadPolicy | None = None, _db: dict | None = None)

A loaded experiment: journal, cells, summaries, and a summary plot.

Typical use::

from cellpy import batch
b = batch.load(name="exp", project="proj")
b.summaries
b.cells["cell_01"]
b.plot()
b.result.report()

Attributes:

  • journal

    The Journal (pages + session).

  • policy

    The LoadPolicy used on the last load.

  • pages (DataFrame) –

    Journal table (filename, mass, group, …).

  • cell_names (list[str]) –

    Labels in journal order.

  • cells (CellStore) –

    Lazy CellStore of CellpyCell objects.

  • summaries (DataFrame) –

    Combined per-cycle summaries.

  • result (BatchResult | None) –

    BatchResult from the last update.

cell_names property

cell_names: list[str]

Cell labels in journal order.

cells property

cells: CellStore

Loaded cells, keyed by label (b.cells['cell_01']).

pages property

pages: DataFrame

Journal table (filename, mass, group, raw_file_names, …).

result property

result: BatchResult | None

Per-cell load outcomes from the last update / load.

Use b.result.report() for a tidy frame (outcome, source, error).

summaries property

summaries: DataFrame

Combined per-cycle summary frame across the batch (cached).

tests property

tests: DataFrame

Per-test metadata across the batch (tidy long-format).

One row per (cell, test_id) with the native TestMeta fields plus cell/group/sub_group keys -- the per-test records a merged (campaign) cell carries. Empty frame if no cell exposes test metadata.

combine_summaries

combine_summaries(**_kwargs) -> pl.DataFrame

Rebuild and return the combined summary frame (clears the cache).

create_journal

create_journal(**kwargs) -> Journal

Populate the journal from the configured database (if any).

Mirrors the legacy init() -> create_journal() flow: init stores the db config, create_journal performs the read.

drop

drop(label: str) -> 'Batch'

Remove label from the journal and the store now.

Parameters:

  • label (str) –

    Cell name as in b.cell_names.

drop_cells_marked_bad

drop_cells_marked_bad() -> 'Batch'

Drop every label listed in journal.session["bad_cells"].

export_journal

export_journal(path: Path | str | None = None) -> Path

Alias of save.

export_project

export_project(destination: Path | str, *, journal_path: Path | str | None = None) -> Path

Write a shareable bundle: .cellpy files plus a rewritten journal.

2.x replacement for 1.x duplicate_cellpy_files(location="standard"). Each loaded cell is saved to <destination>/<label>.cellpy, journal cellpy_file_name is rewritten to those paths (cwd-relative posix when possible), and the journal is saved (Batch.save semantics: cellpy_batch_<name>.json in the cwd unless journal_path is given).

Does not copy raw files or clear raw_file_names. Unloaded cells raise ValueError — call update first.

Parameters:

  • destination (Path | str) –

    directory for the .cellpy files (created if needed).

  • journal_path (Path | str | None, default: None ) –

    optional journal JSON path; default is cwd cellpy_batch_<name>.json.

Returns:

  • Path

    Path of the written journal.

from_cells classmethod

from_cells(cells: Mapping[str, Any] | Sequence[Any], *, groups: Mapping[str, Any] | None = None, sub_groups: Mapping[str, Any] | None = None, group_labels: Mapping[Any, str] | None = None, selected: Mapping[str, bool] | None = None, name: str = 'in_memory', project: str = 'in_memory', policy: LoadPolicy | None = None) -> 'Batch'

Build a batch from already-loaded CellpyCell objects.

The pieces a batch needs -- a journal pages frame (polars, keyed by filename) plus a populated cell store -- are constructed here, so a GUI/notebook holding cells in memory can feed them straight to collect_summaries / collect_cycles or batch.plot() without writing a journal to disk.

Parameters:

  • cells (Mapping[str, Any] | Sequence[Any]) –

    {label: CellpyCell} or a sequence of cells (labels are taken from cell.cell_name, falling back to cell_001 ..., de-duplicated).

  • groups / sub_groups

    optional {label: value} maps (defaults: group 1 for all; sub_group 1..N).

  • group_labels (Mapping[Any, str] | None, default: None ) –

    optional {group: display label} map.

  • selected (Mapping[str, bool] | None, default: None ) –

    optional {label: bool} (defaults True) for the journal selected column.

  • name / project

    journal name/project.

from_db classmethod

from_db(name: str, project: str, policy: LoadPolicy | None = None, **db_kwargs) -> 'Batch'

Build a batch by reading a database (Excel or JSON).

link() -> 'Batch'

No-op in batch v3 (the store loads lazily); kept for the surface.

load

load(**overrides) -> BatchResult

Load cells (alias of update, kept for the legacy surface).

Takes the same executor / on_progress / progress / policy overrides as update, e.g. b.load(executor="threads").

make_summaries

make_summaries() -> pl.DataFrame

Alias of combine_summaries.

mark_as_bad

mark_as_bad(label: str) -> None

Flag label in journal.session['bad_cells'] (does not drop).

Parameters:

  • label (str) –

    Cell name as in b.cell_names.

paginate

paginate() -> tuple[Path, ...]

Create the batch project folders and return their paths.

plot

plot(backend: str | None = None, show: bool = False, **kwargs) -> Any

Plot combined summaries for this batch.

Parameters:

  • backend (str | None, default: None ) –

    Plotting backend (None uses the configured default).

  • show (bool, default: False ) –

    If True, display the figure immediately.

  • **kwargs

    Forwarded to the summary plotter (columns, grouping, …).

Returns:

  • Any

    A figure from the plotting layer (backend-dependent).

recalc

recalc(**overrides) -> BatchResult

Reload every cell and remake step tables and summaries.

Same kwargs as update.

report

report(check: bool = True) -> pl.DataFrame

QC-style per-cell status frame (not the same as result.report()).

Parameters:

  • check (bool, default: True ) –

    kept for the legacy surface; the frame is always built.

save

save(path: Path | str | None = None) -> Path

Write the journal JSON (default cellpy_batch_<name>.json in cwd).

Parameters:

  • path (Path | str | None, default: None ) –

    Destination file. None uses the default name from journal.name.

update

update(on_progress=None, executor: str = 'serial', progress=None, **overrides) -> BatchResult

Load every cell, caching them in the store.

executor is "serial" (default), "threads" or "processes". "threads" mainly speeds up reopening cells from local .cellpy files; a first load of remote raw files does not overlap on the wire, and "processes" usually loses to spawn overhead on Windows. Process workers strip live cells; a raw load is written to .cellpy in the worker so save_cellpy=True can persist without None.save. progress is None (auto: TTY or Jupyter), False (off), True (force), or a callable that receives progress events. on_progress(i, n, result) still wins when set (3-arg callback). Known LoadPolicy fields in overrides update the policy; unknown (legacy) kwargs like testing are forwarded to the loader (cellpy.get) via loader_kwargs.

Journal

Journal dataclass

Journal(name: str | None = None, project: str | None = None, pages: DataFrame = pl.DataFrame(), session: dict = _empty_session(), meta: dict = dict())

A batch journal: the cells of an experiment plus session/meta state.

Attributes:

  • name (str | None) –

    batch name.

  • project (str | None) –

    project name.

  • pages (DataFrame) –

    one row per cell; filename is a column (keys-in-columns).

  • session (dict) –

    mutable session state (starred/bad_cells/bad_cycles/notes).

  • meta (dict) –

    free-form metadata carried through save/load (name, project, time_stamp, project_dir, ...).

cell_names property

cell_names: list[str]

The cell labels, in page order.

Aggregation

aggregate

Batch aggregation.

Turns a set of loaded cells into one tidy, long-format frame with cell / group / sub_group key columns -- replacing the legacy wide/multiindex join_summaries machinery. This is the frame the collectors redesign (Epic B) builds on, so it lands here as batch.aggregate.combine_summaries.

combine_summaries

combine_summaries(cells: Mapping[str, Any], journal: Journal | None = None) -> pl.DataFrame

Concatenate per-cell summaries into one tidy long-format frame.

Each row keeps its cell's summary columns plus cell/group/ sub_group keys. Cells without a summary are skipped. Returns an empty frame when nothing has a summary.

combine_tests

combine_tests(cells: Mapping[str, Any], journal: Journal | None = None) -> pl.DataFrame

Per-test metadata across the batch as one tidy long-format frame.

One row per (cell, test_id) carrying the native TestMeta fields plus cell/group/sub_group keys. Surfaces the per-test records that a merged (campaign) cell holds in Data.tests at the batch level. Returns an empty frame when no cell exposes test metadata.