cellpy.utils.batch
#
Routines for batch processing of cells (v2).
Module Contents#
Classes#
A convenience class for running batch procedures. |
Functions#
|
Create a Batch from a journal file |
|
Returns an initialized instance of the Batch class. |
|
Iterate through all journals in given folder. |
|
Load a batch from a journal file or create a new batch and load it if the journal file does not exist. |
|
Load a journal file. |
|
Retrieve pages from a Journal file. |
|
Returns an empty instance of the Batch class. |
|
Execute a batch run, either from a given file_name or by giving the name and project as input. |
Attributes#
- class Batch(*args, **kwargs)[source]#
A convenience class for running batch procedures.
The Batch class contains (among other things):
iterator protocol
a journal with info about the different cells where the main information is accessible as a pandas.DataFrame through the
.pages
attributea data lookup accessor
.data
that behaves similarly as a dict.
The initialization accepts arbitrary arguments and keyword arguments. It first looks for the
file_name
anddb_reader
keyword arguments.Usage:
b = Batch((name, (project)), **kwargs)
Examples
>>> b = Batch("experiment001", "main_project") >>> b = Batch("experiment001", "main_project", batch_col="b02") >>> b = Batch(name="experiment001", project="main_project", batch_col="b02") >>> b = Batch(file_name="cellpydata/batchfiles/cellpy_batch_experiment001.json")
- Parameters:
name (str) – (project (str))
- Keyword Arguments:
file_name (str or pathlib.Path) – journal file name to load.
db_reader (str) – data-base reader to use (defaults to “default” as given in the config-file or prm-class).
frame (pandas.DataFrame) – load from given dataframe.
default_log_level (str) – custom log-level (defaults to None (i.e. default log-level in cellpy)).
custom_log_dir (str or pathlib.Path) – custom folder for putting the log-files.
force_raw_file (bool) – load from raw regardless (defaults to False).
force_cellpy (bool) – load cellpy-files regardless (defaults to False).
force_recalc (bool) – Always recalculate (defaults to False).
export_cycles (bool) – Extract and export individual cycles to csv (defaults to True).
export_raw (bool) – Extract and export raw-data to csv (defaults to True).
export_ica (bool) – Extract and export individual dQ/dV data to csv (defaults to True).
accept_errors (bool) – Continue automatically to next file if error is raised (defaults to False).
nom_cap (float) – give a nominal capacity if you want to use another value than the one given in the config-file or prm-class.
- property cells: cellpy.utils.batch_tools.batch_core.Data[source]#
Access cells as a Data object (attribute lookup and automatic loading).
Note
Write
b.cells.x
and press <TAB>. Then a pop-up might appear, and you can choose the cell you would like to retrieve.Warning
It seems that it is not always working as intended, at least not in my jupyter lab anymore. Instead, you can use
b.experiment.data
or writecells = b.cells
and then usecells.x
and press <TAB> to get the pop-up.
- property info_file[source]#
The name of the info file.
Warning
Will be deprecated soon - use
journal_name
instead.
- property journal: cellpy.utils.batch_tools.batch_journals.LabJournal[source]#
- property view[source]#
Show the selected info about each cell.
Warning
Will be deprecated soon - use report() instead.
- combine_summaries(export_to_csv=True, **kwargs) None [source]#
Combine selected columns from each of the cells into single frames.
- Keyword Arguments:
export_to_csv (bool) – export the combined summaries to csv (defaults to True).
**kwargs – sent to the summary_collector.
- Returns:
None
- create_journal(description=None, from_db=True, auto_use_file_list=None, file_list_kwargs=None, **kwargs)[source]#
Create journal pages.
This method is a wrapper for the different Journal methods for making journal pages (
Batch.experiment.journal.xxx
). It is under development. If you want to use ‘advanced’ options (i.e. not loading from a db), please consider using the methods available in Journal for now.- Parameters:
description –
the information and meta-data needed to generate the journal pages:
empty: create an empty journal
dict
: create journal pages from a dictionarypd.DataFrame
: create journal pages from apandas.DataFrame
’filename.json’: load cellpy batch file
’filename.xlsx’: create journal pages from an Excel file.
from_db (bool) – Deprecation Warning: this parameter will be removed as it is the default anyway. Generate the pages from a db (the default option). This will be over-ridden if description is given.
auto_use_file_list (bool) – Experimental feature. If True, a file list will be generated and used instead of searching for files in the folders.
file_list_kwargs (dict) – Experimental feature. Keyword arguments to be sent to the file list generator.
**kwargs – sent to sub-function(s) (e.g.
from_db
->simple_db_reader
->find_files
->filefinder.search_for_files
).
The following keyword arguments are picked up by
from_db
:- Transferred Parameters:
project – None
name – None
batch_col – None
The following keyword arguments are picked up by
simple_db_reader
:- Transferred Parameters:
reader – a reader object (defaults to dbreader.Reader)
cell_ids – keys (cell IDs)
file_list – file list to send to filefinder (instead of searching in folders for files).
pre_path – prepended path to send to filefinder.
include_key – include the key col in the pages (the cell IDs).
include_individual_arguments – include the argument column in the pages.
additional_column_names – list of additional column names to include in the pages.
The following keyword arguments are picked up by
filefinder.search_for_files
:- Transferred Parameters:
run_name (str) – run-file identification.
raw_extension (str) – optional, extension of run-files (without the ‘.’).
cellpy_file_extension (str) – optional, extension for cellpy files (without the ‘.’).
raw_file_dir (path) – optional, directory where to look for run-files (default: read prm-file)
project_dir (path) – subdirectory in raw_file_dir to look for run-files
cellpy_file_dir (path) – optional, directory where to look for cellpy-files (default: read prm-file)
prm_filename (path) – optional parameter file can be given.
file_name_format (str) – format of raw-file names or a glob pattern (default: YYYYMMDD_[name]EEE_CC_TT_RR).
reg_exp (str) – use regular expression instead (defaults to None).
sub_folders (bool) – perform search also in sub-folders.
file_list (list of str) – perform the search within a given list of filenames instead of searching the folder(s). The list should not contain the full filepath (only the actual file names). If you want to provide the full path, you will have to modify the file_name_format or reg_exp accordingly.
pre_path (path or str) – path to prepend the list of files selected from the file_list.
The following keyword arguments are picked up by
journal.to_file
:- Transferred Parameters:
duplicate_to_local_folder (bool) – default True.
- Returns:
None
- drop(cell_label=None)[source]#
Drop cells from the journal.
If
cell_label
is not given,cellpy
will look into the journal for session info about bad cells, and if it finds it, it will remove those from the journal.Note
Remember to save your journal again after modifying it.
Warning
This method has not been properly tested yet.
- Parameters:
cell_label (str) – the cell label of the cell you would like to remove.
- Returns:
cellpy.utils.batch
object (returns a copy if keep_old isTrue
).
- drop_cell(cell_label)[source]#
Drop a cell from the journal.
- Parameters:
cell_label – the cell label of the cell you would like to remove.
- drop_cells(cell_labels)[source]#
Drop cells from the journal.
- Parameters:
cell_labels – the cell labels of the cells you would like to remove.
- drop_cells_marked_bad()[source]#
Drop cells that has been marked as bad from the journal (experimental feature).
- duplicate_cellpy_files(location: str = 'standard', selector: dict = None, **kwargs) None [source]#
Copy the cellpy files and make a journal with the new names available in the current folder.
- Parameters:
location –
where to copy the files. Either choose among the following options:
’standard’: data/interim folder
’here’: current directory
’cellpydatadir’: the stated cellpy data dir in your settings (prms)
or if the location is not one of the above, use the actual value of the location argument.
selector (dict) – if given, the cellpy files are reloaded after duplicating and modified based on the given selector(s).
**kwargs – sent to
Batch.experiment.update
if selector is provided
- Returns:
The updated journal pages.
- duplicate_journal(folder=None) None [source]#
Copy the journal to folder.
- Parameters:
folder (str or pathlib.Path) – folder to copy to (defaults to the
folder). (current) –
- export_journal(filename=None) None [source]#
Export the journal to xlsx.
- Parameters:
filename (str or pathlib.Path) – the name of the file to save the journal to. If not given, the journal will be saved to the default name.
- link(max_cycle=None, force_combine_summaries=False) None [source]#
Link journal content to the cellpy-files and load the step information.
- Parameters:
max_cycle (int) – set maximum cycle number to link to.
force_combine_summaries (bool) – automatically run combine_summaries (set this to True if you are re-linking without max_cycle for a batch that previously were linked with max_cycle)
- load() None [source]#
Load the selected datasets.
Warning
Will be deprecated soon - use
update
instead.
- make_summaries() None [source]#
Combine selected columns from each of the cells into single frames and export.
Warning
This method will be deprecated in the future. Use
combine_summaries
instead.
- mark_as_bad(cell_label)[source]#
Mark a cell as bad (experimental feature).
- Parameters:
cell_label – the cell label of the cell you would like to mark as bad.
- plot(backend=None, reload_data=False, **kwargs)[source]#
Plot the summaries (e.g. capacity vs. cycle number).
- Parameters:
backend (str) – plotting backend (plotly, bokeh, matplotlib, seaborn)
reload_data (bool) – reload the data before plotting
**kwargs – sent to the plotter
- Keyword Arguments:
color_map (str, any) – color map to use (defaults to
px.colors.qualitative.Set1
forplotly
and “Set1” forseaborn
)ce_range (list) – optional range for the coulombic efficiency plot
min_cycle (int) – minimum cycle number to plot
max_cycle (int) – maximum cycle number to plot
title (str) – title of the figure (defaults to “Cycle Summary”)
x_label (str) – title of the x-label (defaults to “Cycle Number”)
direction (str) – plot charge or discharge (defaults to “charge”)
rate (bool) – (defaults to False)
ir (bool) – (defaults to True)
group_legends (bool) – group the legends so that they can be turned visible/invisible as a group (defaults to True) (only for plotly)
base_template (str) – template to use for the plot (only for plotly)
- plot_summaries(output_filename=None, backend=None, reload_data=False, **kwargs) None [source]#
Plot the summaries.
Warning
This method will be deprecated in the future. Use
plot
instead.
- recalc(**kwargs) None [source]#
Run
make_step_table
andmake_summary
on all cells.- Keyword Arguments:
save (bool) – Save updated cellpy-files if True (defaults to True).
step_opts (dict) – parameters to inject to make_steps (defaults to None).
summary_opts (dict) – parameters to inject to make_summary (defaults to None).
indexes (list) – Only recalculate for given indexes (i.e. list of cell-names) (defaults to None).
calc_steps (bool) – Run make_steps before making the summary (defaults to True).
testing (bool) – Only for testing purposes (defaults to False).
- Returns:
None
- remove_mark_as_bad(cell_label)[source]#
Remove the bad cell mark from a cell (experimental feature).
- Parameters:
cell_label – the cell label of the cell you would like to remove the bad mark from.
- report(stylize=True, grouped=False, check=False)[source]#
Create a report on all the cells in the batch object.
Important
To perform a reporting, cellpy needs to access all the data (and it might take some time).
- Parameters:
stylize (bool) – apply some styling to the report (default True).
grouped (bool) – add information based on the group cell belongs to (default False).
check (bool) – check if the data seems to be without errors (0 = no errors, 1 = partial duplicates) (default False).
- Returns:
pandas.DataFrame
- save_journal() None [source]#
Save the journal (json-format).
The journal file will be saved in the project directory and in the batch-file-directory (
prms.Paths.batchfiledir
). The latter is useful for processing several batches using theiterate_batches
functionality.
- show_pages(number_of_rows=5)[source]#
Show the journal pages.
Warning
Will be deprecated soon - use pages.head() instead.
- update(pool=False, **kwargs) None [source]#
Updates the selected datasets.
- Keyword Arguments:
all_in_memory (bool) – store the cellpydata in memory (default False)
cell_specs (dict of dicts) – individual arguments pr. cell. The
cellspecs
key-word argument dictionary will override the **kwargs and the parameters from the journal pages for the indicated cell.logging_mode (str) – sets the logging mode for the loader(s).
accept_errors (bool) – if True, the loader will continue even if it encounters errors.
Additional keyword arguments are sent to the loader(s) if not picked up earlier. Remark that you can obtain the same pr. cell by providing a
cellspecs
dictionary. The kwargs have precedence over the parameters given in the journal pages, but will be overridden by parameters given bycellspecs
.Merging picks up the following keyword arguments:
- Transferred Parameters:
recalc (Bool) – set to False if you don’t want automatic recalculation of cycle numbers etc. when merging several data-sets.
Loading picks up the following keyword arguments:
- Transferred Parameters:
selector (dict) – selector-based parameters sent to the cellpy-file loader (hdf5) if loading from raw is not necessary (or turned off).
- from_journal(journal_file, autolink=True, testing=False) Batch [source]#
Create a Batch from a journal file
- init(*args, empty=False, **kwargs) Batch [source]#
Returns an initialized instance of the Batch class.
- Parameters:
empty (bool) – if True, the batch will not be linked to any database and an empty batch is returned
*args –
passed directly to Batch()
name: name of batch.
project: name of project.
batch_col: batch column identifier.
- Keyword Arguments:
file_name – json file if loading from pages (journal).
default_log_level – “INFO” or “DEBUG”. Defaults to “CRITICAL”.
Other keyword arguments are sent to the Batch object.
Examples
>>> empty_batch = Batch.init(db_reader=None) >>> batch_from_file = Batch.init(file_name="cellpy_batch_my_experiment.json") >>> normal_init_of_batch = Batch.init()
- iterate_batches(folder, extension='.json', glob_pattern=None, **kwargs)[source]#
Iterate through all journals in given folder.
Warning
This function is from ancient times and needs to be updated. It might have grown old and grumpy. Expect it to change in the near future.
- Parameters:
folder (str or pathlib.Path) – folder containing the journal files.
extension (str) – extension for the journal files (used when creating a default glob-pattern).
glob_pattern (str) – optional glob pattern.
**kwargs – keyword arguments passed to
batch.process_batch
.
- load(name, project, batch_col=None, allow_from_journal=True, drop_bad_cells=True, force_reload=False, **kwargs)[source]#
Load a batch from a journal file or create a new batch and load it if the journal file does not exist.
- Parameters:
name (str) – name of batch
project (str) – name of project
batch_col (str) – batch column identifier (only used for loading from db with simple_db_reader)
allow_from_journal (bool) – if True, the journal file will be loaded if it exists
force_reload (bool) – if True, the batch will be reloaded even if the journal file exists
drop_bad_cells (bool) – if True, bad cells will be dropped (only apply if journal file is loaded)
**kwargs – sent to Batch during initialization
- Returns:
populated Batch object (
cellpy.utils.batch.Batch
)
- load_journal(journal_file, **kwargs)[source]#
Load a journal file.
- Parameters:
journal_file (str) – path to journal file.
**kwargs – sent to
Journal.from_file
- Returns:
journal
- load_pages(file_name) pandas.DataFrame [source]#
Retrieve pages from a Journal file.
This function is here to let you easily inspect a Journal file without starting up the full batch-functionality.
Examples
>>> from cellpy.utils import batch >>> journal_file_name = 'cellpy_journal_one.json' >>> pages = batch.load_pages(journal_file_name)
- Returns:
pandas.DataFrame
- naked(name=None, project=None) Batch [source]#
Returns an empty instance of the Batch class.
Examples
>>> empty_batch = naked()
- process_batch(*args, **kwargs) Batch [source]#
Execute a batch run, either from a given file_name or by giving the name and project as input.
Warning
This function is from ancient times and needs to be updated. It might have grown old and grumpy. Expect it to change in the near future.
Examples
>>> process_batch(file_name | (name, project), **kwargs)
- Parameters:
*args – file_name or name and project (both string)
- Keyword Arguments:
backend (str) – what backend to use when plotting (‘bokeh’ or ‘matplotlib’). Defaults to ‘matplotlib’.
dpi (int) – resolution used when saving matplotlib plot(s). Defaults to 300 dpi.
default_log_level (str) – What log-level to use for console output. Chose between ‘CRITICAL’, ‘DEBUG’, or ‘INFO’. The default is ‘CRITICAL’ (i.e. usually no log output to console).
- Returns:
cellpy.batch.Batch
object