ParquetStore

class activitysim.core.workflow.checkpoint.ParquetStore(directory: Path, mode: str = 'a', gitignore: bool = True)

Storage interface for parquet-based table storage.

This store will store each saved table in a parquet-format archive, resulting in a hierarchy of separate files in a defined structure, as opposed to a single monolithic repository files containing all the data.

This interface will fall back to storing tables in a gzipped pickle if the parquet format fails (as might happen if datatypes for some columns are not homogenous and values are stored as “object”).

Methods

__init__(directory[, mode, gitignore])	Initialize a storage interface for parquet-based table storage.
close()	Close this store.
from_hdf(source_filename, dest_filename[, mode])	Create a new checkpoint store from an existing HdfStore.
get_dataframe(table_name[, checkpoint_name])	Load table from store as a pandas DataFrame.
list_checkpoint_names()	Get a list of all checkpoint names in this store.
make_zip_archive(output_filename)	Compress this pipeline into a zip archive.
put(table_name, df[, complib, checkpoint_name])	Store a table.
wipe()	Remove this store, including all subdirectories.

Attributes

extension
filename	The directory location of this ParquetStore.
is_open	This store is open.
is_readonly	This store is read-only.