Using Skim Dataset#

ActivitySim 1.2 offers two internal frameworks for managing skim data.

  • SkimDict, the legacy framework, that stores all skim data in one large omnibus array, with various offset lookups and tools to access values.

  • SkimDataset, an xarray.Dataset based framework, which mimics the SkimDict interface, and adds a number of features optimized specifically for use with sharrow. This framework is automatically used when sharrow is enabled, and there is no user configuration to enable it separately.

Skims in Shared Memory#

These two internal frameworks manage shared memory differently when running ActivitySim in multiprocessing mode.

For SkimDict, shared memory is used only when running with multiprocessing active, and is allocated via the allocate_shared_skim_buffers function, which in turn invokes Network_LOS.allocate_shared_skim_buffers.

For SkimDataset, shared memory is used regardless of whether multiprocessing is active or not. The shared memory allocation is done via the Dataset.shm.to_shared_memory method called at the end of the load_skim_dataset_to_shared_memory function.

Skim Dataset API#

class activitysim.core.skim_dataset.DatasetWrapper(dataset, orig_key, dest_key, time_key=None, *, time_map=None)#

Mimics the SkimWrapper interface to allow legacy code to access data.

Parameters:
dataset: Dataset
orig_key, dest_key: str

name of columns in target dataframe to use as origin and destination lookups, respectively

time_keystr, optional
time_mapMapping, optional

A mapping from time period index numbers to (more aggregate) time period names.

lookup(key, reverse=False)#

Generally not called by the user - use __getitem__ instead

Parameters:
keyhashable

The key (identifier) for this skim object

odbool (optional)

od=True means lookup standard origin-destination skim value od=False means lookup destination-origin skim value

Returns:
impedances: pd.Series

A Series of impedances which are elements of the Skim object and with the same index as df

max(key)#

return max skim value in either o-d or d-o direction

reverse(key)#

return skim value in reverse (d-o) direction

set_df(df)#

Set the dataframe

Parameters:
dfDataFrame

The dataframe which contains the origin and destination ids

Returns:
self (to facilitate chaining)
class activitysim.core.skim_dataset.SkimDataset(dataset)#

A wrapper around xarray.Dataset containing skim data, with time period management.

get_skim_usage()#

return set of keys of skims looked up. e.g. {‘DIST’, ‘SOV’}

Returns:
set:
lookup(orig, dest, key)#

Return list of skim values of skims(s) at orig/dest in skim with the specified key (e.g. ‘DIST’)

Parameters:
orig: list of orig zone_ids
dest: list of dest zone_ids
key: str
Returns:
Numpy.ndarray: list of skim values for od pairs
wrap(orig_key, dest_key)#

Get a wrapper for the given keys.

Parameters:
orig_key, dest_keystr
Returns:
DatasetWrapper
wrap_3d(orig_key, dest_key, dim3_key)#

Get a 3d wrapper for the given keys.

Parameters:
orig_key, dest_keystr
Returns:
DatasetWrapper
activitysim.core.skim_dataset.load_skim_dataset_to_shared_memory(state, skim_tag='taz') → Dataset#

Load skims from disk into shared memory.

Parameters:
stateState
skim_tagstr, default “taz”
Returns:
xarray.Dataset
activitysim.core.skim_dataset.load_sparse_maz_skims(dataset, land_use_index, remapper, zone_system, maz2taz_file_name, maz_to_maz_tables=(), max_blend_distance=None, data_file_resolver=None)#

Load sparse MAZ data on top of TAZ skim data.

Parameters:
datasetxarray.Dataset

The existing dataset at TAZ resolution only.

land_use_indexpandas.Index

The index of the land use table. For two and three zone systems, these index values should be MAZ identifiers.

remapperdict, optional

A dictionary mapping where the keys are the original (nominal) zone id’s, and the values are the recoded (typically zero-based contiguous) zone id’s. Recoding improves runtime efficiency.

zone_systemint

Currently 1, 2 and 3 are supported.

maz2taz_file_namestr
maz_to_maz_tablesCollection[]
max_blend_distanceoptional
data_file_resolverfunction or Callable
Returns:
xarray.Dataset