Shadow Pricing#

The shadow pricing calculator used by work and school location choice.

Structure#

Configuration File: shadow_pricing.yaml

Turning on and saving shadow prices#

Shadow pricing is activated by setting the use_shadow_pricing to True in the settings.yaml file. Once this setting has been activated, ActivitySim will search for shadow pricing configuration in the shadow_pricing.yaml file. When shadow pricing is activated, the shadow pricing outputs will be exported by the tracing engine. As a result, the shadow pricing output files will be prepended with trace followed by the iteration number the results represent. For example, the shadow pricing outputs for iteration 3 of the school location model will be called trace.shadow_price_school_shadow_prices_3.csv.

In total, ActivitySim generates three types of output files for each model with shadow pricing:

trace.shadow_price_<model>_desired_size.csv The size terms by zone that the ctramp and daysim methods are attempting to target. These equal the size term columns in the land use data multiplied by size term coefficients.
trace.shadow_price_<model>_modeled_size_<iteration>.csv These are the modeled size terms after the iteration of shadow pricing identified by the number. In other words, these are the predicted choices by zone and segment for the model after the iteration completes. (Not applicable for simulation option.)
trace.shadow_price_<model>_shadow_prices_<iteration>.csv The actual shadow price for each zone and segment after the of shadow pricing. This is the file that can be used to warm start the shadow pricing mechanism in ActivitySim. (Not applicable for simulation option.)

There are three shadow pricing methods in activitysim: ctramp, daysim, and simulation. The first two methods try to match model output with workplace/school location model size terms, while the last method matches model output with actual employment/enrollmment data.

The simulation approach operates the following steps. First, every worker / student will be assigned without shadow prices applied. The modeled share and the target share for each zone are compared. If the zone is overassigned, a sample of people from the over-assigned zones will be selected for re-simulation. Shadow prices are set to -999 for the next iteration for overassigned zones which removes the zone from the set of alternatives in the next iteration. The sampled people will then be forced to choose from one of the under-assigned zones that still have the initial shadow price of 0. (In this approach, the shadow price variable is really just a switch turning that zone on or off for selection in the subsequent iterations. For this reason, warm-start functionality for this approach is not applicable.) This process repeats until the overall convergence criteria is met or the maximum number of allowed iterations is reached.

Because the simulation approach only re-simulates workers / students who were over-assigned in the previous iteration, run time is significantly less (~90%) than the CTRAMP or DaySim approaches which re-simulate all workers and students at each iteration.

Configuration#

settings activitysim.abm.tables.shadow_pricing.ShadowPriceSettings#

Bases: PydanticReadable

Settings used for shadow pricing.

Fields:

field DAMPING_FACTOR: float = 1#: ctramp-style damping factor

field DAYSIM_ABSOLUTE_TOLERANCE: float = 50#

field DAYSIM_PERCENT_TOLERANCE: float = 10#

field FAIL_THRESHOLD: float = 10#: max percentage of zones allowed to fail

field LOAD_SAVED_SHADOW_PRICES: bool = True#

Global switch to enable/disable loading of saved shadow prices.

This is ignored if global use_shadow_pricing switch is False

field MAX_ITERATIONS: int = 5#: Number of shadow price iterations for cold start.

field MAX_ITERATIONS_SAVED: int = 1#

Number of shadow price iterations for warm start.

A warm start means saved shadow_prices were found in a file and loaded.

field PERCENT_TOLERANCE: float = 5#: zone passes if modeled is within percent_tolerance of predicted_size

field SCALE_SIZE_TABLE: bool = False#

field SEGMENT_TO_NAME: dict[str, str] = {'school': 'school_segment', 'workplace': 'income_segment'}#: Mapping from model_selector to persons_segment_name.

field SHADOW_PRICE_METHOD: Literal['ctramp', 'daysim', 'simulation'] = 'ctramp'#

field SIZE_THRESHOLD: float = 10#: ignore criteria for zones smaller than size_threshold

field TARGET_THRESHOLD: float = 20#: ignore criteria for zones smaller than target_threshold (total employmnet or enrollment)

field WRITE_ITERATION_CHOICES: bool = False#

field school_segmentation_targets: dict[str, str] | None = None#

field shadow_pricing_models: dict[str, str] | None = None#: List model_selectors and model_names of models that use shadow pricing. This list identifies which size_terms to preload which must be done in single process mode, so predicted_size tables can be scaled to population

field workplace_segmentation_targets: dict[str, str] | None = None#

Examples#

Implementation#

activitysim.abm.tables.shadow_pricing.ShadowPriceCalculator(state: State, model_settings: TourLocationComponentSettings, num_processes, shared_data=None, shared_data_lock=None, shared_data_choice=None, shared_data_choice_lock=None, shared_sp_choice_df=None)#

activitysim.abm.tables.shadow_pricing.buffers_for_shadow_pricing(shadow_pricing_info)#

Allocate shared_data buffers for multiprocess shadow pricing

Allocates one buffer per model_selector. Buffer datatype and shape specified by shadow_pricing_info

buffers are multiprocessing.Array (RawArray protected by a multiprocessing.Lock wrapper) We don’t actually use the wrapped version as it slows access down and doesn’t provide protection for numpy-wrapped arrays, but it does provide a convenient way to bundle RawArray and an associated lock. (ShadowPriceCalculator uses the lock to coordinate access to the numpy-wrapped RawArray.)

Parameters:

shadow_pricing_infodict

Returns:

data_buffersdict {<model_selector><shared_data_buffer>}
dict of multiprocessing.Array keyed by model_selector

activitysim.abm.tables.shadow_pricing.buffers_for_shadow_pricing_choice(state, shadow_pricing_choice_info)#

Same as above buffers_for_shadow_price function except now we need to store the actual choices for the simulation based shadow pricing method

This allocates a multiprocessing.Array that can store the choice for each person and then wraps a dataframe around it. That means the dataframe can be shared and accessed across all threads. Parameters ———- shadow_pricing_info : dict Returns ——-

data_buffers : dict {<model_selector> : <shared_data_buffer>} dict of multiprocessing.Array keyed by model_selector

and wrapped in a pandas dataframe

activitysim.abm.tables.shadow_pricing.shadow_price_data_from_buffers_choice(data_buffers, shadow_pricing_info, model_selector)#

Parameters:

data_buffersdict of {<model_selector><multiprocessing.Array>}

multiprocessing.Array is simply a convenient way to bundle Array and Lock we extract the lock and wrap the RawArray in a numpy array for convenience in indexing The shared data buffer has shape (<num_zones, <num_segments> + 1) extra column is for reverse semaphores with TALLY_CHECKIN and TALLY_CHECKOUT

shadow_pricing_infodict

dict of useful info: dtype: sp_dtype, block_shapes : OrderedDict({<model_selector>: <shape tuple>}) dict mapping model_selector to block shape (including extra column for semaphores) e.g. {‘school’: (num_zones, num_segments + 1)

model_selectorstr

location type model_selector (e.g. school or workplace)

Returns:

shared_data, shared_data_lock: shared_data : multiprocessing.Array or None (if single process) shared_data_lock : numpy array wrapping multiprocessing.RawArray or None (if single process)

activitysim.abm.tables.shadow_pricing.shadow_price_data_from_buffers(data_buffers, shadow_pricing_info, model_selector)#

Parameters:

data_buffersdict of {<model_selector><multiprocessing.Array>}

shadow_pricing_infodict

dict of useful info: dtype: sp_dtype, block_shapes : OrderedDict({<model_selector>: <shape tuple>}) dict mapping model_selector to block shape (including extra column for semaphores) e.g. {‘school’: (num_zones, num_segments + 1)

model_selectorstr

location type model_selector (e.g. school or workplace)

Returns:

shared_data, shared_data_lock: shared_data : multiprocessing.Array or None (if single process) shared_data_lock : numpy array wrapping multiprocessing.RawArray or None (if single process)

activitysim.abm.tables.shadow_pricing.load_shadow_price_calculator(state: State, model_settings: TourLocationComponentSettings)#

Initialize ShadowPriceCalculator for model_selector (e.g. school or workplace)

If multiprocessing, get the shared_data buffer to coordinate global_desired_size calculation across sub-processes

Parameters:

stateworkflow.State
model_settingsTourLocationComponentSettings

Returns:

spcShadowPriceCalculator

activitysim.abm.tables.shadow_pricing.add_size_tables(state: State, disaggregate_suffixes: dict[str, Any], scale: bool = True) → None#

inject tour_destination_size_terms tables for each model_selector (e.g. school, workplace)

Size tables are pandas dataframes with locations counts for model_selector by zone and segment tour_destination_size_terms

if using shadow pricing, we scale size_table counts to sample population (in which case, they have to be created while single-process)

Scaling is problematic as it breaks household result replicability across sample sizes It also changes the magnitude of the size terms so if they are used as utilities in expression files, their importance will diminish relative to other utilities as the sample size decreases.

Scaling makes most sense for a full sample in conjunction with shadow pricing, where shadow prices can be adjusted iteratively to bring modelled counts into line with desired (size table) counts.

activitysim.abm.tables.shadow_pricing.get_shadow_pricing_info(state)#

return dict with info about dtype and shapes of desired and modeled size tables

block shape is (num_zones, num_segments + 1)

Returns:

shadow_pricing_info: dict: dtype: <sp_dtype>, block_shapes: dict {<model_selector>: <block_shape>}

activitysim.abm.tables.shadow_pricing.get_shadow_pricing_choice_info(state)#

return dict with info about dtype and shapes of desired and modeled size tables

block shape is (num_zones, num_segments + 1)

Returns:

shadow_pricing_info: dict: dtype: <sp_dtype>, block_shapes: dict {<model_selector>: <block_shape>}