ABM3 Bike Route Choice Model Update

docs/design/supply/bike-logsums.md

@@ -11,23 +11,25 @@ This process generates the output\bikeMgraLogsum.csv and output\bikeTazLogsum.cs
 <!-- ![](../../images/design/bike_logsum_design.png) -->
 <img src="../../images/design/bike_logsum_design.png" width="200"/>
 
-**Inputs:** The AT network is read from input\SANDAG_Bike_Net.dbf and input\SANDAG_Bike_Node.dbf. Coefficients and other settings are read from sandag_abm.properties.
+**Inputs:** The AT network is read from input\SANDAG_Bike_Net.dbf and input\SANDAG_Bike_Node.dbf. The bike model settings and utility files are found in src\asim\scripts\bike_route_choice.
 
 **Network Construction:** Node and edge attributes are processed from the network files. Traversals are derived for edge pairs, using angle to determine turn type.
 
 **Path Sampling:**
 
 - Doubly stochastic, coefficients are randomized and resulting edge cost is randomized
-- Dijkstra's algorithm finds path each origin to destination with minimum cost
+- Dijkstra's algorithm finds path each origin to destination with minimum combined edge and traversal cost
 - Add paths to path alternatives list and calculate path sizes from overlap of alternatives
-- Repeat until minimum number of alternatives and minimum path size is met for each origin/destination pair
-
-**Path Resampling:** Resample paths from alternatives list until minimum path size is met.
+- Repeat for preset number of iterations (default 10)
 
 **Path Choice Utility Calculation:** Calculate bike logsum values for each origin/destination pair from utility expression on each path alternative.
 
-**Outputs:** Bike logsums and times are written to output\bikeMgraLogsum.csv and output\bikeTazLogsum.csv. During ActivitySim preprocessing, TAZ values are added to BIKE_LOGSUM and BIKE_TIME matrices of output\skims\traffic_skims_AM.omx and MGRA values are written to output\skims\maz_maz_bike.csv.
+**Outputs:** Bike logsums and times are written to output\bikeMgraLogsum.csv and output\bikeTazLogsum.csv. Log and trace files are written to output\bike. During ActivitySim preprocessing, TAZ values are added to BIKE_LOGSUM and BIKE_TIME matrices of output\skims\traffic_skims_AM.omx and MGRA values are written to output\skims\maz_maz_bike.csv.
 
 ## Further reading
 
-[Active Transportation Improvements Report (2015)](https://github.com/SANDAG/ABM/wiki/files/at.pdf)
+[Bike Route Choice README](../../../src/asim/scripts/bike_route_choice/README.md)
+
+[ABM3 Bike Model Report (2025)](../../pdf_reports/SANDAG_ABM3_Bike_Model_Report.pdf)
+
+[Active Transportation Improvements Report (2015)](https://github.com/SANDAG/ABM/wiki/files/at.pdf) (Prior bike logsum implementation in Java)

src/asim/scripts/bike_route_choice/README.md

@@ -0,0 +1,136 @@
+# Active Transport Bike Model
+
+This directory contains a Python-based implementation of a bike route 
+choice model first developed in Java. The bike route choice model is a 
+multinomial logit discrete choice model that estimates the probabilities
+of an individual’s choosing among multiple alternative routes between a given
+origin and destination. This discrete choice model forms the basis of both 
+the estimation of the level of service between OD pairs used by the demand 
+models and the estimation of the number of cyclists assigned to network links. 
+The level of service is the expected maximum utility, or logsum, from the model,
+and network link assignments are made with individual route probabilities.
+
+## Setup and Running
+The UV environment used to run AcitivtySim (asim_140) is also used to run the bike model.
+The bike model is launched from the `bike_route_choice.py` file and accepts a
+single (optional) command-line argument specifying the settings YAML file to
+read. The default value for this argument is `bike_route_choice_settings.yaml`.
+
+### Settings
+In the specified settings YAML file, several options are available for configuration:
+
+**Network**
+- `data_dir`: path to the directory in which the model inputs are stored
+- `node_file`: name of the input shapefile containing bike network nodes
+- `link_file`: name of the input shapefile containing bike network links
+- `zone_level`: zone level on which path choice should be performed. Allowed values: `taz`,`mgra`
+- `zone_subset`: subset of zones to use for model testing
+
+**Utilities**
+- `edge_util_file`: name of the file containing the utility function variable specification for link costs
+- `traversal_util_file`: name of the file containing the utility function variable specification for edtraversalge costs
+- `random_scale_coef`: scaling coefficient to use for alternative utilities
+- `random_scale_link`: scaling coefficient to use for randomized edge costs
+
+**Model Hyperparameters**
+- `number_of_iterations`: maximum number of paths which should be found before terminating
+- `number_of_batches`: number of batches into which origin centroids should be divided for sequential processing
+- `number_of_processors`: number of processors to use for processing each batch
+- `max_dijkstra_utility`: cutoff threshold utility (positive) for early termination of the shortest-paths search
+- `min_iterations`: the minimum number of paths found - zone pairs with fewer paths will be discarded
+
+**Output and Caching**
+- `output_path`: path to the directory in which model outputs should be written
+- `output_file_path`: path to the final bike logsum file (bikeMgraLogsum.csv, bikeTazLogsum.csv)
+- `save_bike_net`: whether to write the derived network to a set of CSVs
+- `read_cached_bike_net`: whether to read the derived network from a cache instead of re-deriving from the original network
+- `bike_speed`: the speed in mph to use for calculating bike travel times in output
+
+**Tracing**
+- `trace_bike_utilities`: whether to output the chosen path sets from a specified list of origin-destination pairs
+- `trace_origins`: ordered list of origins whose paths will be output in tracing
+- `trace_destinations`: ordered list of destinations corresponding to the origins for tracing
+- `generate_shapefile`: whether to output the trace paths as a shapefile in addition to CSV
+- `crs`: network coordinate reference system to attach to the output shapefiles
+
+## Utility Calculation
+The variable specifications used in the utility function for edges and traversals
+are stored in `bike_edge_utils.csv` and `bike_traversal_utils.csv`, respectively,
+including the expressions and their associated coefficients. The utility accounts 
+for the distance on different types of facilities, the gain in elevation, turns, 
+signal delay, and navigating un-signalized intersections with high-volume facilities. 
+To account for the correlation in the random utilities of overlapping routes, the
+utility function in the model includes a “path size” measure, described in the 
+following section.
+
+## Path Size
+The size of path alternative $n_i$ in alternative set $\textbf{n}$ is calculated using the
+formula:
+```math
+size(n_i)=\sum_{l\in n_i}\frac{edgeLength(l)}{pathLength(n_i)*pathsUsing(\textbf{n},l)}
+```
+where $pathsUsing$ is the integer number of paths in the alternative set $\textbf{n}$
+which contain link $l$. Its use derives from the theory of aggregate and elemental 
+alternatives, where a link is an aggregation of all paths that use the link. If multiple
+routes overlap, their "size" is less than one. If two routes overlap completely, their
+size will be one-half.
+
+## Path Sampling
+Given an origin, paths are sampled to all relevant destinations in the network by repeatedly
+applying Dijstra’s algorithm to search for the paths that minimize a stochastic link
+generalized cost with a mean given by the additive inverse of the path utility. For each path
+sampling iteration, a random coefficient vector is first sampled from a non-negative
+multivariate uniform distribution with zero covariance and mean equal to the link
+generalized cost coefficients corresponding to the path choice utility function. As the path
+search extends outward from the origin, the random cost coefficients do not vary over links,
+but only over iterations of path sampling. In the path search, the cost of each subsequent
+link is calculated by summing the product of the random coefficients with their respective
+link attributes, and then multiplying the result by a non-negative discrete random edge cost
+multiplier.
+
+## Threshold Utility and Distance
+To improve runtime, the Dijkstra's algorithm implementation allows for early termination once a
+predetermined utility threshold has been reached in its search, with no paths found for zone pairs
+whose utility is beyond the threshold. However, the utility of paths is not necessarily the most
+user-friendly metric - it is reasonable to aim to define the cutoff by distance rather than utility.
+However, because the underlying implementation of Dijkstra's algorithm is only aware of utilities,
+not distances, this is not directly feasible, as utility and distance do not correspond directly.
+
+To address this, the `bike_threshold_calculator` script has been built to provide a handy mechanism
+to search for a utility threshold which approximates a desired distance cutoff. Given a valid 
+configuration YAML file (as described above), the script iteratively performs a binary search,
+using the YAML file's `max_dijkstra_utility` setting as a starting threshold utility and modifying
+its value on subsequent iterations until the target distance is within the allowed margin of error
+(or the maximum number of bike model iterations has been completed). The calling signature of the
+script is shown below and requires a minimum of two command line arguments: the settings filepath
+and the target distance – the remainder of the parameters are optional, with each optional argument 
+requiring those listed before it in sequence:
+
+~~~
+Usage:
+    python bike_threshold_calculator.py <settings filepath> <target distance> [target_margin [percentile [max_iterations]]]
+
+    parameters:
+        settings filepath:  path to YAML file containing bike model settings
+        target distance:    the distance for which the search should aim (in miles)
+        target margin:      the margin of error (< 1) allowed before termination (optional, default: 0.1)
+        percentile:         the percentile of distance to compare against the target (optional, default: 0.99)
+        max iterations:     the most bike model iterations that can be performed in the search (optional, default: 20)
+
+    examples:
+
+        python bike_threshold_calculator.py bike_route_choice_settings_taz.yaml 20 
+        # the resulting 99th %ile distance must be w/in 10% of the 20-mile target distance
+        # equivalent to:
+            python bike_threshold_calculator.py bike_route_choice_settings_taz.yaml 20 0.1 0.99 20
+
+        python bike_threshold_calculator.py bike_route_choice_settings_mgra.yaml 3 0.05
+        # the resulting 99th %ile distance must be w/in 5% of the three-mile target distance
+~~~
+
+With the default parameters used, the script will search until either 20 iterations have elapsed or
+the 99th percentile of distances found is within 10% of the specified target distance. At termination, 
+a CSV named `threshold_results.csv` will be written to the output directory (set in the `output_path` 
+setting in the YAML file) with the input utility thresholds, the iteration runtime, and the chosen 
+percentile of distance. These will also be scatter plotted on a graph and displayed, but note that 
+this graph is not saved automatically to the output directory.

src/asim/scripts/bike_route_choice/bike_edge_utils.csv

@@ -0,0 +1,51 @@
+Label,Description,Expression,Coefficient
+##################################################################################
+# The below coefficients are adapted from Lukawska et al. (2023),,,
+util_distance_general,Edge length (distance) in miles,distance,-10.93
+util_medium_upslope,Edge slope in range [1%-3.5%],"@df.distance * ((df.gain / (df.distance * 5280) >= 0.01) & (df.gain / (df.distance * 5280) <= 0.035)).astype(int)",-9.012
+util_high_upslope,Edge slope > 3.5%,"@df.distance * (df.gain / (df.distance * 5280) > 0.035).astype(int)",-18.19
+# per class penalties - large roadways,,,
+util_lg_protected,Large roadway w/ protected lanes,"@df.distance * ( df.functionalClass.isin([3,4]) & df.cycleTrack ).astype(int)",0.1748
+util_lg_painted,Large roadway w/ painted lanes & no protected lanes,"@df.distance * ( df.functionalClass.isin([3,4]) & (~df.cycleTrack) & (df.bikeClass == 2) ).astype(int)",-3.158
+util_lg_nofacils,Large roadway w/o bike lanes,"@df.distance * ( df.functionalClass.isin([3,4]) & (~df.cycleTrack) & ~(df.bikeClass == 2) ).astype(int)",-2.513
+# per-class penalties - medium roadways,,,
+# util_med_protected,Medium roadway w/ protected lanes,"@df.distance * ( df.functionalClass.isin([5,6]) & df.cycleTrack ).astype(int)",0.000
+util_med_painted,Medium roadway w/ painted lanes & no protected lanes,"@df.distance * ( df.functionalClass.isin([5,6]) & (~df.cycleTrack) & (df.bikeClass == 2) ).astype(int)",-0.5464
+util_med_nofacils,Medium roadway w/o bike lanes,"@df.distance * ( df.functionalClass.isin([5,6]) & (~df.cycleTrack) & ~(df.bikeClass == 2) ).astype(int)",-1.235
+# per class penalties - residential roadways,,,
+util_res_protected,Residential roadway w/ protected lanes,"@df.distance * ( (df.functionalClass == 7) & df.cycleTrack ).astype(int)",-0.9835
+util_res_painted,Residential roadway w/ painted lanes & no protected lanes,"@df.distance * ( (df.functionalClass == 7) & (~df.cycleTrack) & (df.bikeClass == 2) ).astype(int)",0.9288
+util_res_nofacils,Residential roadway w/o bike lanes,"@df.distance * ( (df.functionalClass == 7) & (~df.cycleTrack) & ~(df.bikeClass == 2) ).astype(int)",-1.901
+# per class penalties - non-vehicle routes,,,
+util_cycleway,Cycleway,"@df.distance * ( (~df.functionalClass.isin([3,4,5,6,7,10])) & (df.bikeClass == 2) ).astype(int)",0.4152
+util_sup,Shared-Use Path,"@df.distance * ( (~df.functionalClass.isin([3,4,5,6,7,10])) & (df.bikeClass == 1) ).astype(int)",-1.705
+util_pedzone,Pedestrian Zone,"@df.distance * ( (~df.functionalClass.isin([3,4,5,6,7,10])) & (~df.bikeClass.isin([1,2])) ).astype(int)",-4.021
+# FIXME better way to check wrong way?,,,
+util_wrong_way,Wrong Way - no lanes in direction of travel and no bike lane,"@df.distance * ( (~df.cycleTrack) & (~(df.bikeClass == 2)) & (df.lanes == 0) ).astype(int)",-3.202
+# The above coefficients are adapted from Lukawska et al. (2023),,,
+
+
+##################################################################################
+# The below coefficients are from the ABM2 model and have dubious sourcing (see ABM2 documentation),,,
+# util_class_0_bike_lane_dist,Distance without bike lanes,"@df.distance * np.where((df.bikeClass < 1) | (df.bikeClass > 3), 1, 0)",-0.858
+# util_class_I_bike_lane_dist,Distance on class I bike lanes,"@df.distance * np.where(df.bikeClass == 1, 1, 0)",-0.348
+# util_class_II_bike_lane_dist,Distance on class II bike lanes,"@df.distance * np.where((df.bikeClass == 2) & (~df.cycleTrack), 1, 0)",-0.544
+# util_class_III_bike_lane_dist,Distance on class III bike lanes,"@df.distance * np.where((df.bikeClass == 3) & (~df.bikeBlvd), 1, 0)",-0.858
+# util_art_no_bike_lane,Distance on arterial without bike lanes,"@df.distance * np.where((df.arterial) & (df.bikeClass != 2) & (df.bikeClass != 1), 1, 0)",-1.050
+# util_dist_cycle_track_class_II,Distance on cyle track class II bike lanes,"@df.distance * df.cycleTrack",-0.424
+# util_dist_cycle_track_class_III,Distance on boulevard class III bike routes,"@df.distance * df.bikeBlvd",-0.343
+# FIXME better way to check wrong way distance?,,,
+# util_dist_wrong_way,Distance wrong way -- no lanes in direction of travel and no bike lane means wrong way,"@df.distance * np.where((df.bikeClass != 1) & (df.lanes == 0), 1, 0)",-3.445
+# util_elevation_gain_ft,Cumulative gain in elevation ignoring declines in feet,@df.gain,-0.015
+# util_access_of_highway,"Unallowed access onto interstate, freeway, or expressway","@(df.functionalClass == 1) | (df.functionalClass == 2)",-999.9
+# The above coefficients are from the ABM2 model and have dubious sourcing (see ABM2 documentation),,,
+##################################################################################,,,
+# The below coefficients are from Meister et al. (2024) converted from km to miles,,,
+# util_distance,Total Distance,@df.distance,-0.944
+# util_bike_path,Distance on Bike Path (Class I),"@df.distance * np.where(df.bikeClass == 1, 1, 0)",1.969
+# util_bike_lane,Distance on Bike Lane (Class II),"@df.distance * np.where(df.bikeClass == 2, 1, 0)",1.616
+# util_speed_limit,Distance where speed limit is <= 25 mph,"@df.distance * np.where(df.speedLimit <= 25, 1, 0)",0.081
+# util_low_incline,Distance where slope is > 2% and <= 6%,"@df.distance * np.where((df.gain / (df.distance * 5280) > 0.02) & (df.gain / (df.distance * 5280) <= 0.06), 1, 0)",-1.727
+# util_med_incline,Distance where slope is > 6% and <= 10%,"@df.distance * np.where((df.gain / (df.distance * 5280) > 0.06) & (df.gain / (df.distance * 5280) <= 0.1), 1, 0)",-9.171
+# util_steep_incline,Distance where slope is > 10%,"@df.distance * np.where(df.gain / (df.distance * 5280) > 0.1, 1, 0)",-12.92
+# The above coefficients are from Meister et al. (2024) converted from km to mi,,,