Skip to content

Updates to allow for upgraded vehicle repository#259

Open
kylecarow wants to merge 76 commits into
fastsim-3from
f3/vehicle-repo-changes
Open

Updates to allow for upgraded vehicle repository#259
kylecarow wants to merge 76 commits into
fastsim-3from
f3/vehicle-repo-changes

Conversation

@kylecarow

@kylecarow kylecarow commented Jun 18, 2026

Copy link
Copy Markdown
Collaborator

Resolves #274 and #289

Master issue for all vehicle database work: #288

Based off of #265 / #267, review those first to not get overwhelmed by the huge diff

PR Summary: Vehicle Repository Schema, from_db API, and Related Changes

New: Vehicle Database Schema v1

A new DatabaseSchemaV1 struct (schema_v1.rs) formalizes the directory layout for the fastsim-vehicles repository. It encodes an 8-segment path:

v1/fastsim-{N}/{powertrain}/{make}/{model}/{year}/{variant}/v{N}
Level Description Example
Schema version Describes the format of subsequent organizational levels. v1
FASTSim version Major FASTSim version. fastsim-3
Powertrain Powertrain type (e.g. conv/hev/phev/bev/fcev) hev
Make Vehicle make/manufacturer name. toyota
Model Vehicle model name and any applicable trim information. camry-xle
Year Model year or year range. 2015
Variant Description of active modeling feature set. thermal
Revision Model revision number for updates and corrections. v1

Example full filepath:

v1/fastsim-3/hev/toyota/camry-xle/2015/thermal/v1.yaml

The struct serializes to/from a plain string, so it stores compactly in YAML as a single field.


New: Vehicle.from_db and underlying Rust methods

The high-level Python entry point is Vehicle.from_db(db_path_or_url, schema=1, **kwargs). db_path_or_url is None (default remote), an https://... URL, or a local filesystem path. It supports two input modes:

Mode 1 — Path string (caller supplies the pre-serialized schema path):

Vehicle.from_db(
    db_path_or_url="/path/to/fastsim-vehicles",
    path="v1/fastsim-3/conv/ford/fusion/2012/base/v1",
    # optional: extension="yaml" (default), skip_init=False
)
# Extension can also be embedded directly in the path:
Vehicle.from_db(path="v1/fastsim-3/conv/ford/fusion/2012/base/v1.yaml")
# (cannot specify extension both in path and as a separate kwarg)

Mode 2 — Individual fields (schema path is constructed internally):

Vehicle.from_db(
    db_path_or_url="/path/to/fastsim-vehicles",
    powertrain="conv",       # required: "conv", "hev", "phev", "bev"
    make="Ford",             # required
    model="Fusion",          # required
    year="2012",             # required (string; supports ranges like "2020-2023")
    revision=1,              # required (int or "v1"/"V1" string)
    variant="base",          # optional, default "base"
    fastsim_version=3,       # optional, defaults to installed FASTSim major version
    extension="yaml",        # optional, default "yaml"
    skip_init=False,         # optional
)

The underlying Vehicle.from_db_path_v1 and Vehicle.from_db_fields_v1 are also exposed directly to Python for lower-level use. All methods auto-detect local vs. remote loading.


New Vehicle Metadata Fields

VehicleModel now carries:

  • min_fastsim_version: Version: minimum FASTSim version required to load this file (defaults to current version at serialization time; emits a pre-deserialization warning if the running version is older)
  • db_path: Option<Schema>: optional field recording where in the database this vehicle lives, enabling CI to verify files are stored in the correct location
  • make: Option<String>, model: Option<String> — new optional identity fields
  • year: Option<String>: changed from u32 to Option<String> to support year ranges and missing data

simdrivelabel uses a new parse_model_year_or_2017() helper that falls back to 2017 adjustment coefficients with a stderr warning if year can't be parsed as an integer.


Empty History Fixes

Vehicle history is now skipped when empty (skip_serializing_if = "VehicleStateHistoryVec::is_empty"), reducing file size for vehicles with a save_interval of None. Related Python fixes:

  • to_pydict(flatten=True) and to_dataframe() no longer assert that history is non-empty; both work correctly with no history
  • get_flattened() type signature updated to hist_len: int | None with appropriate guards

Vehicle YAML Files Updated

All f3-vehicles and vehicles files were regenerated to include the new min_fastsim_version, make, model fields and remove stale fields (~70 lines removed per file across ~70 vehicles).


Follow-on Work

  • carve out or rewrite code that automatically generates f3 vehicles from f2 when testing
  • move/remove all but critical vehicle models from:
    • fastsim-core/resources/vehicles
    • cal_and_val/*/f2-vehicles and cal_and_val/*/f3-vehicles
      • this should honestly be completely reorganized, probably into its own repo or fastsim-calibration
    • loose yaml files in fastsim-core/src/vehicles/fastsim-2*.yaml
    • update to have new metadata fields & full current compatibility as needed
    • move all to https://github.com/NatLabRockies/fastsim-vehicles in appropriate schema
  • add more convenience functions for db searching, fuzzy searching, filtering etc.

@kylecarow kylecarow self-assigned this Jun 18, 2026
@kylecarow
kylecarow marked this pull request as draft June 26, 2026 18:39
kylecarow and others added 3 commits June 29, 2026 18:10
Updated Anaconda setup instructions to specify environment name and include pip upgrade.
@kylecarow kylecarow mentioned this pull request Jul 6, 2026
kylecarow and others added 24 commits July 9, 2026 09:20
… started to use plotly and new demo conventions, small tweak to test demos to render plots as JSON (replacement for SHOW_PLOTS)
updated getting started demo with mines demo content, updated getting…
…st using pytest --nbmake (which is set to run by default in pyproject.toml with parallelization - you may need to reinstall dev dependencies before use)
@kylecarow

Copy link
Copy Markdown
Collaborator Author

@robinsteuteville This PR is ready to review! You may want to review the PRs it builds upon #265 & #267 first to not have to look at a huge diff.

Like other PRs with huge diffs I've made recently much of it comes from the cal_and_val folder. After this I plan to carve that out of FASTSim once those vehicles are in the database!

@kylecarow

kylecarow commented Jul 22, 2026

Copy link
Copy Markdown
Collaborator Author

TODO: fix bug with some vehicles of unspecified years having "0" for the year field in serialized yamls Done

@kylecarow kylecarow mentioned this pull request Jul 22, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Skip serializing history if it is empty

1 participant