Skip to content

Eval mode 2 does part-wise Rel-L2 averaging, not joint full-mesh Rel-L2 #5

Description

@vpuri3

Observation

Eval mode 2 (--eval 2: decoupled inference / physical state caching + full mesh decoding) does not appear to do joint prediction + Rel-L2 over all cells in a mesh. It does piecewise predictions over parts, then averages per-part Rel-L2s.

That conflicts with Appendix A.3 Eq. 7 (Rel-L2 over the full mesh of (N) cells).

Evidence

Test-mode dataloader yields one part at a time, not the entire mesh:

https://github.com/thuml/Transolver-3/blob/main/dataset/dataset_drivaerml_surface_numpy_chunk.py

  • train=False: length is #runs × #parts
  • __getitem__ returns a single stride-part

train_surface.test and test_decoupled_inference Stage 2 then compute Rel-L2 on that part and np.mean over batches:

https://github.com/thuml/Transolver-3/blob/main/train_surface.py

So reported L2RE is mean_parts(RelL2(part)), not joint RelL2(entire mesh).

Mode 2 Stage 1 builds a global cache, but Stage 2 still decodes/evaluates part-by-part with that averaging.

Question

Were published surface Rel-L2 numbers computed this way, or via true full-mesh accumulation? If Eq. 7 is intended, eval should sum error/target energies across parts (per run) and take one ratio.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions