Skip to content

ENH: Ingest ITKIOMeshSTL into Modules/IO#6206

Merged
hjmjohnson merged 98 commits intoInsightSoftwareConsortium:mainfrom
hjmjohnson:ingest-iomeshstl
May 5, 2026
Merged

ENH: Ingest ITKIOMeshSTL into Modules/IO#6206
hjmjohnson merged 98 commits intoInsightSoftwareConsortium:mainfrom
hjmjohnson:ingest-iomeshstl

Conversation

@hjmjohnson
Copy link
Copy Markdown
Member

Ingests IOMeshSTL from the standalone remote module InsightSoftwareConsortium/ITKIOMeshSTL into ITK at Modules/IO/IOMeshSTL/. Branch is based directly on upstream/main and contains no v4-tooling files in its diff — independently mergeable.

Local validation: pre-commit run --all-files clean; IOMeshSTLTestDriver builds; 10/10 IOMeshSTL tests pass locally (after ExternalData fetches the four .cid content links from the IPFS gateway).

What this provides

IOMeshSTL adds read/write support for the stereolithography (STL) mesh format in both ASCII and binary encodings — the canonical interchange format for 3D-printing, CAD, and surgical-planning workflows.

The flagship classes are itk::STLMeshIO and the STLMeshIOFactory plug-in registration that lets itk::MeshFileReader / itk::MeshFileWriter discover STL automatically.

Path: Modules/IO/IOMeshSTL/ (preserving upstream module name IOMeshSTL; renaming to MeshSTL would break the public CMake find_package(IOMeshSTL) API for downstream consumers).

Pipeline metrics
Metric Value
Upstream commits → rewritten 143 → 89
Upstream merges → preserved 44 → 24 (no linearization)
Blobs scanned in sanitize 149 (cxx=106, py=0, cmake=28, text=9, skip=6)
Blobs reformatted 108 (clang-format / gersemi / trailing-ws / EOF)
Commit subjects auto-prefixed 42
Subjects truncated to 78 chars 3
Blank lines inserted after subject 23
README.rst → README.md rewrites in itk-module.cmake 4
Merges visible in git log -- Modules/IO/IOMeshSTL 5
Tooling provenance

This PR was produced via the v4 ingestion pipeline (PR #6204) but the v4 scripts are intentionally not included in this PR's diff — they were invoked from a separate worktree. This makes the PR mergeable independently of #6204.

Phase B (upstream archival) is deferred. After this PR merges:

  1. Wait for PR WIP: Add v4 remote-module ingestion (two-phase, per-commit hooks) #6204 (v4 tooling) to land on main.
  2. Run Utilities/Maintenance/RemoteModuleIngest/archive-remote-module.sh IOMeshSTL --itk-pr-number <THIS-PR> to push the deletion + MIGRATION_README.md → README.md promotion to upstream and flip archived=true.

This PR makes no upstream changes; the upstream ITKIOMeshSTL repository remains live until the operator manually runs Phase B.

Follow-up commits in this PR

After the merge commit (ENH: Ingest ITKIOMeshSTL into Modules/IO), the standard cleanup commits land on top:

  • STYLE: Apply gersemi 0.19.3 to Modules/IO/IOMeshSTL/src/CMakeLists.txt. The v4 sanitize pass uses gersemi 0.24.0; ITK's pre-commit pins 0.19.3. The post-merge pre-commit run --all-files gate caught one CMake file where the two versions disagree.
  • DOC: Add Modules/IO/IOMeshSTL/README.md pointing at the archived upstream.
  • COMP: Remove Modules/Remote/IOMeshSTL.remote.cmake (now in-tree).
  • ENH: Enable Module_IOMeshSTL:BOOL=ON in pyproject.toml configure-ci.

Note: no manual itk-module.cmake patch was needed in this PR — the v4 sanitize pass auto-rewrites file(READ "${...}/README.rst" DOCUMENTATION) to read README.md instead, across every historical commit.

Luis Ibanez and others added 30 commits December 30, 2013 12:00
Added internal copy of points in WritePoints(). Added computation
of normal and print out of point coordinates in WriteCells().

Produced first successful write out of the sphere.stl file, and
was able to open it with meshlab.
Introduced a templated function to manage the multiple cases of
point coordinate types.
Still assuming that points are provided in a right handed order.
The .STL (in capitals) extension is now supported too in the filename.
And corresponding test.
Parsed input file both in ASCII and BINARY,
stored points into an std::set.
Added an std::vector to hold, for each cell, the triplet of point Ids
corresponding to the vertices of that particular triangle. Related the
pointIds to positions in the std::set that de-duplicates points.
Now package the information about point coordinates into a buffer
that is passed back to the MeshFileReader.

Adjusted the helper data structures that hold the pointIds and the
iterators to unique points in the a point set.
Now recomposing the cells information in the buffer that goes back
to the MeshFileReader.

Still have debugging issues with reading enough cells, and confounding
the input points.
After adding a tetrahedron test, debugged the process of reading and
consolidating incoming vertices. Fixed the reading of STL files.
Replaced previous hash, with a more consistent compare method.
When transferring points from the Point Map to the points buffer,
the points must be stored in the location indicated by the second
element of the map pairs. Those are the ordered locations of the
points, to which the cell point ids are referring.

Now manage to read and write the sphere.stl file.
It was off due to the initial read of the header used to determine
whether the input file is ASCII or BINARY.
Added these two classes to the STLMeshIO group.
The STL format assumes little endianness in the files:

    http://en.wikipedia.org/wiki/STL_(file_format)#Binary_STL

Here we introduce conversions between System endianness, and little
endianness when reading and writing in BINARY format.
At the end of the test, we now print out the Endianness of the system.
This should be helpful when verifying STL files in BINARY format.
Suggested in code review by Arnaud Gelas.
Splitted the WriteCells() method into BINARY and ASCII cases,
as suggested by Arnaud Gelas in code review.

It makes easier to follow the logic of each individual case.
To make it consistent in all the CMake infrastructure.
That before were used to verify that all cells were triangles.
To address compilation errors from MacOSX-gcc4.2-Release-IOSTL.
As a Remote Module, EXCLUDE_FROM_DEFAULT needs to be added from the module
being inadvertently turned on in a fresh build after it was downloaded in a
previous build.
"PointValueType" was both the template parmeter of WritePointsTyped() and
a typedef of the enclosing class. This was generating errors with GCC on
OSX. Renamed the template parameter to "TPointsBuffer" to fix this error.

Also added a static_cast to make the conversion to the internal value type
explicit.

Also made the parameter for WritePointsTyped() a const pointer to const
values, to clarify its intended use as an input buffer.
BUG: Add EXCLUDE_FROM_DEFAULT.
BUG: Fix WritePointsTyped() to no longer use conflicting typenames
@dzenanz
Copy link
Copy Markdown
Member

dzenanz commented May 4, 2026

in Modules\IO\IOMeshSTL\test\Baseline, there are some weirdly named files (e.g. .ExternalData_MD5_4cd09d26436a53a5adedb33d822cabcd). They should be removed or converted into .cids.

@dzenanz
Copy link
Copy Markdown
Member

dzenanz commented May 4, 2026

In Modules\IO\IOMeshSTL\CMakeLists.txt, we don't need if(NOT ITK_SOURCE_DIR) block.

hjmjohnson added a commit to hjmjohnson/ITK that referenced this pull request May 4, 2026
Five concerns from Greptile's draft review of PR InsightSoftwareConsortium#6206:

P1: STL binary spec is UINT32, but read/write paths declared int32_t.
The high-bit case decremented through negative values (signed-integer
overflow, UB) and SetNumberOfCells silently wrapped to huge unsigned
IdentifierType. Switched both call sites to uint32_t and added
Read/WriteUInt32AsBinary helpers (int32 helpers kept for source compat).

P2: Three unreachable returns after itkExceptionMacro removed.

P2: WriteMeshInformation 'Unable to open file' said 'inputFilename='
while opening the OUTPUT stream. Corrected to 'outputFilename='.

P2: Test reader->Update() now wrapped in ITK_TRY_EXPECT_NO_EXCEPTION
to match writer->Update() and the rest of ITK's tests.

P2 (greptile false positive): STLMeshIO::Read() was suggested for an
override specifier. MeshIOBase has no virtual Read() to override, so
'virtual void Read();' is correct as-is. Reverted.
hjmjohnson added a commit to hjmjohnson/ITK that referenced this pull request May 5, 2026
…terns

Add a 'Known artifacts at PR-review time' section to capture two
things future ingest operators need to expect but that v3/v4 do not
fix automatically:

  - The single ghostflow-check-main 'root commit not allowed' error
    that every Mode-A merge produces.  Maintainers override at merge.
    Calling it out keeps operators from chasing a non-fix.

  - A short table of code-level patterns Greptile has flagged
    post-ingest on multiple modules (IOMeshSTL InsightSoftwareConsortium#6206, RLEImage InsightSoftwareConsortium#6208):
    signed-vs-unsigned size types, missing override, dead return after
    itkExceptionMacro, stray test args, external-friend declarations,
    GetBuffer-const correctness, and allocator-init invariants.

The table is the durable channel for cross-ingest review knowledge —
extend it as new recurring patterns surface.
hjmjohnson and others added 8 commits May 5, 2026 05:58
Brings IOMeshSTL from a configure-time remote fetch into the ITK
source tree at Modules/IO/IOMeshSTL/ using the v4 ingestion
pipeline (whitelist filter-repo + per-commit clang-format + black +
commit-prefix sanitization).

Upstream repo:  https://github.com/InsightSoftwareConsortium/ITKIOMeshSTL.git
Upstream tip:   fb18e5d83415603d977279e0546bb55c1ece623d
Ingest date:    2026-05-04
Whitelist:      IOMeshSTL.list

Per-commit transforms applied across all 89 commits:
  - filter-repo --paths-from-file (whitelist)
  - filter-repo --to-subdirectory-filter Modules/IO/IOMeshSTL
  - clang-format -style=file (ITK main's .clang-format) for *.cxx/.h/.hxx/...
  - black for *.py
  - heuristic ITK prefix added to commit subjects without one

Merge topology preserved: 44 -> 24 merge(s).

Primary author: Luis Ibanez <luis.ibanez@kitware.com>

Co-authored-by: Bradley Lowekamp <blowekamp@mail.nih.gov>
Co-authored-by: Brian Helba <brian.helba@kitware.com>
Co-authored-by: Davis Vigneault <davis.vigneault@gmail.com>
Co-authored-by: Dženan Zukić <dzenan.zukic@kitware.com>
Co-authored-by: Hans J. Johnson <hans-johnson@uiowa.edu>
Co-authored-by: Hans Johnson <hans-johnson@uiowa.edu>
Co-authored-by: Hans Johnson <hans.j.johnson@gmail.com>
Co-authored-by: Jon Haitz Legarreta <jhlegarreta@vicomtech.org>
Co-authored-by: Jon Haitz Legarreta Gorroño <jon.haitz.legarreta@gmail.com>
Co-authored-by: Mathew Seng <mathewseng@gmail.com>
Co-authored-by: Matt McCormick <matt.mccormick@kitware.com>
Co-authored-by: Matt McCormick <matt@mmmccormick.com>
Co-authored-by: Tom Birdsong <tom.birdsong@kitware.com>
Co-authored-by: Zach Williamson <zachary-williamson@uiowa.edu>
Five concerns from Greptile's draft review of PR InsightSoftwareConsortium#6206:

P1: STL binary spec is UINT32, but read/write paths declared int32_t.
The high-bit case decremented through negative values (signed-integer
overflow, UB) and SetNumberOfCells silently wrapped to huge unsigned
IdentifierType. Switched both call sites to uint32_t and added
Read/WriteUInt32AsBinary helpers (int32 helpers kept for source compat).

P2: Three unreachable returns after itkExceptionMacro removed.

P2: WriteMeshInformation 'Unable to open file' said 'inputFilename='
while opening the OUTPUT stream. Corrected to 'outputFilename='.

P2: Test reader->Update() now wrapped in ITK_TRY_EXPECT_NO_EXCEPTION
to match writer->Update() and the rest of ITK's tests.

P2 (greptile false positive): STLMeshIO::Read() was suggested for an
override specifier. MeshIOBase has no virtual Read() to override, so
'virtual void Read();' is correct as-is. Reverted.
In-tree, ITK_SOURCE_DIR is always defined, so the
'if(NOT ITK_SOURCE_DIR) ... find_package(ITK) ... else() itk_module_impl()'
guard is dead code.  Address @dzenanz review comment.
Four .ExternalData_MD5_<hash> files survived the ingest from the
upstream module's local fetch cache.  They are not source — ExternalData
re-creates them on demand alongside the .cid sidecars they correspond
to (sphere.{stl,vtk}, tetrahedron.{stl,vtk}).  Address @dzenanz review
comment.
@hjmjohnson
Copy link
Copy Markdown
Member Author

Both points addressed in the latest push:

  • Standalone-build wrapper: dropped in 93985ca. Modules/IO/IOMeshSTL/CMakeLists.txt is now just itk_module_impl().
  • .ExternalData_MD5_<hash> files: removed in f100c25. They were leftover ExternalData fetch cache from the upstream module — not source. ExternalData regenerates them on demand from the .cid sidecars (sphere.{stl,vtk}, tetrahedron.{stl,vtk}).

Branch was also rebased onto the latest upstream/main (was 127 commits behind) using --rebase-merges to preserve Mode-A merge topology (25 merges, 1 root commit, 95 PR commits — all unchanged from pre-rebase). pre-commit run --all-files clean. Thanks @dzenanz.

@hjmjohnson hjmjohnson requested a review from dzenanz May 5, 2026 11:08
hjmjohnson added a commit to hjmjohnson/ITK that referenced this pull request May 5, 2026
ExternalData stores resolved fetched content as .ExternalData_<algo>_<hash>
files alongside the .cid / .md5 sidecars they correspond to.  These
are local fetch cache, not source — ExternalData regenerates them on
demand at consumer-time.  Upstream remote modules sometimes commit
them inadvertently from a CTest run inside their working tree (e.g.
ITKIOMeshSTL had four such files in test/Baseline/, flagged by
@dzenanz on PR InsightSoftwareConsortium#6206).

Add **/.ExternalData_* to the scaffolding deny-pass so they never
enter ITK history.
hjmjohnson added a commit to hjmjohnson/ITK that referenced this pull request May 5, 2026
Upstream remote modules wrap their top-level CMakeLists.txt in

    if(NOT ITK_SOURCE_DIR)
      find_package(ITK REQUIRED)
      list(APPEND CMAKE_MODULE_PATH ${ITK_CMAKE_DIR})
      include(ITKModuleExternal)
    else()
      itk_module_impl()
    endif()

so the same file works both as a fetched remote module and as a
stand-alone CMake project.  In an in-tree ingested module
ITK_SOURCE_DIR is always defined, so the if-branch is dead code;
@dzenanz flagged it on PR InsightSoftwareConsortium#6206 (IOMeshSTL).

Add patch_standalone_build_guard() that detects the idiom (allowing
2- or 4-space indent variants) and replaces it with a bare
itk_module_impl() before gersemi formatting runs.  Track count via
sanitizer.standalone_guard_patches and print it in the run summary.
@dzenanz
Copy link
Copy Markdown
Member

dzenanz commented May 5, 2026

/azp run ITK.macOS.Python

Copy link
Copy Markdown
Member

@dzenanz dzenanz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good on a glance.

….txt

In-tree, ITK's top-level CMakeLists pins the CMake minimum, so the
per-module declaration is dead code.  Pre-emptive cleanup matching the
v4 ingest-pipeline rule codified in PR InsightSoftwareConsortium#6204
(sanitize-history.py:patch_drop_cmake_minimum_required).
@hjmjohnson hjmjohnson added this to the ITK 6.0 Release Candidate 1 milestone May 5, 2026
@hjmjohnson hjmjohnson merged commit 8c008f2 into InsightSoftwareConsortium:main May 5, 2026
14 of 17 checks passed
hjmjohnson added a commit to hjmjohnson/ITK that referenced this pull request May 6, 2026
Five concerns from Greptile's draft review of PR InsightSoftwareConsortium#6206:

P1: STL binary spec is UINT32, but read/write paths declared int32_t.
The high-bit case decremented through negative values (signed-integer
overflow, UB) and SetNumberOfCells silently wrapped to huge unsigned
IdentifierType. Switched both call sites to uint32_t and added
Read/WriteUInt32AsBinary helpers (int32 helpers kept for source compat).

P2: Three unreachable returns after itkExceptionMacro removed.

P2: WriteMeshInformation 'Unable to open file' said 'inputFilename='
while opening the OUTPUT stream. Corrected to 'outputFilename='.

P2: Test reader->Update() now wrapped in ITK_TRY_EXPECT_NO_EXCEPTION
to match writer->Update() and the rest of ITK's tests.

P2 (greptile false positive): STLMeshIO::Read() was suggested for an
override specifier. MeshIOBase has no virtual Read() to override, so
'virtual void Read();' is correct as-is. Reverted.
hjmjohnson added a commit to hjmjohnson/ITK that referenced this pull request May 6, 2026
…est-iomeshstl

ENH: Ingest ITKIOMeshSTL into Modules/IO
hjmjohnson added a commit to hjmjohnson/ITK that referenced this pull request May 7, 2026
…terns

Add a 'Known artifacts at PR-review time' section to capture two
things future ingest operators need to expect but that v3/v4 do not
fix automatically:

  - The single ghostflow-check-main 'root commit not allowed' error
    that every Mode-A merge produces.  Maintainers override at merge.
    Calling it out keeps operators from chasing a non-fix.

  - A short table of code-level patterns Greptile has flagged
    post-ingest on multiple modules (IOMeshSTL InsightSoftwareConsortium#6206, RLEImage InsightSoftwareConsortium#6208):
    signed-vs-unsigned size types, missing override, dead return after
    itkExceptionMacro, stray test args, external-friend declarations,
    GetBuffer-const correctness, and allocator-init invariants.

The table is the durable channel for cross-ingest review knowledge —
extend it as new recurring patterns surface.
hjmjohnson added a commit to hjmjohnson/ITK that referenced this pull request May 7, 2026
ExternalData stores resolved fetched content as .ExternalData_<algo>_<hash>
files alongside the .cid / .md5 sidecars they correspond to.  These
are local fetch cache, not source — ExternalData regenerates them on
demand at consumer-time.  Upstream remote modules sometimes commit
them inadvertently from a CTest run inside their working tree (e.g.
ITKIOMeshSTL had four such files in test/Baseline/, flagged by
@dzenanz on PR InsightSoftwareConsortium#6206).

Add **/.ExternalData_* to the scaffolding deny-pass so they never
enter ITK history.
hjmjohnson added a commit to hjmjohnson/ITK that referenced this pull request May 7, 2026
Upstream remote modules wrap their top-level CMakeLists.txt in

    if(NOT ITK_SOURCE_DIR)
      find_package(ITK REQUIRED)
      list(APPEND CMAKE_MODULE_PATH ${ITK_CMAKE_DIR})
      include(ITKModuleExternal)
    else()
      itk_module_impl()
    endif()

so the same file works both as a fetched remote module and as a
stand-alone CMake project.  In an in-tree ingested module
ITK_SOURCE_DIR is always defined, so the if-branch is dead code;
@dzenanz flagged it on PR InsightSoftwareConsortium#6206 (IOMeshSTL).

Add patch_standalone_build_guard() that detects the idiom (allowing
2- or 4-space indent variants) and replaces it with a bare
itk_module_impl() before gersemi formatting runs.  Track count via
sanitizer.standalone_guard_patches and print it in the run summary.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

area:IO Issues affecting the IO module area:Python wrapping Python bindings for a class area:Remotes Issues affecting the Remote module type:Enhancement Improvement of existing methods or implementation type:Infrastructure Infrastructure/ecosystem related changes, such as CMake or buildbots type:Testing Ensure that the purpose of a class is met/the results on a wide set of test cases are correct

Projects

None yet

Development

Successfully merging this pull request may close these issues.

10 participants