Add deployment/: ONNX export, C++ inference library, Jetson/DGX-Spark… by aliejabbari · Pull Request #10 · davnords/LoMa

aliejabbari · 2026-06-22T23:05:19Z

… deployment + benchmarks

A self-contained deployment layer under deployment/, without touching the PyTorch code:

ONNX export of all variants (B, B128, R, L, G) — detector + descriptor + matcher, each a single self-contained .onnx, validated vs PyTorch (matchers 100% / 99.95% index agreement; end-to-end 99.5% of matches within 1 px; descriptor cosine 1.0).
Reusable C++ inference library (ONNX Runtime CUDA->CPU fallback, OpenCV, 3-line API, find_package(LoMa)) with example + benchmark apps.
Resolution/keypoint presets (pico/nano/turbo/fast/balanced/wide/quality) tuned for the Jetson Orin Nano 8GB; landscape preset support.
From-source onnxruntime-gpu build for the DGX Spark GB10 (sm_121, CUDA 13, no prebuilt wheel); measured on-device Jetson Orin Nano GPU benchmark (CUDA EP).
Benchmark + charts + RANSAC-inlier analysis (GB10 and Orin Nano).

Uses the dynamo (torch.export) ONNX exporter; no changes to src/loma/.

… deployment + benchmarks A self-contained deployment layer under deployment/, without touching the PyTorch code: - ONNX export of all variants (B, B128, R, L, G) — detector + descriptor + matcher, each a single self-contained .onnx, validated vs PyTorch (matchers 100% / 99.95% index agreement; end-to-end 99.5% of matches within 1 px; descriptor cosine 1.0). - Reusable C++ inference library (ONNX Runtime CUDA->CPU fallback, OpenCV, 3-line API, find_package(LoMa)) with example + benchmark apps. - Resolution/keypoint presets (pico/nano/turbo/fast/balanced/wide/quality) tuned for the Jetson Orin Nano 8GB; landscape preset support. - From-source onnxruntime-gpu build for the DGX Spark GB10 (sm_121, CUDA 13, no prebuilt wheel); measured on-device Jetson Orin Nano GPU benchmark (CUDA EP). - Benchmark + charts + RANSAC-inlier analysis (GB10 and Orin Nano). Uses the dynamo (torch.export) ONNX exporter; no changes to src/loma/.

Copilot

Pull request overview

This PR introduces a self-contained deployment/ layer that exports LoMa model components to ONNX, provides Python benchmarks/validation tooling, and adds a reusable C++ inference library (ONNX Runtime + OpenCV) targeting Jetson Orin Nano and DGX Spark / GB10—without modifying the core src/loma/ PyTorch implementation.

Changes:

Add ONNX export + validation scripts for detector/descriptor/matcher components, plus benchmark suites and chart rendering.
Add Jetson-focused preset exports and an on-device Jetson benchmark runner + comparison charts/data.
Add a C++ inference library (deployment/cpp/) with examples and CMake install/find_package support.

Reviewed changes

Copilot reviewed 23 out of 34 changed files in this pull request and generated 8 comments.

Show a summary per file

File	Description
deployment/viz_matches.py	Generates a qualitative match visualization using the ONNX pipeline.
deployment/requirements.txt	Lists Python dependencies for export/benchmark tooling.
deployment/README.md	Documents ONNX model zoo, benchmarks, Jetson/DGX setup, and usage.
deployment/jetson_charts.py	Renders Jetson benchmark charts and GB10-vs-Orin comparison plots.
deployment/jetson_bench.py	On-device benchmark runner for Jetson using onnxruntime + numpy + PIL.
deployment/export_onnx.py	Exports LoMa components to ONNX and validates ORT vs PyTorch.
deployment/export_jetson.py	Exports Jetson-tuned resolution/keypoint presets (incl. wide/landscape).
deployment/docs/jetson_benchmark.json	Jetson Orin Nano benchmark results (used for README/charts).
deployment/docs/benchmark.json	GB10 benchmark results (used for README/charts).
deployment/cpp/src/loma.cpp	C++ implementation of the detector/descriptor/matcher pipeline via ORT.
deployment/cpp/README.md	Build/usage docs for the C++ library and platform notes.
deployment/cpp/include/loma/loma.hpp	Public C++ API (pimpl) for feature extraction and matching.
deployment/cpp/examples/match_example.cpp	Minimal example to run matching and draw correspondences.
deployment/cpp/examples/benchmark.cpp	Benchmark example reporting latency statistics and throughput.
deployment/cpp/CMakeLists.txt	CMake build/install + exported targets for `find_package(LoMa)`.
deployment/cpp/cmake/LoMaConfig.cmake.in	Package config template for exported CMake targets.
deployment/compare_onnx.py	Per-stage + end-to-end ONNX-vs-PyTorch comparison script.
deployment/build_ort_gpu.sh	Script to build onnxruntime-gpu from source for GB10 (aarch64/CUDA 13).
deployment/benchmark_onnx.py	CPU ORT benchmark + quality checks vs PyTorch per preset.
deployment/benchmark_onnx_gpu.py	GPU ORT benchmark/validation via CUDA EP per preset.
deployment/benchmark_gpu.py	PyTorch CUDA benchmark across presets (GPU reference).
deployment/bench_sweep.py	CUDA EP sweep benchmark + JSON report + publication-style charts.
deployment/.gitignore	Ignores exported ONNX artifacts, build outputs, and ort build dir.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

+REPO=/home/raman/Basir/LoMa
+SRC="$REPO/.ortbuild/onnxruntime"
+VENV_PY="$REPO/.venv/bin/python"
+ORT_VER="v1.24.4"
+CUDA_HOME="/usr/local/cuda"
+CUDNN_HOME="/usr"


+echo "[ort] installing wheel into venv (replaces CPU onnxruntime) ..."
+"$REPO/.venv/bin/pip" install --force-reinstall --no-deps "$WHEEL"
+


+def export_detector(model, outdir, opset, n, im_b, dynamic):
+    log("== detector (DaD, shared) ==")
+    img = sample_image(model, im_b)
+    path = os.path.join(outdir, "loma_detector.onnx")
+    # Detector exports STATIC: dynamic batch hits `if B == 0` in get_normalized_grid,
+    # and dynamic H/W hits data-dependent guards. (Re-export per input size if needed.)
+    dynamic_shapes = None


+        rows.append(dict(preset=p, H=H, W=W, desc=D, kpts=K,
+                         det_ms=round(t_det, 2), dsc_ms=round(t_dsc, 2),
+                         mat_ms=round(t_mat, 2), total_ms=round(total, 1),
+                         fps=round(fps, 2), matches=nmatch))


+    ap = argparse.ArgumentParser()
+    ap.add_argument("--onnx", default="onnx")
+    ap.add_argument("--imgA", default="0015_A.jpg")
+    ap.add_argument("--imgB", default="0015_B.jpg")
+    ap.add_argument("--presets", nargs="+",
+                    default=["pico", "nano", "turbo", "fast"])
+    ap.add_argument("--iters", type=int, default=20)
+    ap.add_argument("--warmup", type=int, default=8)
+    ap.add_argument("--out", default="jetson_bench.json")
+    args = ap.parse_args()


+add_library(loma src/loma.cpp)
+add_library(LoMa::loma ALIAS loma)
+target_include_directories(loma PUBLIC
+  $<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}/include>
+  $<INSTALL_INTERFACE:include>)
+target_include_directories(loma PRIVATE ${ONNXRUNTIME_INCLUDE_DIR})
+target_link_libraries(loma PUBLIC ${OpenCV_LIBS} PRIVATE ${ONNXRUNTIME_LIB})
+set_target_properties(loma PROPERTIES VERSION ${PROJECT_VERSION} SOVERSION 1)


+    int ki = outIndex(detector, "keypoints");
+    int pi = outIndex(detector, "keypoint_probs");
+    const float* kp = det_out[ki].GetTensorData<float>();
+    const float* pr = det_out[pi].GetTensorData<float>();
+    f.n = N;
+    f.kpts.assign(kp, kp + static_cast<size_t>(N) * 2);
+    f.probs.assign(pr, pr + N);


+  std::string dir = argv[1], pa = argv[2], pb = argv[3];
+  std::string preset = argc > 4 ? argv[4] : "fast";
+  int iters = argc > 5 ? std::stoi(argv[5]) : 50;
+  const Preset p = kPresets.at(preset);
+


davnords · 2026-06-24T12:57:12Z

@aliejabbari this looks very promising!

Thank you for your effort. Will look at this in detail shortly.

Does this allow for easy integration into colmap repo?

/ David

davnords · 2026-06-24T12:59:56Z

@aliejabbari Could you add something in the README about this? Looks like many people could be interested in the jetson export.

…+ README - build_ort_gpu.sh: derive REPO from script location, honor env overrides (REPO/SRC/VENV_PY/CUDA_HOME/CUDNN_HOME); install wheel via "$VENV_PY" -m pip. - export_onnx.py: fail fast on --detector-dynamic (detector export is static). - jetson_bench.py: emit rows/metadata matching jetson_charts.py + docs schema (name, detect_ms/describe_ms ×2, match_ms; device/variant/pair); default --out to docs/jetson_benchmark.json so a default run is directly chartable. - cpp/CMakeLists.txt: link ONNX Runtime PUBLIC so consumers resolve its symbols. - cpp/src/loma.cpp: validate detector output shapes before copying to avoid reading past the ORT buffer when Options.num_keypoints mismatches the model. - cpp/examples/benchmark.cpp: validate preset before kPresets.at() lookup. - README.md: add Deployment (ONNX/C++/Jetson) section + checklist entry. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

aliejabbari · 2026-06-27T07:17:48Z

Thanks — addressed all 8 in 8905fa3:
1-2. build_ort_gpu.sh now derives REPO from the script location with env overrides, installs via $VENV_PY -m pip (no more hardcoded /home/raman path).
3. export_onnx.py raises NotImplementedError on --detector-dynamic instead of silently ignoring it.
4-5. jetson_bench.py emits rows/metadata matching jetson_charts.py + docs/jetson_benchmark.json (name, detect_ms/describe_ms ×2, match_ms, device/variant/pair); --out defaults to docs/jetson_benchmark.json so a default run is directly chartable.
6. CMake links ONNX Runtime PUBLIC.
7. loma.cpp validates detector output shapes before copying.
8. benchmark.cpp validates the preset before lookup.

aliejabbari · 2026-06-27T07:18:09Z

Added a Deployment section to the top-level README pointing at deployment/ (ONNX export + C++ + Jetson/DGX-Spark), plus a checklist entry.

On COLMAP: yes — the C++ API is split into extract() (per-image keypoints+descriptors) and matchFeatures() (cross-image), which is exactly the shape COLMAP's custom-feature / custom-match workflow wants. The easiest route is the existing HLoc fork, which already writes keypoints and matches into a COLMAP database; LoMa's matchFeatures() output (index pairs + scores) maps directly to COLMAP's two-view geometry / matches_importer path, skipping NN matching since LoMa is itself the matcher. I'm happy to add a small cpp/examples/colmap_import example that writes keypoints + matches straight into a COLMAP DB if that'd be useful.

aliejabbari · 2026-06-27T07:27:26Z

Went ahead and added it: cpp/examples/colmap_import.cpp (5db338e). It extracts LoMa keypoints per image, matches every pair, and writes cameras/images/keypoints + raw matches straight into a COLMAP SQLite DB, plus a raw match list for geometric verification:

./loma_colmap_import onnx scene.db img1.jpg img2.jpg img3.jpg --preset fast
colmap matches_importer --database_path scene.db --match_list_path scene.db.matches.txt --match_type raw
colmap mapper --database_path scene.db --image_path imgs --output_path sparse

Keypoint indices are recovered from the Match pixel points (replaying loma::to_pixel), so the matches table references the stored keypoints correctly across >2 images. Since LoMa is the matcher, this uses COLMAP's custom-keypoints + imported-matches path (no SIFT/NN). The target is gated on SQLite3 in CMake (libsqlite3-dev) so the rest of cpp/ still builds without it. Compile-checked the examples against the header here (full link needs ONNX Runtime); happy to adjust the camera model / coordinate convention if you have a preference.

…atches into a COLMAP DB Extracts keypoints per image and matches every pair with LoMa, then writes cameras, images, keypoints and raw matches directly into a COLMAP SQLite database (+ a raw match list for `colmap matches_importer`). Keypoint indices are recovered from Match pixel points by replaying loma::to_pixel, so the matches table references the stored keypoints correctly across >2 images. LoMa is the matcher, so this uses COLMAP's custom-keypoints + imported-matches path (no SIFT/NN). Target is gated on SQLite3 in CMake (libsqlite3-dev); the rest of cpp/ builds without it. README documents the full COLMAP flow. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

aliejabbari · 2026-06-27T07:45:46Z

Update — the COLMAP importer is in this PR now (commit 5db338e), no separate PR needed since it builds on deployment/cpp/.

cpp/examples/colmap_import.cpp — LoMa → COLMAP SfM:

extracts LoMa keypoints per image and matches every image pair
writes cameras, images, keypoints + raw matches straight into a COLMAP SQLite DB (plus a <db>.matches.txt raw match list)
recovers per-image keypoint indices from the Match pixel points (replaying loma::to_pixel), so the matches table stays consistent across >2 images
COLMAP custom-keypoints + imported-matches path (LoMa is the matcher, so no SIFT/NN)
CMake target gated on SQLite3 (libsqlite3-dev); the rest of cpp/ builds without it

./loma_colmap_import onnx scene.db img1.jpg img2.jpg img3.jpg --preset fast
colmap matches_importer --database_path scene.db --match_list_path scene.db.matches.txt --match_type raw
colmap mapper --database_path scene.db --image_path imgs --output_path sparse

deployment README has a COLMAP section. Examples compile-checked against the header here (full link needs ONNX Runtime installed). Happy to tweak the camera model or coordinate convention if you'd prefer.

aliejabbari · 2026-07-05T22:53:54Z

Hi @davnords,

Hope you're doing well! Just checking in to see if you've had a chance to review the latest updates. Everything requested, including the COLMAP example, is now ready.

I see a minor merge conflict in README.md which I can resolve right away. Let me know how you'd like to proceed. Thanks!

davnords · 2026-07-05T22:56:12Z

Hi @aliejabbari! Thanks for the great effort. Did not see your response until I was tagged.

I will look at this tomorrow. Looks very promising, I think colmap is very valuable

davnords · 2026-07-05T22:59:22Z

I will also look at using this to make a PR to official colmap repo, would be very valuable. Is this something you are interested in? I talked to Johannes at CVPR and he seemed positive about merging LoMa into main colmap

aliejabbari · 2026-07-05T23:07:36Z

Hi @davnords,

Thanks for getting back to me! That’s great news about COLMAP. I'd absolutely love to collaborate on getting LoMa into the main COLMAP repo. Keep me posted on how you'd like to proceed with that.

davnords · 2026-07-05T23:09:56Z

Great! I will think about how to best integrate to main colmap repo and hopefully get back to you tomorrow,

/ David

davnords · 2026-07-05T23:10:06Z

Thanks again for your work!

davnords · 2026-07-07T09:22:50Z

Sorry had some problems yesterday, will try to look at it today :)

aliejabbari · 2026-07-08T09:04:59Z

No worries at all, David — take your time, no rush! Excited to hear about the COLMAP conversation with Johannes. Let me know whenever you get a chance to look :)

davnords · 2026-07-09T16:00:05Z

Hi @aliejabbari. I finally had time to look through the code and it looks very compelling. Good job.

I took your ONNX export code (with some minor modifications to make things more dynamic) and made a PR for integration into the official colmap repo. Big thanks for this and please tell me if I can accredit you in any further way than the mention down in the PR (colmap/colmap#4524).

As for merging, I will discuss briefly with my co-authors and if all goes smooth will go ahead and merge with some small edits (e.g. README.md).

/ David

aliejabbari · 2026-07-09T17:29:53Z

Thanks a lot, @davnords, really glad it was useful and nice to hear it held up on review. Congrats on getting it in front of the COLMAP team.
You asked how you could accredit further, so one small thing if it's easy. Would you be open to adding a Co-authored-by trailer on the export commit? That way the credit stays with the git history once it's upstream. No worries at all if it complicates the merge, the mention in the PR already means a lot.
The InLoc results really speak for themselves too. That DUC2 gap over LightGlue is a strong argument for the whole pipeline.
I'll be following the review and happy to jump in if anything on the ONNX side needs fixing. Looking forward to seeing it merged.

davnords · 2026-07-09T17:44:40Z

@aliejabbari could you send me your email so i can add you as co-author to the commit?

aliejabbari · 2026-07-09T17:59:48Z

Of course, thanks again, David! My email is alijabbari.contact@gmail.com.

davnords · 2026-07-09T18:16:20Z

Done: colmap/colmap@e6761b2 :)

aliejabbari · 2026-07-10T20:03:33Z

Awesome, thanks a lot! 🙏:)

Copilot AI review requested due to automatic review settings June 22, 2026 23:05

Copilot started reviewing on behalf of aliejabbari June 22, 2026 23:05 View session

Copilot AI reviewed Jun 22, 2026

View reviewed changes

		echo "[ort] installing wheel into venv (replaces CPU onnxruntime) ..."
		"$REPO/.venv/bin/pip" install --force-reinstall --no-deps "$WHEEL"

Conversation

aliejabbari commented Jun 22, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

davnords commented Jun 24, 2026

Uh oh!

davnords commented Jun 24, 2026

Uh oh!

aliejabbari commented Jun 27, 2026

Uh oh!

aliejabbari commented Jun 27, 2026

Uh oh!

aliejabbari commented Jun 27, 2026

Uh oh!

aliejabbari commented Jun 27, 2026

Uh oh!

aliejabbari commented Jul 5, 2026

Uh oh!

davnords commented Jul 5, 2026

Uh oh!

davnords commented Jul 5, 2026

Uh oh!

aliejabbari commented Jul 5, 2026

Uh oh!

davnords commented Jul 5, 2026

Uh oh!

davnords commented Jul 5, 2026

Uh oh!

davnords commented Jul 7, 2026

Uh oh!

aliejabbari commented Jul 8, 2026

Uh oh!

davnords commented Jul 9, 2026

Uh oh!

aliejabbari commented Jul 9, 2026

Uh oh!

davnords commented Jul 9, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

aliejabbari commented Jul 9, 2026

Uh oh!

davnords commented Jul 9, 2026

Uh oh!

aliejabbari commented Jul 10, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

davnords commented Jul 9, 2026 •

edited

Loading