torch, trt inference update

[MinGiSa]

I have updated the PyTorch and TensorRT inference sample code. The results can be checked below, and the currently identified issue is that the TensorRT inference results show slight discrepancies. Further adjustments are required, and the sample code should be used for reference purposes only.

Key Points:

Sample code updated for PyTorch/TensorRT inference

TensorRT outputs exhibit minor inconsistencies (needs refinement)

Experimental stage — verify results before deployment

pip install tensorrt-cu12 cuda-python pycuda

※ I tested this setup with Python 3.11, torch 2.5.1+cu121, CUDA 12.1, and TensorRT 10.9. Please note that you may need to install TensorRT 10.9 separately instead of using pip install. I'm sorry that the code isn't complete.

Torch Result
00000 | 1325.5949020385742 ms | Image: 004, Anomaly Score: 0.3086
00001 | 111.27161979675293 ms | Image: 002, Anomaly Score: 0.3567
00002 | 110.40759086608887 ms | Image: 018, Anomaly Score: 0.3325
00003 | 112.01262474060059 ms | Image: 008, Anomaly Score: 0.3067
00004 | 111.31048202514648 ms | Image: 011, Anomaly Score: 0.2809
00005 | 114.17841911315918 ms | Image: 000, Anomaly Score: 0.3426
00006 | 115.04459381103516 ms | Image: 001, Anomaly Score: 0.3089
00007 | 114.02297019958496 ms | Image: 014, Anomaly Score: 0.3014
00008 | 115.77630043029785 ms | Image: 016, Anomaly Score: 0.3057
00009 | 115.22364616394043 ms | Image: 006, Anomaly Score: 0.3094
00010 | 115.58794975280762 ms | Image: 017, Anomaly Score: 0.2981
00011 | 118.33333969116211 ms | Image: 003, Anomaly Score: 0.3535
00012 | 116.68038368225098 ms | Image: 013, Anomaly Score: 0.2942
00013 | 117.12312698364258 ms | Image: 009, Anomaly Score: 0.2592
00014 | 116.47987365722656 ms | Image: 010, Anomaly Score: 0.3185
00015 | 116.58573150634766 ms | Image: 007, Anomaly Score: 0.3066
00016 | 117.93375015258789 ms | Image: 019, Anomaly Score: 0.2923
00017 | 117.4318790435791 ms | Image: 012, Anomaly Score: 0.2557
00018 | 116.3625717163086 ms | Image: 005, Anomaly Score: 0.3261
00019 | 118.41893196105957 ms | Image: 015, Anomaly Score: 0.2912

ONNX Result
00000 | 595.3686237335205 ms | Image: 004, Anomaly Score: 0.3086
00001 | 103.70206832885742 ms | Image: 002, Anomaly Score: 0.3567
00002 | 102.04100608825684 ms | Image: 018, Anomaly Score: 0.3324
00003 | 102.7829647064209 ms | Image: 008, Anomaly Score: 0.3066
00004 | 101.47380828857422 ms | Image: 011, Anomaly Score: 0.2808
00005 | 100.68964958190918 ms | Image: 000, Anomaly Score: 0.3425
00006 | 100.78644752502441 ms | Image: 001, Anomaly Score: 0.3089
00007 | 100.05760192871094 ms | Image: 014, Anomaly Score: 0.3014
00008 | 101.07970237731934 ms | Image: 016, Anomaly Score: 0.3057
00009 | 103.20234298706055 ms | Image: 006, Anomaly Score: 0.3094
00010 | 102.74052619934082 ms | Image: 017, Anomaly Score: 0.2981
00011 | 105.89814186096191 ms | Image: 003, Anomaly Score: 0.3535
00012 | 103.76238822937012 ms | Image: 013, Anomaly Score: 0.2942
00013 | 101.75347328186035 ms | Image: 009, Anomaly Score: 0.2592
00014 | 127.05779075622559 ms | Image: 010, Anomaly Score: 0.3185
00015 | 107.2390079498291 ms | Image: 007, Anomaly Score: 0.3066
00016 | 108.24441909790039 ms | Image: 019, Anomaly Score: 0.2923
00017 | 106.25267028808594 ms | Image: 012, Anomaly Score: 0.2557
00018 | 103.95479202270508 ms | Image: 005, Anomaly Score: 0.3261
00019 | 108.18338394165039 ms | Image: 015, Anomaly Score: 0.2911

TRT Result
00000 | 139.35 ms | Image: 004, Score: 0.3339
00001 | 97.73 ms | Image: 002, Score: 0.3769
00002 | 93.88 ms | Image: 018, Score: 0.3679
00003 | 95.98 ms | Image: 008, Score: 0.3183
00004 | 95.63 ms | Image: 011, Score: 0.3063
00005 | 94.87 ms | Image: 000, Score: 0.3677
00006 | 95.44 ms | Image: 001, Score: 0.3435
00007 | 93.76 ms | Image: 014, Score: 0.3250
00008 | 94.06 ms | Image: 016, Score: 0.3253
00009 | 94.59 ms | Image: 006, Score: 0.3426
00010 | 94.76 ms | Image: 017, Score: 0.3201
00011 | 95.95 ms | Image: 003, Score: 0.3690
00012 | 95.12 ms | Image: 013, Score: 0.3146
00013 | 96.25 ms | Image: 009, Score: 0.2840
00014 | 96.11 ms | Image: 010, Score: 0.3539
00015 | 97.61 ms | Image: 007, Score: 0.3384
00016 | 96.04 ms | Image: 019, Score: 0.3152
00017 | 94.86 ms | Image: 012, Score: 0.2786
00018 | 97.17 ms | Image: 005, Score: 0.3903
00019 | 102.38 ms | Image: 015, Score: 0.3302