enable dqm comparison job for rntuple#2651
Conversation
|
enable rntuple |
|
A new Pull Request was created by @smuzaffar for branch master. @akritkbehera, @cmsbuild, @iarspider, @raoatifshad, @smuzaffar can you please review it and eventually sign? Thanks. |
|
cms-bot internal usage |
|
please test though RNTuple comparison job will fail but lets see what works |
cmsbuild
added
tests-started
tests-pending
and removed
tests-pending
tests-started
labels
|
Pull request #2651 was updated. |
6 similar comments
|
Pull request #2651 was updated. |
|
Pull request #2651 was updated. |
|
Pull request #2651 was updated. |
|
Pull request #2651 was updated. |
|
Pull request #2651 was updated. |
|
Pull request #2651 was updated. |
|
please test |
|
-1 Failed Tests: RelVals-RNTUPLE Failed RelVals-RNTUPLE
Comparison SummarySummary:
RNTUPLE Comparison SummarySummary:
Max Memory Comparisons exceeding threshold@cms-sw/core-l2 , I found 132 workflow step(s) with memory usage exceeding the error threshold:
|
|
(not related to this PR)
I assume this report is for comparing RNTuple RelVals to default RelVals. Large increase is presently expected, but maybe there is some value in keeping them and recording the values in GitHub comments? If the default RelVals would issue a warning as well, would we have two Couple of further notes: It would be nice if the order of workflows in the summary message #2651 (comment) would be the same as in the tables https://cmssdt.cern.ch/SDT/jenkins-artifacts/baseLineComparisonsRNTUPLE/CMSSW_16_1_X_2026-01-15-1100+96a0b5/72771/maxmem-comparison/maxmem_summary.html In the summary page https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-96a0b5/50657/summary.html I see
@gartung Would you be able look into polishing these aspects? (we should open a separate task issue for the work) |
I think the order is determined by when the tests complete so it is not deterministic. RNTUPLE is a flavor of test and might have completed first. |
|
It might be alphabetical order in which case 2653 should fix it. |
@makortel , yes this is for RNTuple Relvals comparison with default IBs relvals (non-rntuple). I personally think that we should only have a short message e.g.
yes, each comparison job will add an extra Max Memory Comparison section. For now we are running comparison for default and rntuple only but if in future we enabled it for GPUs then each GPU can add this section. @gartung , order of the items on the summary page is alphabetical, so if you change these items to start with |
|
please test with cms-sw/cmssw#50096 |
|
-1 Failed Tests: Build ClangBuild Failed BuildI found compilation error when building: /cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02928/el8_amd64_gcc13/external/gcc/13.4.0-6908cfdf803923e783448096ca4f0923/bin/c++ -O3 -pthread -pipe -Werror=main -Werror=pointer-arith -Werror=overlength-strings -Wno-vla -Werror=overflow -std=c++20 -ftree-vectorize -Werror=array-bounds -Werror=format-contains-nul -Werror=type-limits -fvisibility-inlines-hidden -fno-math-errno --param vect-max-version-for-alias-checks=50 -Xassembler --compress-debug-sections -Wno-error=array-bounds -Warray-bounds -fuse-ld=bfd -march=x86-64-v3 -felide-constructors -fmessage-length=0 -Wall -Wno-non-template-friend -Wno-long-long -Wreturn-type -Wextra -Wpessimizing-move -Wclass-memaccess -Wno-cast-function-type -Wno-unused-but-set-parameter -Wno-ignored-qualifiers -Wno-unused-parameter -Wunused -Wparentheses -Werror=return-type -Werror=unused-value -Werror=unused-label -Werror=address -Werror=format -Werror=sign-compare -Werror=write-strings -Werror=delete-non-virtual-dtor -Werror=strict-aliasing -Werror=narrowing -Werror=unused-but-set-variable -Werror=reorder -Werror=unused-variable -Werror=conversion-null -Werror=return-local-addr -Wnon-virtual-dtor -Werror=switch -fdiagnostics-show-option -Wno-unused-local-typedefs -Wno-attributes -Wno-psabi -Wno-error=unused-variable -DBOOST_DISABLE_ASSERTS -flto=auto -fipa-icf -flto-odr-type-merging -fno-fat-lto-objects -Wodr -fPIC tmp/el8_amd64_gcc13/src/FWIO/RNTupleTempInput/bin/edmRNTupleTempFileUtil/CollUtil.cc.o tmp/el8_amd64_gcc13/src/FWIO/RNTupleTempInput/bin/edmRNTupleTempFileUtil/EdmRNTupleTempFileUtil.cpp.o -Wl,-E -Wl,--hash-style=gnu -Wl,--as-needed -Wl,-z,noexecstack -L/data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_16_1_X_2026-02-10-1100/biglib/el8_amd64_gcc13 -L/data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_16_1_X_2026-02-10-1100/lib/el8_amd64_gcc13 -L/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02928/el8_amd64_gcc13/cms/cmssw-patch/CMSSW_16_1_X_2026-02-10-1100/biglib/el8_amd64_gcc13 -L/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02928/el8_amd64_gcc13/cms/cmssw-patch/CMSSW_16_1_X_2026-02-10-1100/lib/el8_amd64_gcc13 -L/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02928/el8_amd64_gcc13/cms/cmssw-patch/CMSSW_16_1_X_2026-02-10-1100/external/el8_amd64_gcc13/lib -L/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02928/el8_amd64_gcc13/cms/cmssw/CMSSW_16_1_X_2026-02-09-2300/lib/el8_amd64_gcc13 -L/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02928/el8_amd64_gcc13/cms/cmssw/CMSSW_16_1_X_2026-02-09-2300/biglib/el8_amd64_gcc13 -L/data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_16_1_X_2026-02-10-1100/static/el8_amd64_gcc13 -L/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02928/el8_amd64_gcc13/cms/cmssw-patch/CMSSW_16_1_X_2026-02-10-1100/static/el8_amd64_gcc13 -L/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02928/el8_amd64_gcc13/cms/cmssw/CMSSW_16_1_X_2026-02-09-2300/static/el8_amd64_gcc13 -lFWStorageServices -lFWCoreCatalog -lFWCoreServiceRegistry -lSimDataFormatsRandomEngine -lDataFormatsCommon -lFWCoreParameterSet -lFWCoreAbstractServices -lFWCoreMessageLogger -lDataFormatsProvenance -lFWCorePluginManager -lFWCoreReflection -lFWCoreUtilities -lROOTNTuple -lHist -lMatrix -lROOTVecOps -lTree -lNet -lThread -lMathCore -lRIO -lboost_program_options -lCore -lboost_thread -lboost_date_time -lCLHEP -lpcre -lbz2 -luuid -ltbb -llzma -lz -lcms-md5 -lcrypt -ldl -lrt -lstdc++fs -ltinyxml2 -o tmp/el8_amd64_gcc13/src/FWIO/RNTupleTempInput/bin/edmRNTupleTempFileUtil/edmRNTupleTempFileUtil
Copying tmp/el8_amd64_gcc13/src/FWIO/RNTupleTempInput/bin/edmRNTupleTempFileUtil/edmRNTupleTempFileUtil to productstore area:
>> Compiling src/FWIO/RNTupleTempInput/bin/edmRNTupleTempStorage.cc
/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02928/el8_amd64_gcc13/external/gcc/13.4.0-6908cfdf803923e783448096ca4f0923/bin/c++ -c -DCMS_MICRO_ARCH='x86-64-v3' -DGNU_GCC -D_GNU_SOURCE -DBOOST_SPIRIT_THREADSAFE -DPHOENIX_THREADSAFE -DBOOST_MATH_DISABLE_STD_FPCLASSIFY -DBOOST_UUID_RANDOM_PROVIDER_FORCE_POSIX -DBOOST_MPL_IGNORE_PARENTHESES_WARNING -DCMSSW_GIT_HASH='CMSSW_16_1_X_2026-02-10-1100' -DPROJECT_NAME='CMSSW' -DPROJECT_VERSION='CMSSW_16_1_X_2026-02-10-1100' -Isrc -Ipoison -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02928/el8_amd64_gcc13/cms/cmssw-patch/CMSSW_16_1_X_2026-02-10-1100/src -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02928/el8_amd64_gcc13/external/pcre/8.43-6d98fda3bfd074ebb583e2d6a2c75d25/include -isystem/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02928/el8_amd64_gcc13/external/boost/1.80.0-b819d3899535842b3b08dcd6a725af1a/include -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02928/el8_amd64_gcc13/external/bz2lib/1.0.6-d113e1c6278c07eeaff5f84db9548446/include -isystem/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02928/el8_amd64_gcc13/lcg/root/6.36.09-8a6db48e639169d76071b7f4a20dbfbf/include -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02928/el8_amd64_gcc13/external/xz/5.6.4-b9c4ffbc390ed320a5d57fd552e29a05/include -I/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02928/el8_amd64_gcc13/external/zlib/1.2.13-589f6bb51bbeba38a7adf5a10ea8a093/include -O3 -pthread -pipe -Werror=main -Werror=pointer-arith -Werror=overlength-strings -Wno-vla -Werror=overflow -std=c++20 -ftree-vectorize -Werror=array-bounds -Werror=format-contains-nul -Werror=type-limits -fvisibility-inlines-hidden -fno-math-errno --param vect-max-version-for-alias-checks=50 -Xassembler --compress-debug-sections -Wno-error=array-bounds -Warray-bounds -fuse-ld=bfd -march=x86-64-v3 -felide-constructors -fmessage-length=0 -Wall -Wno-non-template-friend -Wno-long-long -Wreturn-type -Wextra -Wpessimizing-move -Wclass-memaccess -Wno-cast-function-type -Wno-unused-but-set-parameter -Wno-ignored-qualifiers -Wno-unused-parameter -Wunused -Wparentheses -Werror=return-type -Werror=unused-value -Werror=unused-label -Werror=address -Werror=format -Werror=sign-compare -Werror=write-strings -Werror=delete-non-virtual-dtor -Werror=strict-aliasing -Werror=narrowing -Werror=unused-but-set-variable -Werror=reorder -Werror=unused-variable -Werror=conversion-null -Werror=return-local-addr -Wnon-virtual-dtor -Werror=switch -fdiagnostics-show-option -Wno-unused-local-typedefs -Wno-attributes -Wno-psabi -Wno-error=unused-variable -DBOOST_DISABLE_ASSERTS -flto=auto -fipa-icf -flto-odr-type-merging -fno-fat-lto-objects -Wodr -fPIC -MMD -MF tmp/el8_amd64_gcc13/src/FWIO/RNTupleTempInput/bin/edmRNTupleTempStorage/edmRNTupleTempStorage.cc.d src/FWIO/RNTupleTempInput/bin/edmRNTupleTempStorage.cc -o tmp/el8_amd64_gcc13/src/FWIO/RNTupleTempInput/bin/edmRNTupleTempStorage/edmRNTupleTempStorage.cc.o
src/FWIO/RNTupleTempInput/bin/edmRNTupleTempStorage.cc: In function 'int main(int, const char**)':
src/FWIO/RNTupleTempInput/bin/edmRNTupleTempStorage.cc:176:3: error: 'TApplication' was not declared in this scope
176 | TApplication application("edmRNTupleTempStorage", nullptr, nullptr);
| ^~~~~~~~~~~~
gmake: *** [tmp/el8_amd64_gcc13/src/FWIO/RNTupleTempInput/bin/edmRNTupleTempStorage/edmRNTupleTempStorage.cc.o] Error 1
>> Building binary edmRNTupleTempStorage
/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02928/el8_amd64_gcc13/external/gcc/13.4.0-6908cfdf803923e783448096ca4f0923/bin/c++ -O3 -pthread -pipe -Werror=main -Werror=pointer-arith -Werror=overlength-strings -Wno-vla -Werror=overflow -std=c++20 -ftree-vectorize -Werror=array-bounds -Werror=format-contains-nul -Werror=type-limits -fvisibility-inlines-hidden -fno-math-errno --param vect-max-version-for-alias-checks=50 -Xassembler --compress-debug-sections -Wno-error=array-bounds -Warray-bounds -fuse-ld=bfd -march=x86-64-v3 -felide-constructors -fmessage-length=0 -Wall -Wno-non-template-friend -Wno-long-long -Wreturn-type -Wextra -Wpessimizing-move -Wclass-memaccess -Wno-cast-function-type -Wno-unused-but-set-parameter -Wno-ignored-qualifiers -Wno-unused-parameter -Wunused -Wparentheses -Werror=return-type -Werror=unused-value -Werror=unused-label -Werror=address -Werror=format -Werror=sign-compare -Werror=write-strings -Werror=delete-non-virtual-dtor -Werror=strict-aliasing -Werror=narrowing -Werror=unused-but-set-variable -Werror=reorder -Werror=unused-variable -Werror=conversion-null -Werror=return-local-addr -Wnon-virtual-dtor -Werror=switch -fdiagnostics-show-option -Wno-unused-local-typedefs -Wno-attributes -Wno-psabi -Wno-error=unused-variable -DBOOST_DISABLE_ASSERTS -flto=auto -fipa-icf -flto-odr-type-merging -fno-fat-lto-objects -Wodr -fPIC tmp/el8_amd64_gcc13/src/FWIO/RNTupleTempInput/bin/edmRNTupleTempStorage/edmRNTupleTempStorage.cc.o -Wl,-E -Wl,--hash-style=gnu -Wl,--as-needed -Wl,-z,noexecstack -L/data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_16_1_X_2026-02-10-1100/biglib/el8_amd64_gcc13 -L/data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_16_1_X_2026-02-10-1100/lib/el8_amd64_gcc13 -L/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02928/el8_amd64_gcc13/cms/cmssw-patch/CMSSW_16_1_X_2026-02-10-1100/biglib/el8_amd64_gcc13 -L/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02928/el8_amd64_gcc13/cms/cmssw-patch/CMSSW_16_1_X_2026-02-10-1100/lib/el8_amd64_gcc13 -L/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02928/el8_amd64_gcc13/cms/cmssw-patch/CMSSW_16_1_X_2026-02-10-1100/external/el8_amd64_gcc13/lib -L/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02928/el8_amd64_gcc13/cms/cmssw/CMSSW_16_1_X_2026-02-09-2300/lib/el8_amd64_gcc13 -L/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02928/el8_amd64_gcc13/cms/cmssw/CMSSW_16_1_X_2026-02-09-2300/biglib/el8_amd64_gcc13 -L/data/cmsbld/jenkins/workspace/ib-run-pr-tests/CMSSW_16_1_X_2026-02-10-1100/static/el8_amd64_gcc13 -L/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02928/el8_amd64_gcc13/cms/cmssw-patch/CMSSW_16_1_X_2026-02-10-1100/static/el8_amd64_gcc13 -L/cvmfs/cms-ib.cern.ch/sw/x86_64/nweek-02928/el8_amd64_gcc13/cms/cmssw/CMSSW_16_1_X_2026-02-09-2300/static/el8_amd64_gcc13 -lROOTNTuple -lROOTVecOps -lTree -lNet -lThread -lMathCore -lRIO -lboost_program_options -lCore -lboost_thread -lboost_date_time -lpcre -lbz2 -llzma -lz -lcrypt -ldl -lrt -o tmp/el8_amd64_gcc13/src/FWIO/RNTupleTempInput/bin/edmRNTupleTempStorage/edmRNTupleTempStorage
Failed Clang BuildI found compilation error while trying to compile with clang. Command used: >> Local Products Rules ..... started >> Local Products Rules ..... done >> Creating project symlinks >> Entering Package FWIO/RNTupleTempInput >> Compile sequence completed for CMSSW CMSSW_16_1_X_2026-02-10-1100 gmake: *** [There are compilation/build errors. Please see the detail log above.] Error 1 Command exited with non-zero status 1 Command being timed: "scram build -k -j 32 COMPILER=llvm compile BUILD_LOG=yes" User time (seconds): 54.55 System time (seconds): 6.86 Percent of CPU this job got: 689% |
cmsbuild
added
tests-rejected
tests-started
and removed
tests-pending
tests-rejected
labels
|
@makortel , should we re-run the tests now ? |
|
Yes (apparently my earlier attempt didn't pick the latest update to the PR) |
|
please test with cms-sw/cmssw#50096 |
|
+1 Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-96a0b5/51235/summary.html Comparison SummarySummary:
RNTUPLE Comparison SummarySummary:
Max Memory Comparisons exceeding threshold RNTUPLE@cms-sw/core-l2 , I found 133 workflow step(s) with memory usage exceeding the error threshold: Expand to see workflows ...
|
|
please test |
|
+1 Summary: https://cmssdt.cern.ch/SDT/jenkins-artifacts/pull-request-integration/PR-96a0b5/51281/summary.html Comparison SummarySummary:
RNTUPLE Comparison SummarySummary:
Max Memory Comparisons exceeding threshold RNTUPLE@cms-sw/core-l2 , I found 133 workflow step(s) with memory usage exceeding the error threshold: Expand to see workflows ...
|
|
@makortel , cms-sw/cmssw#50096 has fixed the hanging issue. Should we go ahead and integrate this change. This should allow to run PR comparison for rntuple relvals |
Sounds good to me |
|
+externals |
cmsbuild
added
fully-signed
externals-approved
and removed
pending-signatures
externals-pending
labels
|
This pull request is fully signed and it will be integrated in one of the next master IBs (tests are also fine). This pull request will now be reviewed by the release team before it's merged. @ftenchini, @sextonkennedy, @mandrenguyen (and backports should be raised in the release meeting by the corresponding L2) |
4 participants
This change enables running PR DQM/Reco comparison for RNTuple tests.