Failed to read precursor charge and monoisotopicMz

[Bohui2447]

Hi, the readIndex function can not extract precursor charge and monoisotopicMz.

library(rawrr)
raw_file = '../001_rawData/test.raw'
tmp = readIndex(rawfile = raw_file)

Here is the result of the readIndex

It works well two months before, but recent days, it can not extract these two columns.

[tobiasko]

Hi @Bohui2447,

could you please run the example code for readIndex incl. a session info?

> Idx <- rawrr::sampleFilePath() |> rawrr::readIndex()
> head(Idx)
  scan                                                     scanType   StartTime precursorMass
1    1                    FTMS + c NSI Full ms [350.0000-1800.0000] 0.001619751     1075.0000
2    2 FTMS + c NSI Full ms2 487.2567@hcd28.00 [140.0000-1015.0000] 0.005839446      487.2567
3    3 FTMS + c NSI Full ms2 644.8226@hcd28.00 [140.0000-1335.0000] 0.006989346      644.8226
4    4 FTMS + c NSI Full ms2 683.8279@hcd28.00 [140.0000-1415.0000] 0.008149761      683.8279
5    5 FTMS + c NSI Full ms2 547.2980@hcd28.00 [140.0000-1135.0000] 0.009297695      547.2980
6    6 FTMS + c NSI Full ms2 669.8381@hcd28.00 [140.0000-1385.0000] 0.010449778      669.8381
  MSOrder charge masterScan dependencyType monoisotopicMz
1      Ms      0         NA             NA              0
2     Ms2      2         NA             NA              0
3     Ms2      2         NA             NA              0
4     Ms2      2         NA             NA              0
5     Ms2      2         NA             NA              0
6     Ms2      2         NA             NA              0
> sessionInfo()
R version 4.4.1 (2024-06-14)
Platform: aarch64-apple-darwin20
Running under: macOS Sonoma 14.6

Matrix products: default
BLAS:   /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib 
LAPACK: /Library/Frameworks/R.framework/Versions/4.4-arm64/Resources/lib/libRlapack.dylib;  LAPACK version 3.12.0

locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

time zone: Europe/Zurich
tzcode source: internal

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] rawrr_1.14.0       SpectroPipeR_0.4.2

loaded via a namespace (and not attached):
  [1] splines_4.4.1           later_1.4.1             tibble_3.2.1           
  [4] R.oo_1.27.0             cellranger_1.1.0        preprocessCore_1.68.0  
  [7] XML_3.99-0.18           factoextra_1.0.7        PECA_1.42.0            
 [10] lifecycle_1.0.4         tcltk_4.4.1             edgeR_4.4.2            
 [13] doParallel_1.0.17       vroom_1.6.5             globals_0.16.3         
 [16] processx_3.8.6          lattice_0.22-6          MASS_7.3-65            
 [19] flashClust_1.01-2       backports_1.5.0         magrittr_2.0.3         
 [22] sass_0.4.9              openxlsx_4.2.8          limma_3.62.2           
 [25] rmarkdown_2.29          jquerylib_0.1.4         yaml_2.3.10            
 [28] httpuv_1.6.15           summarytools_1.1.3      zip_2.3.2              
 [31] askpass_1.2.1           reticulate_1.42.0       iq_1.10.1              
 [34] DBI_1.2.3               RColorBrewer_1.1-3      lubridate_1.9.4        
 [37] zlibbioc_1.52.0         R.cache_0.16.0          purrr_1.0.4            
 [40] R.utils_2.13.0          BiocGenerics_0.52.0     aroma.apd_0.7.0        
 [43] sva_3.54.0              GenomeInfoDbData_1.2.13 IRanges_2.40.1         
 [46] S4Vectors_0.44.0        ggrepel_0.9.6           listenv_0.9.1          
 [49] genefilter_1.88.0       aroma.affymetrix_3.2.2  umap_0.2.10.0          
 [52] RSpectra_0.16-2         annotate_1.84.0         R.rsp_0.46.0           
 [55] parallelly_1.43.0       svglite_2.1.3           codetools_0.2-20       
 [58] xml2_1.3.8              DT_0.33                 DNAcopy_1.80.0         
 [61] tidyselect_1.2.1        UCSC.utils_1.2.0        farver_2.1.2           
 [64] matrixStats_1.5.0       stats4_4.4.1            base64enc_0.1-3        
 [67] jsonlite_2.0.0          gghalves_0.1.4          survival_3.8-3         
 [70] iterators_1.0.14        emmeans_1.11.0          systemfonts_1.2.1      
 [73] foreach_1.5.2           tools_4.4.1             ragg_1.3.3             
 [76] pryr_0.1.6              snow_0.4-4              Rcpp_1.0.14            
 [79] glue_1.8.0              gridExtra_2.3           xfun_0.51              
 [82] mgcv_1.9-1              MatrixGenerics_1.18.1   GenomeInfoDb_1.42.3    
 [85] dplyr_1.1.4             withr_3.0.2             BiocManager_1.30.25    
 [88] fastmap_1.2.0           GGally_2.2.1            openssl_2.3.2          
 [91] digest_0.6.37           timechange_0.3.0        R6_2.6.1               
 [94] mime_0.13               estimability_1.5.1      textshaping_1.0.0      
 [97] colorspace_2.1-1        effsize_0.8.1           RSQLite_2.3.9          
[100] R.methodsS3_1.8.2       UpSetR_1.4.0            tidyr_1.3.1            
[103] generics_0.1.3          hexbin_1.28.5           httr_1.4.7             
[106] htmlwidgets_1.6.4       scatterplot3d_0.3-44    ggstats_0.9.0          
[109] pkgconfig_2.0.3         gtable_0.3.6            blob_1.2.4             
[112] XVector_0.46.0          htmltools_0.5.8.1       multcompView_0.1-10    
[115] ROTS_1.34.0             kableExtra_1.4.0        scales_1.3.0           
[118] tidyverse_2.0.0         Biobase_2.66.0          leaps_3.2              
[121] png_0.1-8               doSNOW_1.0.20           R.huge_0.10.1          
[124] knitr_1.50              PSCBS_0.67.0            rstudioapi_0.17.1      
[127] tzdb_0.5.0              reshape2_1.4.4          checkmate_2.3.2        
[130] nlme_3.1-167            ggcorrplot_0.1.4.1      cachem_1.1.0           
[133] stringr_1.5.1           parallel_4.4.1          AnnotationDbi_1.68.0   
[136] pillar_1.10.1           grid_4.4.1              vctrs_0.6.5            
[139] promises_1.3.2          xtable_1.8-4            aroma.core_3.3.1       
[142] cluster_2.1.8.1         evaluate_1.0.3          isoband_0.2.7          
[145] magick_2.8.6            readr_2.1.5             mvtnorm_1.3-3          
[148] cli_3.6.4               locfit_1.5-9.12         compiler_4.4.1         
[151] rlang_1.1.5             crayon_1.5.3            R.devices_2.17.2       
[154] labeling_0.4.3          modelr_0.1.11           aroma.light_3.36.0     
[157] ps_1.9.0                affy_1.84.0             plyr_1.8.9             
[160] forcats_1.0.0           pander_0.6.6            stringi_1.8.4          
[163] viridisLite_0.4.2       BiocParallel_1.40.0     quarto_1.4.4           
[166] munsell_0.5.1           Biostrings_2.74.1       Matrix_1.7-3           
[169] rapportools_1.2         hms_1.1.3               patchwork_1.3.0        
[172] R.filesets_2.15.1       bit64_4.6.0-1           future_1.34.0          
[175] ggplot2_3.5.1           KEGGREST_1.46.0         statmod_1.5.0          
[178] FactoMineR_2.11         shiny_1.10.0            broom_1.0.7            
[181] memoise_2.0.1           affyio_1.76.0           bslib_0.9.0            
[184] bit_4.6.0               readxl_1.4.5     

In addition, could you please provide the file header of your raw file by using

> readFileHeader(rawrr::sampleFilePath())
$`RAW file`
[1] "sample.raw"

$`RAW file version`
[1] "66"

$`Creation date`
[1] "2/13/2018 11:09:03 AM"

$Operator
[1] "ExactiveUser"

$`Number of instruments`
[1] 1

$Description
[1] ""

$`Instrument model`
[1] "Q Exactive HF-X Orbitrap"

$`Instrument name`
[1] "Q Exactive HF-X Orbitrap"

$`Instrument method`
[1] "C:/Xcalibur/methods/__General_methods/STM/autoQC01_GT20min_RT35min.meth"

$`Serial number`
[1] "Exactive Series slot #6114"

$`Software version`
[1] "2.9-290033/2.9.0.2926"

$`Firmware version`
[1] "rev. 1"

$Units
[1] "None"

$`Mass resolution`
[1] "0.500"

$`Number of scans`
[1] 574

$`Number of ms2 scans`
[1] 546

$`Scan range`
[1]   1 574

$`Time range`
[1] 0.00 0.78

$`Mass range`
[1]  140 1805

$`Scan filter (first scan)`
[1] "FTMS + c NSI Full ms [350.0000-1800.0000]"

$`Scan filter (last scan)`
[1] "FTMS + c NSI Full ms2 487.2567@hcd28.00 [140.0000-1015.0000]"

$`Total number of filters`
[1] "21"

$`Sample name`
[1] ""

$`Sample id`
[1] ""

$`Sample type`
[1] "Unknown"

$`Sample comment`
[1] ""

$`Sample vial`
[1] ""

$`Sample volume`
[1] "0"

$`Sample injection volume`
[1] "0"

$`Sample row number`
[1] "1"

$`Sample dilution factor`
[1] "0"

$`Sample barcode`
[1] ""

$`User text 0`
[1] ""

$`User text 1`
[1] ""

$`User text 2`
[1] ""

$`User text 3`
[1] ""

$`User text 4`
[1] ""

[Bohui2447]

Hi @tobiasko, please see the result by running the code

Idx <- rawrr::sampleFilePath() |> rawrr::readIndex()
head(Idx)
scan scanType StartTime precursorMass MSOrder charge
1 1 FTMS + c NSI Full ms [350.0000-1800.0000] 0.001619751 1075.0000 Ms 0
2 2 FTMS + c NSI Full ms2 487.2567@hcd28.00 [140.0000-1015.0000] 0.005839446 487.2567 Ms2 2
3 3 FTMS + c NSI Full ms2 644.8226@hcd28.00 [140.0000-1335.0000] 0.006989346 644.8226 Ms2 2
4 4 FTMS + c NSI Full ms2 683.8279@hcd28.00 [140.0000-1415.0000] 0.008149761 683.8279 Ms2 2
5 5 FTMS + c NSI Full ms2 547.2980@hcd28.00 [140.0000-1135.0000] 0.009297695 547.2980 Ms2 2
6 6 FTMS + c NSI Full ms2 669.8381@hcd28.00 [140.0000-1385.0000] 0.010449778 669.8381 Ms2 2
masterScan dependencyType monoisotopicMz
1 NA NA 0
2 NA NA 0
3 NA NA 0
4 NA NA 0
5 NA NA 0
6 NA NA 0

sessionInfo()
R version 4.4.1 (2024-06-14 ucrt)
Platform: x86_64-w64-mingw32/x64
Running under: Windows 11 x64 (build 26100)

Matrix products: default

locale:
[1] LC_COLLATE=English_United States.utf8 LC_CTYPE=English_United States.utf8
[3] LC_MONETARY=English_United States.utf8 LC_NUMERIC=C
[5] LC_TIME=English_United States.utf8

time zone: America/New_York
tzcode source: internal

attached base packages:
[1] parallel stats4 stats graphics grDevices utils datasets methods base

other attached packages:
[1] doParallel_1.0.17 iterators_1.0.14 foreach_1.5.2 scales_1.4.0 ggplot2_3.5.2
[6] dplyr_1.1.4 limma_3.62.1 PSMatch_1.10.0 rawrr_1.15.10 MSnbase_2.32.0
[11] ProtGenerics_1.38.0 S4Vectors_0.44.0 mzR_2.40.0 Rcpp_1.0.14 Biobase_2.66.0
[16] BiocGenerics_0.52.0

loaded via a namespace (and not attached):
[1] remotes_2.5.0 rlang_1.1.4 magrittr_2.0.3
[4] clue_0.3-66 matrixStats_1.4.1 compiler_4.4.1
[7] callr_3.7.6 vctrs_0.6.5 reshape2_1.4.4
[10] profvis_0.4.0 stringr_1.5.1 pkgconfig_2.0.3
[13] crayon_1.5.3 fastmap_1.2.0 XVector_0.46.0
[16] ellipsis_0.3.2 utf8_1.2.4 promises_1.3.2
[19] sessioninfo_1.2.3 ps_1.9.1 UCSC.utils_1.2.0
[22] preprocessCore_1.68.0 purrr_1.0.2 MultiAssayExperiment_1.32.0
[25] zlibbioc_1.52.0 cachem_1.1.0 GenomeInfoDb_1.42.3
[28] jsonlite_1.8.9 later_1.4.1 DelayedArray_0.32.0
[31] BiocParallel_1.40.0 cluster_2.1.6 R6_2.6.1
[34] stringi_1.8.4 RColorBrewer_1.1-3 pkgload_1.4.0
[37] GenomicRanges_1.58.0 SummarizedExperiment_1.36.0 usethis_3.1.0
[40] IRanges_2.40.1 httpuv_1.6.15 Matrix_1.7-0
[43] igraph_2.1.2 tidyselect_1.2.1 rstudioapi_0.17.1
[46] abind_1.4-8 codetools_0.2-20 affy_1.84.0
[49] miniUI_0.1.2 processx_3.8.6 curl_6.2.2
[52] pkgbuild_1.4.7 lattice_0.22-6 tibble_3.2.1
[55] plyr_1.8.9 withr_3.0.2 shiny_1.10.0
[58] desc_1.4.3 urlchecker_1.0.1 pillar_1.10.2
[61] affyio_1.76.0 BiocManager_1.30.25 MatrixGenerics_1.18.1
[64] MALDIquant_1.22.3 ncdf4_1.23 generics_0.1.3
[67] xtable_1.8-4 glue_1.8.0 lazyeval_0.2.2
[70] tools_4.4.1 mzID_1.44.0 QFeatures_1.16.0
[73] vsn_3.74.0 fs_1.6.6 XML_3.99-0.17
[76] grid_4.4.1 impute_1.80.0 tidyr_1.3.1
[79] MsCoreUtils_1.18.0 devtools_2.4.5 colorspace_2.1-1
[82] GenomeInfoDbData_1.2.13 cli_3.6.3 S4Arrays_1.6.0
[85] AnnotationFilter_1.30.0 pcaMethods_1.98.0 gtable_0.3.6
[88] digest_0.6.37 SparseArray_1.6.0 htmlwidgets_1.6.4
[91] farver_2.1.2 memoise_2.0.1 htmltools_0.5.8.1
[94] lifecycle_1.0.4 httr_1.4.7 mime_0.12
[97] statmod_1.5.0 MASS_7.3-60.2

readFileHeader(rawfile = raw_file)
$RAW file
[1] "ea00150_qy_ultra_pro0_96f73.raw"

$RAW file version
[1] "66"

$Creation date
[1] "4/17/2025 1:26:21 AM"

$Operator
[1] "Eclipse"

$Number of instruments
[1] 6

$Description
[1] ""

$Instrument model
[1] "Orbitrap Eclipse"

$Instrument name
[1] "Orbitrap Eclipse"

$Instrument method
[1] "C:/Xcalibur/methods/qyu/faims/frag_optimization/faims_90min_hrMS2_500agcMs2_it86_hcd4_cid4.meth"

$Serial number
[1] "EXRFSN40454"

$Software version
[1] "4.2.4321"

$Firmware version
[1] ""

$Units
[1] "None"

$Mass resolution
[1] "0.500"

$Number of scans
[1] 63856

$Number of ms2 scans
[1] 60100

$Scan range
[1] 1 63856

$Time range
[1] 0.00 90.02

$Mass range
[1] 110 2000

$Scan filter (first scan)
[1] "FTMS + c NSI cv=-40.00 Full ms [400.0000-1400.0000]"

$Scan filter (last scan)
[1] "FTMS + c NSI cv=-80.00 Full ms [400.0000-1400.0000]"

$Total number of filters
[1] "24493"

$Sample name
[1] "qy_ultra_pro0_96f73"

$Sample id
[1] "1"

$Sample type
[1] "Unknown"

$Sample comment
[1] ""

$Sample vial
[1] "G:A1"

$Sample volume
[1] "0"

$Sample injection volume
[1] "3"

$Sample row number
[1] "0"

$Sample dilution factor
[1] "1"

$Sample barcode
[1] ""

$User text 0
[1] ""

$User text 1
[1] ""

$User text 2
[1] ""

$User text 3
[1] ""

$User text 4
[1] ""

BTW, I have two batches of raw files, it works well for the first batch of files. The errors maybe caused by the recently update of ThermoFisher system.

head(readIndex(raw_file)) # first batch of raw file
scan scanType StartTime precursorMass MSOrder
1 1 FTMS + c NSI cv=-40.00 Full ms [400.0000-1400.0000] 0.001384367 NA Ms
2 2 FTMS + c NSI cv=-60.00 Full ms [400.0000-1400.0000] 0.005798195 NA Ms
3 3 FTMS + c NSI cv=-40.00 d Full ms2 416.7851@cid22.00 [110.0000-2000.0000] 0.006965951 416.7851 Ms2
4 4 FTMS + c NSI cv=-40.00 d Full ms2 416.7851@cid25.00 [110.0000-2000.0000] 0.009305821 416.7851 Ms2
5 5 FTMS + c NSI cv=-40.00 d Full ms2 416.7851@cid28.00 [110.0000-2000.0000] 0.011366104 416.7851 Ms2
6 6 FTMS + c NSI cv=-40.00 d Full ms2 416.7851@cid31.00 [110.0000-2000.0000] 0.013426832 416.7851 Ms2
charge masterScan dependencyType monoisotopicMz
1 0 NA FALSE NA
2 0 NA FALSE NA
3 3 1 TRUE 416.7851
4 3 1 TRUE 416.7851
5 3 1 TRUE 416.7851
6 3 1 TRUE 416.7851

readFileHeader(rawfile = raw_file)
$RAW file
[1] "ea00022_qy_ultra_pro0_96f2.raw"

$RAW file version
[1] "66"

$Creation date
[1] "3/28/2025 8:57:32 PM"

$Operator
[1] "Eclipse"

$Number of instruments
[1] 6

$Description
[1] ""

$Instrument model
[1] "Orbitrap Eclipse"

$Instrument name
[1] "Orbitrap Eclipse"

$Instrument method
[1] "C:/Xcalibur/methods/qyu/faims/frag_optimization/faims_90min_hrMS2_500agcMs2_it86_hcd4_cid4.meth"

$Serial number
[1] "EXRFSN40454"

$Software version
[1] "4.0.4091"

$Firmware version
[1] ""

$Units
[1] "None"

$Mass resolution
[1] "0.500"

$Number of scans
[1] 69512

$Number of ms2 scans
[1] 65600

$Scan range
[1] 1 69512

$Time range
[1] 0.00 90.01

$Mass range
[1] 110 2000

$Scan filter (first scan)
[1] "FTMS + c NSI cv=-40.00 Full ms [400.0000-1400.0000]"

$Scan filter (last scan)
[1] "FTMS + c NSI cv=-80.00 Full ms [400.0000-1400.0000]"

$Total number of filters
[1] "27533"

$Sample name
[1] "qy_ultra_pro0_96f2"

$Sample id
[1] "1"

$Sample type
[1] "Unknown"

$Sample comment
[1] ""

$Sample vial
[1] "R:A2"

$Sample volume
[1] "0"

$Sample injection volume
[1] "5"

$Sample row number
[1] "0"

$Sample dilution factor
[1] "1"

$Sample barcode
[1] ""

$User text 0
[1] ""

$User text 1
[1] ""

$User text 2
[1] ""

$User text 3
[1] ""

$User text 4
[1] ""

[tobiasko]

So it works for all files recorded using software version 4.0.4091, after upgrading to 4.2.4321 it always failed? Looks like all files were recorded with the very same method and on the very same hardware?

[Bohui2447]

It's tricky, because it works for files generated by the software version 4.2.4321 but failed for files generated by version 4.0.4091.

[tobiasko]

That would mean our latest rawrr.exe fails to read data generated with old software versions. That would be really bad! You have the latest rawrr package version (1.15), could you please check:

> list.files(rawrrAssemblyPath())
[1] "eula.txt"                                          
[2] "rawrr_build.log.28cb22339cb1"                      
[3] "rawrr.exe"                                         
[4] "ThermoFisher.CommonCore.BackgroundSubtraction.dll" 
[5] "ThermoFisher.CommonCore.Data.dll"                  
[6] "ThermoFisher.CommonCore.MassPrecisionEstimator.dll"
[7] "ThermoFisher.CommonCore.RawFileReader.dll"        

on your system?

[Bohui2447]

list.files(rawrrAssemblyPath())
[1] "eula.txt" "rawrr.exe"

This is another tricky I've met. When I use readSpectrum function, it reports error say I don't have the .dll files in the rawrrAssemblyPath. Then I run rawrr::installRawFileReaderDLLs() and rawrr::installRawrrExe(), it says my R can not connect to the github. Even I manually download all necessary files and put in the Assembly folder can not solve the errors. So I have to clone the rawrr github into my R. Now the readSpectrum works but there's no .dll files when running list.files(rawrrAssemblyPath())

[tobiasko]

Now I am totally confused! :-) So your current rawrr.exe is from where? After installing the package there was no rawrr.exe in your assembly path and you were unable to download it using installRawrrExe?

[cpanse]

@Bohui2447, thanks for reporting. Is it possible that you provide the raw file so that we can reproduce the problem?

[Bohui2447]

@cpanse The raw file is too large to upload here. How can I send it to you privately?

[cpanse]

@Bohui2447 A link to cp@fgcz.ethz.ch or a Google Drive or Dropbox would help. C

[cpanse]

@Bohui2447, please verify if the new version resolves this issue.

install.packages('https://fgcz-ms.uzh.ch/~cpanse/rawrr/rawrr_1.17.2.tar.gz')
library(rawrr)
rawrr::installRawrrExe(force = T)