vs. RRO

[zheng-da]

FlashR

For awesomer

library(FlashR)
mat <- fm.runif.matrix(1000000, 1000)
for (i in 1:5) {
    start <- Sys.time(); res <- crossprod(mat); end <- Sys.time()
    print(end - start)
}
for (i in 1:5) {
    start <- Sys.time(); res <- svd(mat); end <- Sys.time()
    print(end - start)
}
for (i in 1:5) {
    start <- Sys.time(); res <- prcomp(mat); end <- Sys.time()
    print(end - start)
}
source("/home/zhengda/FlashR-learn/R/lda.R")
for (i in 1:5) {
    start <- Sys.time(); res <- lda.default(mat, as.integer(fm.runif(nrow(mat), min=0, max=3))); end <- Sys.time()
    print(end - start)
}

For home desktop,

library(FlashR)
mat <- fm.runif.matrix(100000, 1000)
for (i in 1:5) {
    start <- Sys.time(); res <- crossprod(mat); end <- Sys.time()
    print(end - start)
    res <- NULL
    gc()
}
for (i in 1:5) {
    start <- Sys.time(); res <- svd(mat); end <- Sys.time()
    print(end - start)
    res <- NULL
    gc()
}
for (i in 1:5) {
    start <- Sys.time(); res <- prcomp(mat); end <- Sys.time()
    print(end - start)
    res <- NULL
    gc()
}
source("/home/zhengda/research/FlashR-learn/R/lda.R")
for (i in 1:5) {
    start <- Sys.time(); res <- lda.default(mat, as.integer(fm.runif(nrow(mat), min=0, max=3))); end <- Sys.time()
    print(end - start)
    res <- NULL
    gc()
}

[zheng-da]

For RRO,

mat <- matrix(runif(1000000000), 1000000, 1000)
for (i in 1:5) {
    start <- Sys.time(); res <- crossprod(mat); end <- Sys.time()
    print(end - start)
    res <- NULL
    gc()
}
for (i in 1:5) {
    start <- Sys.time(); res <- svd(mat); end <- Sys.time()
    print(end - start)
    res <- NULL
    gc()
}
for (i in 1:5) {
    start <- Sys.time(); res <- prcomp(mat); end <- Sys.time()
    print(end - start)
    res <- NULL
    gc()
}
library(MASS)
for (i in 1:5) {
    start <- Sys.time(); res <- lda(mat, as.integer(runif(nrow(mat), min=0, max=3))); end <- Sys.time()
    print(end - start)
    res <- NULL
    gc()
}

[zheng-da]

When FlashR runs on NVM on the desktop, there are multiple problems:

• systemd-journald consumes 100% of a CPU core.
• It seems FlashR access much more data than it should.

$ service systemd-journald stop

It seems that we should disable systemd-journald. If we do so, ubuntu will crash.

[zheng-da]

Test the performance of crossprod on a rectangular matrix and a square matrix.
try it on awesomer and my desktop.