Endorse use case notes

[jbrezmorf]

• Following parts of the library should be separated and possibly outsourced:

  • distributed sampling (pools), responsible for job distribution and input/output communication
  • workspace and file exchange, should be interoperable with communication in pools, but separated
  • sampler : iterative choice of optimal number of samples
  • storage, could be a "memory storage" using a generic caching mechanism
  • MLMC quantities
  • MLMC estimator
  • moments and MEM and other complex estimation tools based on simple MLMC estiamtor
  • MLMC plots

• moments_fn is special parameter of estimator, should rather work like a function applied to some quantity, getting a derived quantity for moments. Similarly one can derive a quantity for the covariance matrix estimate etc.

• Introduce kind of logging to MLMC (blog).
  In particular failed call to qsub should be reported and possibly other warnings. Monitoring of the main sampling algorithm would be useful as well.

• implement Quantity.shape ... redirected to the QType

• implement support for kind of isinstance, in particular for asserts

• Estimate.get_level_samples ... n_samples=None get just single chunk, the public interface should rather return all samples possibly with some high limit

• HDF5 is thread unsafe, would be fine to have text file based storage for small scale projects, possibly having a directory with necessary list files and just append to them.

• Sampler should provide high level functions to work with samples and apply kind of operation [run, remove (seed), invalidate (result)] on particular set [scheduled, failed, success, particular level, ..]. Whould be great if we are able to have some cmd line tool to do that. E.g. scheduler could write a sampler config into working directory , then a general purpose tool can use that config to monitor/modify the sampling process.