EPIC: Roadmap v1.0

[pabloitu]

As mentioned in #239, API has stablized and we need to complete minor work for v1.0.0. Here we agree on the tasks, and issues, needed to be completed before the major release.

Features:

• Correction to M-test with resampling, and addition of new MLL test #268 (v0.7.0)
• Implementation of the Brier Score and its consistency test. #232 (v0.7.0)
• Implement Diebold–Mariano Test, an evolved T-test #276

Major fix:

• Spatial binning doesn't properly account for floating point precision #255 (v0.7.0)

Moderate Fixes:

• Fixed documentation errors and warnings. #275 (v0.7.0).
• Migrate from setup.py to pyproject.toml + setup.cfg (PEP 660) #274
• How to deal with forecast rates ≤ 0 in any LL-based test? #226
• T-test: deal with corner cases that raise RuntimeWarnings #225

Minor Fixes:

• CartesianGrid2D not having magnitude attribute instantiation #257 (v0.7.0)
• Link to CSEP Website #256 (v0.7.0)
• global region does not support different resolutiosn #187 (v0.7.0)
• Refactor binomial to binary in tests/ and csep.core/binomial_evaluations.py #216
• Add binomial_evaluations to top-level __init__.py #215
• add new-zealand and japan testing regions #176
• read basic catalog using csv.reader #90

Fixes needing closing (?):

• catalog-based m-test #196 (v0.7.0)
• version v0.7.0 roadmap #195 (v0.7.0)
• Dealing with dependencies in pyCSEP #192 (v0.7.0)
• Plot comparison test failing #214

Refactoring:

• Plot Refactoring #263, which closes Plot Refactoring  #260, Basemaps downloading requires a timeout option #258
• Clean up unused functionality (e.g., documents.py, file.py, log.py and many other functions within)
• Refactor core-modules structure (might break API): In which module should gridded non-poisson tests (brier, negbinom, binary) be placed? Should each have its own module? I believe we should abstract any coupling between scores, tests and distributions: Some users might be interested in the tests, and some only the scores of the test.
• Refactor core modules to harmonize function signatures and include type hinting (not breaking API)
• Clean-up example/tutorial folders, and decide what goes to tutorials and what to notebooks.

[wsavran]

Hi @pabloitu,

Some thoughts here... Overall the roadmap to 1.0 looks good. We probably want to add some incremental releases in here, and we can iterate on during our next call. Also, I would elevate #255 to being a major issue. This has implications for most of the inner workings of the code and evaluations.