Skip to content

0.2.0: calibrated intervals by default, with the served method reported #17

Description

@mstrathman

Problem

The default prediction interval is the in-sample Gaussian band, which we have measured as overconfident on smooth series (57% coverage at a nominal 90%; see the benchmarks page). The calibrated alternative (interval_method: "conformal") is opt-in, restricted to the statistical models, and refuses on series too short to calibrate. A product whose contract is "fail loudly, never serve a flattering number" should not serve a flattering band by default.

Proposal

Make the interval method adaptive by default, with the served method always visible:

  • Use conformal when the series and the serving model allow it (enough history for the calibration folds; statistical models).
  • Fall back to the residual band otherwise, and report which method served in the result document as a new "interval" field ("conformal" | "residual"), so the fallback is visible rather than silent.
  • interval_method stays as an explicit override in both directions.

Since auto model selection already runs rolling-origin machinery in the default path, the marginal cost of default calibration is far smaller than when the current default was chosen.

Scope

  • Serving change in the aggregate path plus the new document field.
  • forecast_rows() unchanged (the field rides in the document).
  • Tests: default-path coverage assertions, short-series fallback visibility, override behavior.
  • Benchmarks: re-verify the coverage table; update the docs claim from "the conformal option exists" to "calibrated by default where history allows".
  • Changelog entry under the 0.2.0 section; this is the headline change for that release.

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions