g3_fit rewrite

[willbutler42]

Following on from this pull request and discussion, I have created a new branch with the aim of re-writing g3_fit. The overall aim will be to create a series of functions that will summarise a particular component of a g3 model´s reported output, these can then be called individually, or in batches (to create a custom fit) or collectively (what the updated g3_fit will do). These functions can potentially be used directly by gadgetplots also.

I have committed an example for survey indices here. I´m planning to update other components in a similar fashion in the upcoming weeks, we can use this thread to discuss and highlight other improvements that might be useful @lentinj @MikkoVihtakari @bthe @vbartolino :

access functions

• Should the access functions be able to generate reports from a model/objective function or just process the reports?
• What access functions should we have? One for each component of g3_fit (excluding params and score)?

g3_fit

• Should g3_fit take the objective function as an argument? See here. My instinct says yes.
• Is the end goal to have g3_fit and its components in gadget3? If so, what is required? Re-writing in base R (with existing imports) shouldn´t be a problem. Can make use of the g3_array_* functions and then it will just be merges and withins mostly
• Is the current output sufficient for multispecies models or would some restructuing/additional components help? @vbartolino
• Additional parameters:

1. calculate F for a specific age-range
2. define the age at which fish are recruited to the fishery. For this, the recruitment column of res.by.year could be taken from stock.full with some fiddling to align steps.
3. Is there any need for the printatstart parameter? Rather than printing biomass at the end of the timestep, we just project a year to get the biomass at the beginning of the next step now.

• g3_fit should be able to take in a list of parameter (for retro´s, jitter´s etc) and do this in parallel. This should help.
• With half an eye on gadgetplots html´s , I think an update.g3_fit function would be beneficial. This could absorb g3_retro/jitter/bootstrap classes and summarise accordingly. For instance, by supplying the bootstrap, the relevant confidence intervals will be added to the fit; or an additional component "retro" will be added which contains all the information needed to plot.

[lentinj]

Ooo, excellent :)

Should g3_fit take the objective function as an argument?

The best separation of concerns would be gotten by taking the reporting output as an input. Taking in reporting output would make it possible to have lots of little functions, so you can extract what you need and not bother with the rest. Which is basically what's going on.

This is likely impractical today, as you'll probably need to get at the parameters & model data---but we could shoe-horn these into the reporting output to help here. A list of stock names & dimensions seems like a good thing to add too.

In an ideal world the g3_fit components would all be methods on a reporting object. We don't necessarily have one of those, only the list that obj.fn$report() churns out, but maybe one day we should. We currently try hard to expose TMB for tinkering. But for day-to-day g3 that's not really that useful, in a similar maner to g3_tmb_fn(), something that wraps the TMB function which we can hang a simple optim & fit methods.

One for each component of g3_fit (excluding params and score)?

That sounds like a good strategy to me.

Is the end goal to have g3_fit and its components in gadget3? If so, what is required?

If we're encouraging the use of them in the vignettes, then it seems sensible. I'm hoping g3_array_agg takes most of the sting out of writing them, especially with gadget-framework/gadget3#238.

Some tests would make the components a lot easier to support. Some old g3 tests are a mess that needs rewriting, but latter tests are fairly high level & easy enough to write, e.g. you could easily copy-paste test-action_grow-weightjones.R & use this as a base to test one of the methods has useful return output. Note that it works against the R model (and then if enabled uses ut_tmb_r_compare2 to ensure TMB produces something equivalent). The upshot of this is working with the R & TMB reporting output. But keeping these equivalent is a good plan regardless, and maybe we should go further & drop the attributes for a list of reporting values.

calculate F for a specific age-range

I looked into what it'd take to report F directly out of gadget3 recently, but I think I got a bit stumped about what it should do with aggregation in the general case.

define the age at which fish are recruited to the fishery

Couldn't you define this with __renewalnum & __spawnednum---aggregate the first year by age and see what's non-zero? This sort of thing is tricky with gadget3, as there's the potential for the answer to be complicated (e.g. with multiple recruitment steps, and/or varying recruitment age over time). But this is another reason to break g3_fit up into components, you can use the rest without worrying that recruit-at-age will fall over.

Is there any need for the printatstart parameter?

[I added some code to detect if there's any printatend, I'd assume you can drop the printatstart parameter, rely on code similar to this, and if a report is missing then bleat that you need to add the relevant reporting option to g3a_report_detail().

Ultimately it'd be nice to remove this weirdness entirely if we can.

g3_fit should be able to take in a list of parameter (for retro´s, jitter´s etc) and do this in parallel.

I'm a bit wary of building in automatic parallelisation, mostly because of the resultant debugging pains we've had. It'd be nice if we could throw the problem at targets, and get that to worry about the debugging output, and possibly integrate into a wider pipeline later. But I dunno how feasible that is currently.

Being able to incorporate new runs into an existing fit object sounds like a very nice feature though.

Finally(!), it'd also be nice to think about how we handle age-based gadget3 in a replacement g3_fit(). Abundance reports are currently a very different shape, and I'm not sure how you'd prefer the output.

[lentinj]

calculate F for a specific age-range

(from meeting earlier today)

I'm not averse to adding (instantaneous) F calculations inside g3, vs. adding reporting of all types of mortality.

Bjarki also suggests you could turn it into a likelihood function, to keep F within a known range.

We could do something like:-

• g3 has an "over" option, similar to the likeihood components, that alters what you want the calculated F values aggregated as (by age, by age/length, etc).
• g3_fit can then aggregate this to get "mean-F for these age ranges".

...of course, this needs to happen before it gets used :)

[vbartolino]

Is the current output sufficient for multispecies models or would some restructuing/additional components help? @vbartolino

I do some output manipulation of fit$predator.prey to derive predicted diet composition and predation mortality, but it seems I already can get everything needed. With g2 we had a dedicated fit$stomachcontent but I do not see the need as I derive most of it from fit$stockdist.