IRF access

[Bonnarel]

in section "2.1.4 Access and Description of IRFs"

The content of this section looks very similar to the content of a DataLink {links} table

Except maybe the publisher_did.

irf_type can be tackled by the content_qualifier FIELD with a special irf_type vocabulary.

Nothing prevents in the DataLink 1.1 spec to use content_qualifier for something different than dataproduct_type. We just have to use the full vocabulary term (URI with the root vocabulary address)

Last but not least : DataLink {links} endpoint can be used as a service, (independantly from discovery services). Although not often used this functionality is allowed by the DataLink spec.

[Bonnarel]

More on this
The section reads
"In the table access protocol (TAP) service we could add an IRF Table,
with the following columns:
• event-list datapublisher_id
• irf_type, category of response: EffectiveArea, PSF, etc.
• irf_description, one line explanation for the role of the file
• Access.url, URL to point to the IRF
• Access.format, format of IRF
• Access.size, size of IRF file"
event-list datapublisher_id : to be added
irf_type : can be stored in DataLink content_qualifier FIELD with a specific ivoa irf_type vocabulary (to be defined)
irf_description : the DataLink description FIELD is appropriate
Access.url --> access_url FIELD
Access.format --> content_type FIELD
Access.size --> content_length FIELD

in semantics we could have a new "irf" term as sub-term of
"calibration"

in ID, we should have the list of identifiers of related event-lists

[bkhelifi]

In this proposal, there is one thing that I do not understand and I have a naive question:

You have introduced a new DataLink column irf_type but you have not defined content_type that is a mandatory field.
But, why not using content_type instead of creating a new column? And add a dedicated vocabulary (here https://www.ivoa.net/rdf/datalink/core/2022-01-27/datalink.html) with this hierarchy:

• #calibration
  • #bias, #cutout, #flat, #arf (new), #rmf (new), #bkg (new), #aeff (new), #edisp (new), #psf (new), #exposure (new)

PS: later, one could have in the future new #auxiliary products (#systematics: resource with array(s) containing systematic error values)

[Bonnarel]

Dear Bruno,
To make it clear, because I an not sure if you comment my comment or the initial draft text :

In this proposal, there is one thing that I do not understand and I have a naive question:

You have introduced a new DataLink column irf_type but you have not defined content_type that is a mandatory field. But, why not using content_type instead of creating a new column? And add a dedicated vocabulary (here https://www.ivoa.net/rdf/datalink/core/2022-01-27/datalink.html) with this hierarchy:

* #calibration
  
  * #bias, #cutout, #flat, #arf (new), #rmf (new), #bkg (new), #aeff (new), #edisp (new), #psf (new), #exposure (new)

PS: later, one could have in the future new #auxiliary products (#systematics: resource with array(s) containing systematic error values)

At that time (early April) I made a comment on the text I quoted which was in the draft. But this text was cancelled in the new version. So irf_type was not my proposal. I wrote it could be managed by a DataLink-type table
I suggested content_qualifier to be the place to fill with the IRF type (to be defined accurately by HEIG experts)
Apart, from acces_URL, size, ID and managment thing such as service, authorization, error, etc... we have four meaningful FIELDS : semantics, content_type, content_qualifier and description.
Description is totally free and for a human user.
semantics is there for the relationship to the primary item (the one in the ObsCore table).
content_type is definitely a format (mime type).
content_qualifier is there to describe the actual content of the link
It should be read this way

  ObsCore  obs_publisher                      |       semantics                |                     | content_qualifier    |  content_type

     dataset 1                        _has_             |           #this                 |       _at_     http://.../../1 |  _whis is a_  dataproduct_type:event_list |  _formatted as _   application/x-votable+xml
    dataset 1                        _has_              |       #calibration         |       _at_  http://.../.../2  | _which is a_  irf_type:gti  |_formatted as_ application/tmoc                                                                                                                                             
    dataset 1                        _has_              |       #calibration         |     _at_  http://.../.../3 | _which is  a_ irf_type:psf |  _formatted as_ application/fits                                                                                                                                                   

Of course you could also refine #calibration hierarchy to give there the different irf flavors. But this would probably avoid to use these irf_type else where. (a #bkg is for calibration but could also be accesed directly ?). At some point people didn't want to refine too much semantics that's why we had introduced content_qualifier

About usage and meaning of the different FIELDs in DataLink response this is specified in https://www.ivoa.net/documents/DataLink/20231215/REC-DataLink-1.1.html#tth_sEc3 (section 3 of DataLink 1.1 spec) and with additional text in section 3 of the implementation note (https://www.ivoa.net/documents/Notes/DataLinkImp/20240719/NOTE-DataLinkImp-1.0-20240119.html#tth_sEc3)

[msdemlei]

On Fri, May 09, 2025 at 03:35:30PM +0000, Bruno Khélifi wrote: bkhelifi left a comment (ivoa/HighEnergyObsCoreExt#2) - #calibration - #bias, #cutout, #flat, #arf (new), #rmf (new), #bkg (new), #aeff (new), #edisp (new), #psf (new), #exposure (new)

I think I'd also prefer "irf subtypes" (disclaimer: I still don't have much or an idea of what I'm talking about here) could be handled through semantics rather than an extra column. Making the identifiers *a bit* more speaking would probably be a win. I know what a PSF is, but for, say, arf... uh. It sounds cute, though :-) Off-list, I've brainstormed with Ian a bit, and we came up with *#response-function as what I think would be a generic term for several of these. Perhaps we could start with this? If there's an identified requirement to have a more fine-grained distinction between the various sorts of response functions *machine-readably* (if it doesn't have to be machine readable, the description column would do), it would be cool if we could spell the associated use cases out as concretely as we can. Even then, I expect it would be great if we introduced hierarchy here so that people could say "give me everything that's possibly a usable response function" or "weed out all response functions". That's particularly nice if there's lots of them. Even in a rendering directed towards humans (see http://dc.g-vo.org/flare_survey/q/mdl/dlmeta?ID=ivo%3A//org.gavo.dc/~%3Fflare_survey/data/plates/ESO040_004362.fits with a web browser as an example) it's nice if these get uncluttered: If all these are folded in by default (as for the #counterpart lot in the example), that'll be a friendly gesture towards unsuspecting users.

[iannevans]

I think I'd also prefer "irf subtypes" (disclaimer: I still don't
have much or an idea of what I'm talking about here) could be handled
through semantics rather than an extra column. Making the
identifiers a bit more speaking would probably be a win. I know
what a PSF is, but for, say, arf... uh. It sounds cute, though :-)

Off-list, I've brainstormed with Ian a bit, and we came up with
*#response-function as what I think would be a generic term for several
of these. Perhaps we could start with this?

I also prefer to start with #response-function for instrument responses since many responses are not "calibrations" in the traditional sense, and I think that thinking about them as being "calibrations" muddies the water. If anything, they are "de-calibrations". By that I mean that traditional calibrations map observables (e.g., DN) to physical variables (e.g., fluxes), whereas de-calibrations map physical variables (e.g., flux per energy bin) to observables (e.g., counts per pulse height analyzer bin). They are used in a way that is totally different to traditional calibrations (where you typically apply calibrations to your observables to extract physical quantities and then rarely if ever go back to the calibrations again). For HEA, instrument responses are used directly as part of the data analysis steps directly because a typical data analysis process involves making a model of the physical behavior of the source, folding the model through the responses to predict the observables, and using forward fitting to optimize the model.

I also try to avoid using the term "IRF" in any normative sense because its meaning varies depending on who you talk to. Some groups have used IRF as shorthand for ARF * RMF, which is a very explicit response, whereas elsewhere it has been used as a general term for any instrument response function. I think it's fine for any facility to use the term IRF as they wish, but for an international standard that hopefully supports facilities worldwide, it's somewhat confused history suggests that it may not be the best choice.

[msdemlei]

On Thu, Jun 26, 2025 at 12:53:03PM -0700, Ian Nigel Evans wrote: I also prefer to start with #response-function for instrument responses since many responses are not "calibrations" in the

So, I think we ought to go ahead with a VEP to add it; Ian, could you review ivoa-std/VEPs#16 and, if at all possible, make it so the used-in queries actually return something? And while I'm talking about HE-related VEPs: There is also still ivoa-std/VEPs#15, which was discussed extensively... um... @loumir, do you remember where? It would be good if we found some resolution there as well, in particular as regards the relationship and its used-in.

[iannevans]

So, I think we ought to go ahead with a VEP to add it; Ian, could you
review ivoa-std/VEPs#16 and, if at all
possible, make it so the used-in queries actually return something?

And while I'm talking about HE-related VEPs: There is also still
ivoa-std/VEPs#15, which was discussed
extensively... um... @loumir, do you remember where? It would be
good if we found some resolution there as well, in particular as
regards the relationship and its used-in.

Hi Markus,

While I agree these should be done from my perspective, I think the HEIG has to coalesce around the vocabulary terms and definitions before proceeding. That process will be through completing the PEN for HEA IVOA ObsCore Extensions and Data Discovery.

Thank you,
--Ian