Notes on expansion of DATS.json metadata to cover LORIS use case

[emmetaobrien]

The larger-scale objective here is for the DATS specification developed for CONP to expand into a superset of possible use cases, with LORIS as the first use case considered. We also do not wish to break compatibility with the core DATS schema unless unavoidable.

The context for these modifications is retaining one DATS editor, which will be fed a parameter determining whether it is being used from CONP or LORIS, and will have templates storing different required fields, autofilled fields and so on determined by that parameter. The functionality that generates a skeletal README from a completed DATS.json is of interest in LORIS usage.

New fields will be added in a subsection extraProperties->LORIS of the DATS.json file unless specified otherwise.

Field-specific details: representing LORIS fields in DATS

ID: Required. Will be project-specific, may cross link to instance later. Can be multiple (LORIS ID, various external IDs).

Name: Required. Use existing title field.

Alias: Required. New field. Three or four letter code, derived from name manually.

Recruitment target: Required. Needs values for both "projected" and "counted". Check whether existing number of participants field can support this behaviour, if not create new field for it.

Use GUID: maybe add later
Use PII storage: maybe add later

Cohort association: may not be necessary at this level? If it is:

cohort->name: required
cohort->useEDC: default value "no"
cohort->recruitmentTarget: optional

Start and end date: Can be handled in existing dates field format, with editor requiring both values to be filled. Simple verification of start < end also in editor.

Candidate age: new field. Sanity check can be in editor.

Sites: use expanded version of existing origin fields.
origin->site: as there can be multiple "sites" at one institution.
origin->alias: new field
origin->consortium: maps onto LORIS network name

Populate pulldown list of sites for given instance and project?

Visits: new fields, name and label pair. Visit names need to be unique (should in theory only need unique visit + project combination, but this has not been implemented.)
Optional min->max age range for visit.
Optional optimum min->max age range for visit (narrower).

List and number(?) of instruments - new fields.
Instrument source - new field, dropdown from fairly small fixed list.

MRI modalities and tissue sample types: use existing types or keywords fields
drop-down list: MRI_T1, MRI_T1a... and then TISSUE_Blood etc.

consent configs: use existing REB_statement field. Upgrade existing text with a couple of other options. This may be further expanded later.

Other CONP DATS properties

extraProperties->files: should not be required for LORIS. May again want "projected" and "counted" options.

creators: required.

ORCID: recommended (becoming increasingly useful, but administrators and co-ordinators may be legitimate creators who do not have an ORCID)

licenses: maybe recommended.

formats: required?

size and units: optional, label as "projected"/"counted".

authorization: check whether current CONP usage and/or core DATS schema require both this and privacy and if not remove one of them.

origin: confirm that city/province/country subfields are by institution.

registrationPage: remove from LORIS initially, revisit later maybe

primaryPublications: stays recommended

dimensions: recommended or required for storing the metrics the study is assessing. Needs clearer labelling in the UI.

contact: keep as is

isAbout: keep as is, needs clearer labelling in the UI.

acknowledges: keep as is?

spatial coverage: optional, probably not relevant to LORIS

languages: make a broader property under extraProperties, it is currently only associated with experiments, and expand with codes for languages (look this up).

validation: not necessary for LORIS

accessibility: not necessary for LORIS

[emmetaobrien]

Checking the core DATS dataset schema at https://github.com/datatagsuite/schema/blob/master/dataset_schema.json it looks like privacy is required at that level, but authorizations is a required component of access entries under distribution. privacy applies at the level of a whole dataset, while authorizations allows for variations between different methods of engaging with a dataset, though we are not at this point using it that way.

Also ID can only be a single value, so the LORIS + external IDs case will need a new field.

[emmetaobrien]

The current setup with origin is not sufficiently flexible to support LORIS needs as expressed at the meeting of December 20 2024.

Establishing a structure that is sufficiently flexible and powerful is straightforward in theory. In practice, we will need to decide whether to upgrade the existing structure (probably better in principle, but manually updating ~150 entries already in CONP is not a small task and may not be practical) or setting up an alternate LORIS-specific set of fields.

[christinerogers]

Unrelated to the Dec.20 meeting discussions - from EEGNet LORIS use case and ~10 datasets with DATS.json

Had suggested in discussions with Patrick a field called platform. to store the platform URL.
@samirdas @ridz1208 this wouldn't open it up to platforms besides, and removes the origin connotations if feasible for implementation efficacy.

[GHPBZ]

We can't use "platform" because it is reserved for another use in schema, but we are looking for reasonable alternatives to cover a number of cases that need to distinguish between data collectors, data providers, etc.

[christinerogers]

ah thanks for that clarification, Patrick.

The main challenge in collapsing Privacy vs. Authorization I see is that (almost) all LORIS instances are Registered access, even if the data is Open consent. In these cases, would "Registered" or "Open" be stored, @emmetaobrien ?

[emmetaobrien]

That might well be a case where privacy should be "open" and authorization should be "registered" in the DATS.json file. Doing that does not commit us to how we display the information and which of those two fields we foreground, if there are reasons to emphasise one or the other.

[emmetaobrien]

belated notes from most recent meeting:

origin->site will need to be distinct from origin->institution as site can refer to multiple possibilties (scanning site, data-gathering site &c.)

origin->network also needs to be distinct from origin->consortium.

The proposed plan of action is to have the presence of these fields determined by a parameter set on calling the editor, which will indicate whether it is in a LORIS or CONP context (or later, other contexts such as BHI.) Having them all as optional extraProperties in DATS.json, and enforcing any fields that are "required in LORIS context but not needed otherwise" in the editor rather than the schema, is my current recommendation for implementing this.