Example: Combine text fields and publish semi-structured data

[duncandewhurst]

Sometimes multiple text fields in a publisher's data source map to a single text field in OCDS.

There are several options for handling this, depending on the nature of the data and whether there is a reason users would need to separate the values or identify which field they were mapped from:

1. Simply concatenate the values
2. Concatenate the values with a separator which does not otherwise appear in the fields, e.g. ';'
3. Use a newline and a prefix to separate the values, e.g.

<value 1>

<prefix>: <value 2>

In all cases the approach used should be documented in the publication policy.

We can document this in a worked example, with reference to the guidance shared in CRM-6884.

[jpmckinney]

Sometimes multiple text fields in a publisher's data source map to a single text field in OCDS.

We might want a section to explain when this might be the case. For example, it could be the case that there is only one candidate field that currently exists in OCDS or extensions, but the appropriate solution might be to (1) create an extension field or (2) omit a source field.

Cases for creating an extension field:

• The source fields have distinct semantics, and there are use cases for preserving the distinction in the OCDS data.

Cases for omitting a source field:

• The source fields have identical semantics. For example, the data system might require re-entry of prior information, allowing for a scenario where the information is imperfectly re-entered. The publisher considers the original field to be authoritative, in which case the repeated field can be omitted from the OCDS mapping.

Cases for merging source fields:

• The source fields have identical semantics. For example, the publisher aggregates data from multiple sources, which independently collect the same information about the same processes in such a way that differences can be introduced. The publisher considers none of the fields to take precedence.
• The source fields have semantics that are arguably identical. For example, one field takes the perspective of why the contracting process failed, and the other takes the perspective of why the contracting process was restarted. They describe the same facts, but from different perspectives. Both questions could be satisfied by the same answer (e.g. no qualified bids), but the actual wording of the fields' values might differ due to this difference in perspective. A potential test for this case could be that, if a codelist were used, both fields would use the same code.
• The source fields have distinct semantics, but it is not necessary to preserve the distinction. For example, two fields have narrower semantics than an OCDS field, e.g. there's a free-text field for quality criteria and another for cost criteria, which are narrower than awardCriteriaDetails. Since it's all free-text, and since any submitter will need to read both, the distinction isn't especially important to preserve.