HTML descriptions

[saumier]

Email 2025-SEP-12 from @saumier:
I propose the following approach for formatting text descriptions (schema:description) in the Artsdata core graph. This approach will restrict the allowed HTML tags and attributes (sanitize), and also convert text with problematic line breaks, tabs, and spaces into simple formatted HTML.

1. Sanitize
   Allowed tags:

<p>, <br>, <ul>, <ol>, <li>
<b>, <strong>, <i>, <em>
<blockquote>, <a>
<h1>, <h2>, <h3>, <h4>, <h5>, <h6>

Allowed attributes:
href, rel, target (for <a> tags)

2. Format line breaks
   Convert single newlines (\n) into <br />, and two or more consecutive newlines (\n\n or \r\n\r\n) into </p><p>. Wrap the entire text with <p>...</p>.

Example:
"Title\nFirst line\n\nSecond section" → <p>Title<br />First line</p><p>Second section</p>

3. Normalize whitespace
   Strip leading and trailing whitespace (spaces, tabs), and replace all internal whitespace (spaces, tabs) with a single space.

Example:
" one two\t\t\t\t\tthree " → "one two three"

Note: this will make all description fields html. Event a single line of text will have <p>.

[saumier]

Email 2025-SEP-12 from @fjjulien :

This is all quite standard and sounds more than fine to me.

Will windows-1252 characters be detected and converted to UTF-8? Since we developed this conversion capacity, I would like it to be reused in text sanitization too.

[saumier]

All schema:descriptions loaded into graphs in Artsdata are already UTF-8. The conversion (like windows-1252 to URF-8) happens in the pipeline before the graph is populated. So yes, all the characters will be UTF-8 in Artsdata.