St. as Street or Saint

[vr8hub]

In typogrify (line 180), we add an abbreviation tag to a list of titles, and change the space following, if present, to a no-break space. St. is one of the abbreviations included.

However, St. can be an abbreviation for Street, not just Saint (which I assume is why we don't tag St. with name-title like we do the other titles), and when it is that, then the no-break space afterwords shouldn't be added.

Since we don't add name-title to St. because the code can't know which it is, shouldn't we not add a no-break space after it for the same reason?

One thought I had is to only add the no-break space (which would have to be done separately from the rest of the titles, obviously) if what follows the St. is an uppercase word. If we wanted to use that logic, we could then add name-title under the same circumstances.

I ran a quick test on my copy of the corpus, and it looks like the above would do the right thing. I looked for any character following the St. to allow for space, no-break space, or dash (lots of things like St.-Auguste in the corpus).

(for i in ./*/ ; do gc -h -o "St\.</abbr>.[A-Z]\w+" $i/src/epub/text/*.xhtml ; done) | sort | uniq

[acabal]

What does gc do?

I think that logic is mostly sound except it would fail when St. is at the end of a sentence, like He walked down Oak St. Then he turned right. But if that case is in the small minority then we could just add it to the list of things to keep an eye out for in manual review.

[vr8hub]

Sorry, I've had that alias so long I forget it's an alias—grep -E --color=always.

Yes, that happens, but it is extremely rare. Of the 826 unique instances that the above script identifies (the total numbers are obviously much higher), it looks like only three are instances of that. And in all three of those cases, a couple of things are true: there's an "eoc" clause on the abbreviation, and the word that's prior to St. is an ordinal, i.e. ends in st/nd/rd/th. The eoc is added by a subsequent run of typogrify, so I don't think we can depend on that, but looking for the ordinal, while again not foolproof (a "real" word could end in those letters, or it could be Main St.), should further reduce the errors to a really small number.

I'll work up something and submit a PR.

[vr8hub]

This has turned out to be a bit complicated, because of the interaction between typogrify and semanticate.

Currently, typogrify unconditionally gives all the name-title like abbreviations, including St, a following no-break space if they are currently followed by a space. It also ensures they have a trailing period.
Semanticate gives the name-title like abbreviations (not including St) an abbr element with a z3998:name-title tag, if they do not already have an abbr element (if they do have an abbr element, it doesn't update it for name-title). It unconditionally gives St. (and a couple of other non-name title abbrs) an abbr element if it doesn't already have one.

Because everything is unconditional, it doesn't matter whether typogrify or semanticate is run first. Typogrify currently works around an existing abbr element if one is there when adding the no-break space. Semanticate doesn't care about the space when adding the element.

Making things conditional for St is going to cause one of two different kinds of problems.

1. Make the order matter (for St only). Change typogrify to add the no-break space on St using the criteria above, but only if it doesn't have an abbr element. That way if there are any false positives, they can be manually removed before semanticate. Then change semanticate to add an abbr element with a name-title tag to St if and only if it is followed by a no-break space. With this, we don't update previously cleared false positives if we rerun typogrify (because it ignores St's with abbr elements), and we only add name-title to the St's that have a following no-break space. However, if semanticate happened to be run first, then all the St.'s would get a plain abbr tag (same as currently, so no worse off), and when typogrify was subsequently run they would not get a no-break space since they already had an abbr element. (Only partially worse than now, since some of the St.'s shouldn't get no-break spaces and currently are.)

2. Constantly re-create false positives. If we keep things so the order doesn't matter, then both typogrify and semanticate will constantly re-create false-positives, even after they've been manually cleared. That is, typogrify will continue to allow an existing abbr element when adding the no-break space, which means manually removed false positives will be re-added every time typogrify runs. And since semanticate can't depend on the no-break space being present, it will have to have the same logic as typogrify for adding name-title, and thus will add name-title on false positives even if the no-break space is cleared after running typogrify. Also, typogrify will not only have to work around plain abbr elements, but abbr elements with name-title tags. Since typogrify and semanticate can be and re run multiple times during a production, I don't think this is a good option.

The Step by Step guide has trained everyone to run typogrify first, so I think the first option above makes the most sense. Again, all of this is only affecting St; everything else stays as is.

[acabal]

For typorify, there are three points:

1. As you noted, typogrify is run first.
2. Typogrify can make unavoidable mistakes, however its output is not manually reviewed unlike semanticate. Therefore the kinds of mistakes it makes must be able to be visually seen during a proofread (like miscurled quotes, etc.).
3. Typogrify is not expected to always do the right thing if run later, i.e. running it again later in the process may introduce changes or do the wrong thing depending on what has happened in the repo in the meantime. However running it a 2nd time is much more amenable to manual review because at that point, the number of changes it makes should be much, much fewer.

So you can design a solution you think is best given those constraints on typogrify. I think the key is that on the first run, typogrify's changes cannot be manually reviewed, because there typically are so many small changes. Thus errors must be visually identifiable. Spaces are not visually identifiable so we want to avoid making spacing mistakes in typogrify.

[vr8hub]

Excellent, thanks. It's good to have all that written down.

But, within those limitations, we can't do what needs to be done. :) What needs to be done is to conditionally add a no-break space following St., and there might be some false positives, and adding a no-break space is done in typogrify. Per the above, though, those false positives can't be reviewed (i.e. found), so the situation would be the same one we're in now—no-break spaces where they shouldn't be.

But, also, what typogrify is currently doing also violates that—adding an unconditional no-break space following St. when it should be conditional, which also results in no-break spaces that shouldn't be there, but still no way to manually review them.

So, both currently and with what's proposed, rule 2 above is being violated.

We could move both the adding of the no-break space and the adding of the name-title to St. to semanticate, but it would be kind of odd for the n-b-s for all the others to be in typogrify and for St. to be in semanticate. It would be odd for semanticate to be dealing with no-break-spaces at all.

[acabal]

If your solution is no worse than the current state of affairs then you can do that.

[vr8hub]

Addressed by #925.