Avoid using the terms "latin1" and "iso-8859-1" for isomorphic encoding/decoding

[domenic]

09878c7 made me aware that the functions latin1toString, latin1fromString propagate the confusion between "latin1" and "isomorphic" that we see in a lot of the JavaScript ecosystem, and have tried to help combat in whatwg/encoding@36fb4e7.

In short, the "latin1" encoding specified in the ISO-8859-1 spec does not provide any encodings for the bytes 0x00 to 0x1F or 0x7F to 0x9F. So a proper latin1 decoder would never return those bytes, and a proper latin1 decoder would throw when given those bytes.

In practice, nobody does this, and we have either:

• Libraries following the windows-1252 mapping (TextEncoder/TextDecoder, the entire web platform, Node.js's modern standard library);
• or libraries following the isomorphic decoding / encoding (a lot of C++ code, Node.js's old Buffer API).

This creates a lot of confusion when people expect one of these interpretations and get the other.

My strong suggestion is to never mention the terms latin1 or iso-8859-1 in public APIs, since they mean windows-1252 for people who read standards and mean something else (usually isomorphic encoding) for people who are coming from certain C++ codebases. (I think V8 is the original source of the confusion, at least in the Node.js ecosystem.) Instead, use the standard and non-overloaded term "isomorphic".

I realize this is a breaking change and might not be one you want to take on, but I thought I should file it, in the interest of making this the best encoding/decoding library for JS.

[ChALkeR]

In short, the "latin1" encoding specified in the ISO-8859-1 spec does not provide any encodings for the bytes 0x00 to 0x1F or 0x7F to 0x9F. So a proper latin1 decoder would never return those bytes, and a proper latin1 decoder would throw when given those bytes.

iso-8859-1 from single-byte.js follows https://unicode.org/Public/MAPPINGS/ISO8859/8859-1.TXT, which maps every possible byte value to Unicode.

Also https://icu4c-demos.unicode.org/icu-bin/convexp?conv=ISO-8859-1&s=IANA.

Also, mapping 0x00-0xff to U+0000/U+00FF is something which has many usecases and is also known as "binary", including in HTML spec.
Perhaps the shorthand methods could be renamed to binary from latin1 (while keeping the deprecated aliases until a major).

Not sure what to do with encodings though

[domenic]

iso-8859-1 from single-byte.js follows https://unicode.org/Public/MAPPINGS/ISO8859/8859-1.TXT, which maps every possible byte value to Unicode.

Also https://icu4c-demos.unicode.org/icu-bin/convexp?conv=ISO-8859-1&s=IANA.

Yep, there are a lot of possible mappings for the bytes which the ISO-8859-1 spec didn't map. You've found one of them; other people have found others.

Perhaps the shorthand methods could be renamed to binary from latin1 (while keeping the deprecated aliases until a major).

That would work, although I think "isomorphic" is probably better since it's a standard term.

Not sure what to do with encodings though

Not sure exactly which problem you're referring to here...

[ChALkeR]

the ISO-8859-1 spec

@domenic It's more complex than just the mapping in that single file

"1. Scope" section of that spec says:

ISO-8859-1 spec explicitly permits using it in conjunction with ISO/IEC 6429 (sample) (and only that).

That is the part of the standard which defines the ranges 0x00-0x1F (C0 control), 0x80-0x9F (C1 control) and 0x7F (not sure where the spec puts it, C0/C1/Cx, citations disagree on that, the original spec is behind a paywall)

ISO-8859-1 doesn't explicitly define those ranges because it defers to ISO/IEC 6429.

A decoder following that is per ISO-8859-1 specification.

there are a lot of possible mappings for the bytes which the ISO-8859-1 spec didn't map.

What are the others?