Add otherwise keyword for case statements

[bolsen]

In the Free Pascal manual: https://www.freepascal.org/docs-html/current/ref/refsu56.html#x166-19000013.2.2

---

I found code in the wild for Free Pascal that uses otherwise in a case statement. Here is a small example:

program testotherwise;

var
  A : Integer;
begin
  A := 2;
  case A of
     1 :
       writeln('OK');
     otherwise
       writeln('WRONG');
  end;
end.

I don't see it mentioned in Delphi documentation: https://docwiki.embarcadero.com/RADStudio/Sydney/en/Declarations_and_Statements_(Delphi)#Case_Statements . Since I don't have Delphi or Windows installed, the best I can do is test with Delphi mode in Free Pascal and it worked fine.

I noticed that the tree sitter parser raised an ERROR in the parse tree when it hit this type of code, seeing this with an Emacs mode I scrapped together. (https://github.com/bolsen/pascal-ts-mode) For emacs tree-sitter, I notice that it stops highlighting when there is an ERROR, until it picks up again for some reason.

[the-Arioch]

Oh, i love aesotherics and archeology :-)

FPC, trying to be one ring to rule them all, seems to hope to assimilate most language extensions, when cost/benefit make it feasible.

As for THE Pascal, the original 1974 language had "undefined behavior" here:

Then, most languages designed by English-speakers considered adding "otherwise" clause here or there, but it was way too verbose in time where each single letter added few thousand bucks to memory bill.

Then half-litterate Gemrnas of ALGOL fame added the "else" word. It was ugly misplaced, but it was reasonably short and it's meaning overlapped plausbily. And it stuck in programming forever.

...later when reading some English literature i was shocked how little use "ubiquitous" else was making in real world texts.

So, when they tried to ascend Pascal from schools to practical languages this lack of deafault case was gaping (along with lack of strings, interactive files, etc). And it seems different flavours of early Pascal fallen upon different sides of this linguistic argument.

USCD Pascal (p-code inventors and the inspiraton to compiled Turbo Pascal) still did have neither "else" not "otherwise" extension:

 DIFFERENCES BETWEEN UCSD PASCAL AND STANDARD PASCAL* * Section 2.2 *
****************************************************** ***************
Version 11.0 February 1979

2.2.1 CASE STATEMENTS

Jensen and Wirth on page 31, state that if there is no label
equal to the value of the case statement selector, the result of the
case statement is undefined. UCSD Pascal defines that if there is
no label matching the value of the case selector then the next
statement executed" is the statement following the case statement. For
example, the following sample program will only output the line "THAT'S
ALL FOLKS" since the case statement will "fall through" to the WRITELN
statement following the case statement:

PROGRAM FALLTHROUGH;
VAR CH:CHAR;
BEGIN
CH: =' A';
CASE CH OF
'B': WRlTELN(OUTPUT, 'HI THERE');
'C': WRITELN (OUTPUT, 'THE CHARACTER IS A "e"')
END; ,
WRITELN (OlTl'pur, 'THAT' 'S ALL FOLKS');
END.

So, as late as 1979 the most popular language flavour still had none.

---

I think TurboPascal/Delphi never accepted "otherwise" option but extended it with "else" early on. I used to have a link running TP 5.5 in any browser including Firefox/Android, but not at the moment. However there is TP/BP 7 page still: https://dos.zone/turbo-pascal7/

Nor Delphi 2012/XE2

TP1-TP3 were somewhat different compilers, but they were even more memory-restricted (they could not even afford running editor an compiler at once) so i speculate they also uses bare minimum, else and nothing else.

I do not have TP3 at hands now though to do a certain test, if it matters.

Nor do i have Virtual Pascal, once a very strong contender to inherit from TP/BP.

---

On the opposite side of the dispute we might have ANSI Pascal (Somewhat late), GNU Pascal and even SGI Pascal

https://techpubs.jurassic.nl/library/manuals/0000/007-0740-030/sgi_html/ch01.html

interesting flavour, it also has return (x) operator, like recent delphi using Exit(x) pseudo-procedures. Not sure if any survived code in the wild uses return

same dies SUN Pascal: https://www.dcs.ed.ac.uk/home/SUNWspro/3.0/pascal/lang_ref/ref_state.doc.html#601

GNU considers "else" to be non-standard clone of "otherwise" and promises to support both
https://www.gnu-pascal.de/gpc/otherwise.html
notice how it use is not contained to code blocks alone!

otherwise also defines the default value in an Extended Pascal structured values of array type.
otherwise is an ISO 10206 Extended Pascal extension.

I wonder if this another "otherwise" use is supported.

[the-Arioch]

UCSD_PASCAL_II.0_Users_Manual_Mar79.pdf

if you feel dead set on supporting 'em all, here it the USCD ref, should be of historical improtance. If any ancient flavour has realistical chance to be of matter today, i guess that should be the multi-platform USCD, which used to be operating system of its own on many mini-computers, like 8-bit Basic was sort-of-OS in then home computers. I expect USCD-style programs sources survive many more than any other of Pascal-likes.

USCD is also interesting in extending Pascal with SQL-like syntax, where parameters were passed as "language clauses" rather literals or variables. A page 138 has CLOSE ( F, LOCK) exampple, LOCK being a keyword AFAIR not var. Maybe there was more, not sure, few years passed since i digged in.

Then, files. Like i said above, standard Pascal saw file variables as a special kind of pointers. It was very elegant and homogenious, but totally incompatible with interactive computers. The PUT/GET procedures, dicussed abundantly in Pascal paper books, and making one of "killer features" of the language in the era when many language had no any I/O (leasing it to ad hoc extensions vendors would cram together some way), they were removed from the language in TurboPascal+ and read/write were re-defined as wholistic procedures, not macro-substitutions of core statements.

USCD was not so bold, they dared not to abandon them, but they had to amend file typesand routines badly to make them work again. See 2.2.'1 READ AND READLN for starters.

Then "segment" procedures and "interactive" file types would be more keywords modern Delphi-likes probably miss, at least in the specific contexts.

[bolsen]

Haha - @the-Arioch : this explained very much my "huh?" moment (when seeing it with an error, then not seeing it in the delphi doc) :)

[the-Arioch]

yep, Delphi is the largest Pascal flavor but there, but not the only And FPC was (or wanted to be) a refuge for users of all other dying compilers/flavours

[the-Arioch]

tha is why FPC had those modes, presets tailored to mimic specific historic implementations So your attempt to test "otherwise" using Delphi mode of all else was funny, first reaction but totally wrong direction. Gnu or mac mode maybe could be a better first try. I did not check myself, but i would suspect "case-otherwise" still would work even in TP or Delphi mode, because is is a very local context, where namespace conflict with user-defined idents (vars, procs) is unthnkable of. Would thay be some global-scope keyword, i would then expect FPC to pro-actively hide it in Delphi mode, for the sake of the least required modification to make alien source code compile. I do not immediately remember any specific example though. Just used to move some code from Delphi once or twice. I believe (maybe wrongly) that there are two major "native pascal" flavors out there, ones that follow USCD-TP-Delphi road, and others that were mostly condensced into ANSI Extended Pascal. Then there are edge cases like DotNet (mostly RemObjects Oxygene and Pascal.ABC ) and JS/WebAsm (Smart Mobile Studio guess was the name of the latest incarnation) That said, i am not sure whether Oxygene should even be considered a very edgy Pascal, or a separate language of algol-modula-pascal-ada family. It started as Delphi.NET but their stated goal was to embrace DotNet to the fullest rather than keeping code compatibility, so probably not

[bolsen]

So your attempt to test "otherwise" using Delphi mode of all else was funny, first reaction but totally wrong direction. Gnu or mac mode maybe could be a better first try.

I figured that this wouldn't have been the best test, since it could still gladly parse unique keywords in the first place. But I think the Delphi documentation is more revealing, specifically the listing of keywords here: https://docwiki.embarcadero.com/RADStudio/Athens/en/Fundamental_Syntactic_Elements_(Delphi). But then, this means that Free Pascal (which seems to be the only other relevant Pascal compiler in my view) supports extra keywords. Supporting "Pascal" seems to support all its dialects (ugh, even if otherwise := 1 is used 😄 )

I suppose that otherwise is not in the same league here as specialize and the @ symbol, which is supported here, but worth to note that supporting "all dialects" is a thing. I saw in the wild, with preprocessor wrappers to allow these keywords. Ugh! 😄

  TStuffDictionary = {$ifdef FPC}specialize{$endif} TObjectDictionary<String, TStuff>;

This is kSpecialize

and :

Obj.DoThis({$ifdef FPC}@{$endif} Add);

This is kAt

Those are the normal ones. There is also a lot of keywords that really only apply to specific platforms or targets (objcclass, etc.) ... those look like strange additions, but what do I know? 😄

(For myself, I am just hobby-hacking in Pascal, so only use FP, so hated to see a chunk of code just not syntax-highlighting. I don't know how NeoVim handles it, but it is probably the same)

[the-Arioch]

Well, it certainly up to you to decide upon which body of sources your parser should work. For example, i never saw case-otherwise in real use, albeit heard of it.

Anyway, i guess i gave you something to digest and would you find some exotics other day you probably would have better leads to investigate :-)

[Isopod]

Thanks for your suggestion, by sheer coincidence I also stumbled across this not too long ago while I was looking at some older Pascal code. I actually already have a local branch supporting the otherwise keyword and a bunch of other compatibility improvements for older Pascal dialects. I was working on that earlier this year, but then life kinda got in the way and now the branch is in a half-finished state.

Thing is, I was working on a bunch of things at once, I also tried to make the parser more robust in case it encounters things it doesn't recognize (often due to the preprocessor, i.e. IFDEFS). That required writing a custom scanner, but then the problem was that the generated parser suddenly became huge because – apparently – Tree-Sitter does not deal with "external tokens" very efficiently. It didn't seem worth the bloat, so I experimented a lot (I wasn't initially sure what had even caused the parser to become so huge) and tried to find a compromise. Before that, I was also adding custom inline assembler support (which kind of lead me down the whole external scanner rabbithole in the first place, because that made the automatic error recovery in Tree-Sitter so much worse as Tree-Sitter suddenly thought everything it didn't recognize was assembler...)

I was in the process of cleaning all this up, so I was rebasing and cherry-picking commits left and right, but then I unfortunately had to leave it and now my local repository is in a quite messy state. I hope I can revisit that at some point as I did like some of the improvements, but it will probably take a while to untangle and it's not the highest priority right now (I also haven't really used Pascal in a while).

Thank you for the MR, but I'd rather not merge it like that as the problem with Tree-Sitter is that everything has side-effects, as I described earlier. In other dialects (which are today much more common) otherwise would be a valid identifier, and having this as a keyword elsewhere in the grammar could lead to really messy error recovery (see my assembler experience). This is one of the biggest shortcomings of Tree-Sitter imo. If we do add this, it should at the very least be behind a feature flag (like objc, fpc, delphi etc.) and it needs careful testing to make sure it doesn't cause regressions in more common modern code.

The reference material you dug up might come in handy if I ever do get back to it, however, so thank you for that.

[the-Arioch]

I think problematic error recovery is inevitable, because for what i

understood tree-sitter tries to avoid being top-down scanner, for the sake of recognizing half-written code, which is not correct yet. However this specific code pattern should be relatively safe, as it is a very specific place of the specific CASE construct, where nothing but keyword can appear anyway. This namespace extension should not leak out, should it? And sure, supporting ALL the possible Pascal extentions would probably be hard. Dialect flags would probably not be very helpful exactly because FPC seeks to integrate ans assimilate all the dialects, possibly mixing their special gimmicks in one file. And even in "one lineage" it would be complex. The USCD close extension is one example. Another could be inline keyword. In Delphi that is a parameterless function attribute (directive), but in TurboPascal this was pre-assembler thing, embedding executable binary opcodes into the code blocks. Like Inline( $90 / $cc / $cd / $20); Or the DOS-specific array Mem[ $0400 : $0009 ] := 10; Those non-standard separators were trivial when using hands-written parser, but not sure generic frameworks would appreciate all those special cases

…

Message ID: ***@***.***>

[Isopod]

Dialect flags would probably not be very helpful exactly because FPC seeks to integrate ans assimilate all the dialects, possibly mixing their special gimmicks in one file.

Well, at least one could disable it if it is not desired. It also serves as documentation by making it clearer which parts of the grammar come from where.

Another more selfish reason is that if I add more stuff that I already have in my own half-finished branch, it makes my life even harder if I ever want to get that stuff rebased.

Those non-standard separators were trivial when using hands-written parser, but not sure generic frameworks would appreciate all those special cases

True, but all of this pales in comparison to the problems the preprocessor creates. You would not think how many ways people can come up with to intersplice preprocessor directives with their code; you try to support one of those cases and it breaks another. All of those generic parsing frameworks are based on the theory of context-free grammars, and preprocessed languages are very much the opposite of that.

Still, I try to get at least somewhat reasonable syntax highlighting even in those cases. Wordy languages like Pascal are especially tricky because Tree-Sitter doesn't even have a notion of tokens (well... it sort of does, but it's complicated™), so every keyword it encounters could in theory also be a regular word. So it tends to want to highlight keywords like begin, end, procedure etc. as regular identifiers whenever it encounters a "syntax error". In C-like languages this is not as much of a problem, it has more guidance to correct itself and get back onto the right track. This is what I tried to correct with the custom scanner, but then all of a sudden it bloats the parser by 100x. You can't win.