Shell Syntax

Shell Syntax

Shell command parsing with MuParser and argument types.

Overview

Shell Syntax is the parsing and evaluation infrastructure that powers JNode's command-line interface and scripting capabilities. At its core, it consists of two main systems: a high-level declarative syntax framework (Syntax, Argument, SyntaxBundle) that commands use to describe expected arguments, and a low-level backtracking parser (MuParser) that validates user input against those descriptions. This design separates the "what arguments are expected" from the "how to parse them", making it easy to define complex command-line interfaces without writing parsing code.

The system is centered in the org.jnode.shell.syntax package (~200 references) and is built around the plugin system, allowing commands and their syntaxes to be registered declaratively. When a user invokes a shell command, the flow is: SyntaxManager locates the command's SyntaxBundle, the bundle's Syntax trees are compiled into MuSyntax graphs, MuParser performs a backtracking parse of the user's input tokens, and the resulting ArgumentBundle is populated with validated values before execute() is called.

Key Components

Class / File	Role
shell/src/shell/org/jnode/shell/syntax/SyntaxManager.java	Maps command aliases to SyntaxBundles; manages registration and lookup
shell/src/shell/org/jnode/shell/syntax/MuParser.java	Backtracking parser; validates and populates arguments from token stream
shell/src/shell/org/jnode/shell/syntax/Syntax.java	Abstract syntax tree node (SequenceSyntax, OptionSyntax, RepeatSyntax, etc.)
shell/src/shell/org/jnode/shell/syntax/Argument.java	Base class for typed argument holders (FileArgument, StringArgument, FlagArgument, etc.)
shell/src/shell/org/jnode/shell/syntax/SyntaxBundle.java	Container holding one or more Syntax alternatives for a command alias
shell/src/shell/org/jnode/shell/syntax/ArgumentBundle.java	Container holding Argument instances for a command; populated by MuParser

How It Works

Command Registration

Commands register their syntax via the plugin system using the org.jnode.shell.syntaxes extension point. The SyntaxManager interface (NAME = SyntaxManager.class) is registered in the InitialNaming namespace and provides:

• add(SyntaxBundle bundle) — register syntax bundles embedded in commands
• add(String alias, ArgumentSpec<?>[] argSpecs) — register specs for non-native commands
• getSyntaxBundle(String alias) / getArgumentBundle(String alias) — lookup by command alias
• createSyntaxManager() — create child managers for nested scopes

Syntax Compilation

Each Syntax subclass (e.g., SequenceSyntax, OptionSyntax, AlternativesSyntax, RepeatSyntax) implements prepare(ArgumentBundle bundle) which produces a MuSyntax tree:

• MuSymbol — matches literal tokens (e.g., -o, --force)
• MuArgument — matches and validates arguments against an Argument instance
• MuPreset — injects preset values for hidden/fixed arguments
• MuSequence — sequences of elements processed left-to-right
• MuAlternation — alternatives explored via backtracking

MuParser Backtracking

MuParser is the runtime engine. It uses a deque-based syntax stack and a backtrack stack of ChoicePoint entries. The parsing loop:

1. Pops the next MuSyntax from the stack
2. Matches symbol tokens (MuSymbol) or validates against arguments (MuArgument)
3. For MuArgument, calls Argument.accept(token, flags) which delegates to doAccept() for type conversion and validation
4. On failure, pops the next alternative from the current ChoicePoint, restores the syntax stack and source position, and undoes argument modifications via undoLastValue()
5. On success, all arguments are populated in the ArgumentBundle

The DEFAULT_STEP_LIMIT is 10000 parse steps to guard against pathological grammars.

Argument Types

The Argument<V> class hierarchy provides typed, validated argument holders. Key types include StringArgument, IntegerArgument, LongArgument, FileArgument, HostNameArgument, PortNumberArgument, FlagArgument, and EnumArgument. Each implements doAccept(Token) for type conversion and complete() for tab-completion suggestions.

Argument flags control parsing behavior:

Flag	Meaning
MANDATORY	At least one value must be supplied
OPTIONAL	May be omitted (default)
MULTIPLE	Zero or more values allowed
EXISTING	Value must denote an existing entity
NONEXISTENT	Value must NOT exist

Tab-Completion

The same MuParser operates in completion mode when completions is non-null. Instead of throwing on parse failure, it explores all alternatives and calls Argument.complete() on partial tokens to produce suggestions (e.g., FileArgument lists directory entries).

Gotchas & Non-Obvious Behavior

• Backtracking modifies argument state — MuParser records which arguments were modified and calls undoLastValue() when backtracking; do not perform side effects in doAccept()
• Step limit prevents infinite loops — grammars with complex alternations can exhaust the DEFAULT_STEP_LIMIT; adjust via the stepLimit parameter if needed
• EXISTING and NONEXISTENT are not logical negations — they check different validation properties; a file can be non-existent for creation but inaccessible for reading
• Labels must be unique within an ArgumentBundle; the parser matches labels across syntax tree nodes to argument instances
• SharedStack avoids copying — SharedStack is used instead of LinkedList for the syntax stack to reduce allocation overhead during backtracking

Related Pages

• Syntax — Deep-dive documentation of the full syntax framework, Argument types, and parsing internals
• Shell-Commands — Parent hub for the command framework
• Plugin-System — How commands and syntaxes are registered via extensions
• CLI — Command-line interface concepts