and contains the next tokens:
</body>
<list><item>
- <symbol> a non-empty sequence of the 66 characters + - . 0..9 A..Z _ a..z <newline/>
- this sequence can represents the identifiers and the numerical constants of most programming languages
+ <ebnf>
+ <prod of="symbol"/> <def/>
+ <prod of="symbol_char"/> <prod of="symbol_char"/> <star/>
+ <stop/> <newline/>
+ <prod of="symbol_char"/> <def/>
+ <chr of="+"/> <or/> <chr of="-"/> <or/> <chr of="."/> <or/>
+ <chr of="0"/> <or/> <etc/> <or/> <chr of="9"/> <or/>
+ <chr of="A"/> <or/> <etc/> <or/> <chr of="Z"/> <or/>
+ <chr of="_"/> <or/> <chr of="a"/> <or/> <etc/> <or/> <chr of="z"/>
+ <stop/>
+ </ebnf> <newline/>
+ this token can represent the identifiers and the numerical constants of most programming languages;
</item><item>
- <string> a sequence of characters wrapped with the delimiter " <newline/>
+ <ebnf>
+ <prod of="string"/> <def/>
+ <chr of="""/> <open/>
+ <prod of="string_char"/> <or/>
+ <chr of="\"/> <prod of="escape"/>
+ <close/> <star/> <chr of="""/>
+ <stop/>
+ </ebnf> <newline/>
every valid UTF-8 character whose code point is greater than U+001F is accepted eccept " \ <newline/>
the next commonly accepted escape sequences are recognized: \0 \a \b \t \n \v \f \r \e \" \\ <newline/>
morover, the escape sequences \x <two hexadecimal digits> and \u <four hexadecimal digits>
allow to specify a character by its code point <newline/>
finally the escape sequences \( for U+0002 and \) for U+0003 are available
</item><item>
- <qualifier> the character :
+ <ebnf>
+ <prod of="qualifier"/> <def/> <chr of=":"/> <stop/>
+ </ebnf>
</item><item>
- <left> each of the characters ( < [ { <newline/>
- this token starts a compound symbolic expression
+ <ebnf>
+ <prod of="left"/> <def/>
+ <chr of="("/> <or/> <chr of="<"/> <or/> <chr of="["/> <or/> <chr of="{"/>
+ <stop/>
+ </ebnf> <newline/>
+ this token starts a compound symbolic expression;
</item><item>
- <right> each of the characters ) > ] } <newline/>
- this token ends a compound symbolic expression
+ <ebnf>
+ <prod of="right"/> <def/>
+ <chr of=")"/> <or/> <chr of=">"/> <or/> <chr of="]"/> <or/> <chr of="}"/>
+ <stop/>
+ </ebnf> <newline/>
+ this token ends a compound symbolic expression;
</item><item>
<ignored> each of the characters U+0009..U+000D U+0020 , ; = <newline/>
these characters are ignored and separate the other tokens