Man page - toke(1)

Packages contains this manual

Manual

TOKE

NAME
SYNOPSIS
DESCRIPTION
OPTIONS
SEE ALSO
AUTHORS

NAME

toke - OpenBIOS tokenizer

SYNOPSIS

toke [options] forth-file

DESCRIPTION

toke is an open-source FCode Tokenizer from the OpenBIOS project.

OPTIONS

-h, -?

Print a brief help message and then exit.

-i, --ignore-errors

Generate a Binary Output even if errors were reported.

-l, --load-list

Collect the names of fload ed files into an FLoad-List file. The names collected are in the same form as they were presented in the fload statements.
The name of the FLoad-List File is derived from the name of the binary output file, by replacing its extension with .fl , or, if the binary output file name had no extension, merely appending the extension .fl .
The binary output file name used for this purpose is either the one specified on the command line, or the one created by default.

-o, --output-name OutputFileName ,

Direct the binary output (FCode result of Tokenization) to the named file instead of to the default-named file. This option is not valid when multiple input files are named.

-P, --dependencies

Collect the fully-resolved pathnames of floaded and ENCODEd files into a dependency-list file. The names collected are in the form that is presented to the host operating system: shell environment Variables and related expressions will be fully expanded, and the directory within the include-list in which the file was found will be attached.
The name of the dependency-list file will be the same as that of the FLoad-list file, except that its extension will be .P instead of .fl .

-v, --verbose

Print additional messages (including advisories) during tokenization.

-I, --Include directory

This tokenizer supports the notion of an include-list. The user creates the include-list by specifying a number of -I directory pairs on the command-line. All file-reads, whether for an fload command or an encode-file directive, will involve a search for the named file through the directories of the include-list, in the order they were supplied on the command-line.
If no include-list is created, file-reads are relative to the current working Directory. If an include-list is created, file-reads are restricted to the directories within it. For the current working directory to be included in the file-search, it must be specified explicitly. -I. will accomplish that quite effectively.

-T, --Trace symbol

This tokenizer supports the notion of a trace-list. The User creates the trace-list by specifying a number of -T symbol pairs on the command-line.
When a name is defined, whether as an FCode, an alias, a Macro or anything else, either in normal tokenization mode or "Tokenizer Escape"-mode, if it matches a symbol that has been added to the trace list, a trace note message will be issued indicating that a definition of that name has been created. Subsequent trace note Messages will be issued when the definition of that name is invoked.
This trace-symbols feature can be helpful during maintenance of legacy code, for instance, when multiple symbols carry the same name.

-d, --define Symbol[=Value]

Define a command-line symbol and optionally, assign a value to it. If you wish the value to contain spaces or quotes, you can accomplish that using the shell escape conventions. This sequence may be repeated. Once a Symbol is defined on the command-line, it stays in effect for the duration of the entire batch of tokenizations (i.e., if there are multiple input files named on the command line). Command-line Symbols can be tested for purposes of conditional tokenization, or their assigned values can be evaluated.

-f, --flag [no]<FlagName>

The tokenizer recognizes a specific set of special-feature flag-names; each is associated with a specific non-standard variant behavior. Pass the flag-name as an argument to the -f switch to enable the behavior; to disable it, precede the flag-name with the optional string no .
The settings of the special-feature flags can also be changed or displayed from within the source input file.
The special-feature flags are all initially set to be enabled, except where noted.
The flag-names and their associated special-features are as follows:
Local-Values

Support IBM-style Local Values ("LV"s). Initially disabled.

LV-Legacy-Separator

Allow Semicolon for Local Values Separator ("Legacy").

LV-Legacy-Message

Display a Warning Message when Semicolon is used as the Local Values Separator.

ABORT-Quote

Allow ABORT" macro.

Sun-ABORT-Quote

ABORT" with implicit IF ... THEN

Abort-Quote-Throw

Use -2 THROW, rather than ABORT, in an Abort" phrase

String-remark-escape

Allow "\ (Quote-Backslash) to interrupt string parsing.

Hex-remark-escape

Allow \ (Backslash) to interrupt hex-sequence parsing within a string.

C-Style-string-escape

Allow the C-style String-Escape pairs \n \t and \xx\ to be treated as special characters in string parsing.

Always-Headers

Override occurrences of the standard directive headerless in the source with -- effectively -- headers to make all definitions have a header. Occurrences of the directive external will continue to behave in the standard manner. Initially disabled.

Always-External

All definitions will be made as though under the external directive; occurrences of either Standard directive headerless or headers in the source will be overridden. This special-feature flag will also override the Always-Headers special-feature flag in the event that both have been specified. Initially disabled.

Warn-if-Duplicate

Display a WARNING message whenever a definition is made whose name duplicates that of an existing definition. Disabling this flag will suspend the duplicate-names test globally, until it is re-enabled. A Directive is supported that will suspend the test for the duration of only a single definition, without affecting global behavior.

Obsolete-FCode-Warning

Display a WARNING message whenever an FCode function is invoked that the Standard identifies as obsolete.

Trace-Conditionals

Issue advisory messages about the state of conditional tokenization. (Remember that advisory messages are displayed only if the verbose option -v is set.) Initially disabled.

Upper-Case-Token-Names

Lower-Case-Token-Names When outputting the names of headered functions ( token-names ) to the binary output file, override the character-case in which the names appeared in the source, and convert them to Upper- or Lower- -Case, respectively. (These flags do not affect text string sequences, whose character-case is always preserved.) Initially disabled.

Big-End-PCI-Rev-Level

Save the Revision Level of the Vendor’s ROM field of the PCI Header in big-endian byte-order, rather than little-endian as per the general PCI Standard convention. (This flag does not affect any other field
of the PCI Header). Initially disabled.

Ret-Stk-Interp

Allow return-stack operations during interpretation. While the standard specifies that usage of the operators >r , r@ , and r> while interpreting is allowed, actual practice in the industry is inconsistent. Developers who wish to take a more cautious approach to this question can disable this flag so that any attempt to use the operators >r , r@ , and r> in the interpreting state will generate an ERROR Message.

Also, the pseudo-flag-name help will cause a list of the flag-names and their associated special-features to be printed.

SEE ALSO

detok (1).

AUTHORS

Stefan Reinauer <stepan@openbios.org>
David L. Paktor <dlpaktor@us.ibm.com>

This manual page was written by Aurelien Jarno <aurel32@debian.org> for the Debian GNU/Linux system (but may be used by others).