Fork: 0
tundra / tconfpy
Browse code
Added sections on namespace processing.
Formatting cleanup.
Changed API argument from InitialSymTbl to InitialSymTable.
master
1 parent fa51c45 commit 34a451ef5c7ee4e0086f4d2c0651ca45d7448110
@tundra tundra authored on 8 Apr 2004
Patch
Unified Split
Showing 1 changed file
View
827
■■■■■
 tconfpy.3
brevity, the code examples here assume the following Python import
syntax:
 
.nf
from tconfpy import *
from tconfpy import *
.fi
 
If you prefer the more pedestrian:
 
.nf
import tconfpy
import tconfpy
.fi
 
you will have to prepend all references to a \*(TC object with
\'tconfpy.\'. So \'retval=ParseConfig(...\' becomes
to be processed is a required parameter, all the others are optional
and default as described below:
 
.nf
from tconfpy import *
 
retval = ParseConfig(cfgfile,
InitialSymTbl={},
AllowNewVars=True,
AllowNewNamespaces=True,
Debug=False,
LiteralVars=False
)
from tconfpy import *
 
retval = ParseConfig(cfgfile,
InitialSymTable={},
AllowNewVars=True,
AllowNewNamespaces=True,
Debug=False,
LiteralVars=False
)
 
where:
 
.fi
 
The the name of a file containing configuration information
 
.TP
.B InitialSymTbl (Default: {})
.B InitialSymTable (Default: {})
 
 
A pre-populated symbol table (a Python dictionary). As described
below, this must contain valid \'VarDescriptor\' entries for each
 
An object of type \'tconfpy.RetObj\' used to return parsing results.
 
 
.SS Reasons For Passing An Initial Symbol Table
.SS The Initial Symbol Table API Option
 
The simplest way to parse a configuration file is just to call
the parser with the name of that file:
 
 
.nf
retval = ParseConfig("myconfigfile")
retval = ParseConfig("MyConfigFile")
.fi
 
Assuming your configuration file is valid, \'ParseConfig()\' will
return a symbol table populated with all the variables defined in the
table to the parser that contains pre-defined symbols/values of
your own choosing. Why on earth would you want to do this? There
are a number of reasons:
 
.nf
1) You may wish to write a configuration file which somehow depends
on a pre-defined variable that only the calling program can know:
 
.if [APPVERSION] == 1.0
# Set configuration for older application releases
.else
# Set configuration for newer releases
.endif
 
In this example, only the calling application can know its own
version, so it sets the variable APPVERSION in a symbol table
which is passed to \'ParseConfig()\'.
 
2) You may wish to "protect" certain variable names be creating
them ahead of time and marking them as "Read Only". This is
useful when you want a variable to be available for use
within a configuration file, but you do not want users to
be able to change its value. In this case, the variable can
be referenced in a string substitution or conditional test,
but cannot be changed.
 
3) You may want to place limits on what values can be assigned to
a particular variable. When a variable is newly defined in a
a configuration file, it just defaults to being a string
variable without any limits on its length or content. But
variables that are created by a program have access to the
variable's "descriptor". By setting various attribues
of the variable descriptor you can control variable type,
content, and range of values. In other words, you can have
\*(TC "validate" what values the user assigns to particular
variables. This substantially simplifies your application because
no invalid variable value will ever be returned from the parser.
 
.fi
 
.IP \(bu 4
You may wish to write a configuration file which somehow depends
on a pre-defined variable that only the calling program can know:
 
.nf
.if [APPVERSION] == 1.0
# Set configuration for older application releases
.else
# Set configuration for newer releases
.endif
.fi
 
In this example, only the calling application can know its own
version, so it sets the variable APPVERSION in a symbol table
which is passed to \'ParseConfig()\'.
 
.IP \(bu 4
You may wish to "protect" certain variable names be creating
them ahead of time and marking them as "Read Only". This is
useful when you want a variable to be available for use
within a configuration file, but you do not want users to
be able to change its value. In this case, the variable can
be referenced in a string substitution or conditional test,
but cannot be changed.
 
.IP \(bu 4
You may want to place limits on what values can be assigned to
a particular variable. When a variable is newly defined in a
a configuration file, it just defaults to being a string
variable without any limits on its length or content. But
variables that are created by a program have access to the
variable's "descriptor". By setting various attribues
of the variable descriptor you can control variable type,
content, and range of values. In other words, you can have
\*(TC "validate" what values the user assigns to particular
variables. This substantially simplifies your application because
no invalid variable value will ever be returned from the parser.
 
 
.SS How To Create An Initial Symbol Table
 
A \*(TC "Symbol Table" is really nothing more than a Python
descriptor". Creating new variables in the symbol table involves
nothing more than this:
 
.nf
 
from tconfpy import *
 
# Create an empty symbol table
MySymTable = {}
# Create descriptor for new variable
MyVarDes = VarDescriptor()
 
# Code to fiddle with descriptor contents goes here
MyVarDes.Value = "MyVal"
 
# Now load the variable into the symbol table
MySymTable["MyVariableName"] = MyVarDes
from tconfpy import *
# Create an empty symbol table
MySymTable = {}
# Create descriptor for new variable
MyVarDes = VarDescriptor()
# Code to fiddle with descriptor contents goes here
MyVarDes.Value = "MyVal"
 
# Now load the variable into the symbol table
MySymTable["MyVariableName"] = MyVarDes
 
# Repeat this process for all variables, then call the parser
 
retval = ParseConfig("MyConfigFile", InitialSymTable=MySymTable)
 
.fi
 
 
it matches the type declared for that variable. For example,
suppose you did this when defining the variable, \'foo\':
 
.nf
VarDescriptor.Type = TYPE_INT
VarDescriptor.Type = TYPE_INT
.fi
 
Now suppose the user puts this in the configuration file:
 
.nf
foo = bar
foo = bar
.fi
 
This will cause a type mismatch error because \'bar\' cannot be coerced
into an integer type - it is a string.
That's what this attribute is for. \'LegalVals\' explictly lists
every legal value for the variable in question. If the list is
empty,then this validation check is skipped.
 
.B IMPORTANT:
If you change the content of \'LegalVals\', make sure it is always a
Python list. \*(TC's validation logic presumes this attribute
to be a list and will blow up nicely if it is not.
 
The exact semantics of LegalVals varies depending on
the type of the variable.
 
.nf
observe case and test for either \'True\' or \'False\':
 
.nf
 
.if [boolvar] != False # This works fine
 
.if [boolvar] != FALSE # This does not work - Case is not being observed
 
.if [boolvar] != Off # Neither does this - Only True and False can be tested
 
.fi
.if [boolvar] != False # This works fine
 
.if [boolvar] != FALSE # This does not work - Case is not being observed
 
.if [boolvar] != Off # Neither does this - Only True and False can be tested
 
.fi
 
 
.SS The Initial Symbol Table And Lexical Namespaces
 
The
.B CONFIGURATION LANGUAGE REFERENCE
section below discusses lexical namespaces in some detail from the
user's point-of-view. However, it is useful for the programmer
to understand how they are implemented.
 
\*(TC is written to use a pre-defined variable named \'NAMESPACE\' as
the place where the current namespace is kept. If you do not
define this variable in the initial symbol table passed to the parser,
\*(TC will create it automatically with an initial value of "".
 
From a programmer's perspective, there are are few important things
to know about namespaces and the \'NAMESPACE\' variable:
 
 
.IP \(bu 4
You can manually set the initial namespace to something other than
"". You do this by creating the \'NAMESPACE\' variable in the initial
symbol table passed to the parser, and setting the \'Value\' attribute
of its descriptor to whatever you want as the initial namespace.
 
.IP \(bu 4
Each time the user enters a new namespace, it is added to the
\'LegalVals\' attribute of the \'NAMESPACE\' variable descriptor.
This is done to keep a "history" of all namespaces encountered during
parsing.
 
.IP \(bu 4
If you are inhibiting the creation of new namespaces (via the
\'AllowNewNamespaces = False\' option described below) you can
enumerate the namespaces the user
.B is
allowed to use by pre-defining them. You do this by adding each
permitted namespace to the \'LegalVals\' attribute of the
\'NAMESPACE\' variable descriptor in the initial symbol table.
 
.IP \(bu 4
When the call to \'ParseConfig()\' completes, the \'Value\'
attribute of the \'NAMESPACE\' variable descriptor will contain
the namespace that was in effect when the parse completed. i.e.,
It will contain the last namespace used.
 
.IP \(bu 4
Similarly, on parser completion, the \'LegalVals\' attribute of the
\'NAMESPACE\' variable descriptor will contain the name of all the
namespaces encountered during parsing. This is
.B not
true if you suppressed new namespace creation with the
\'AllowNewNamespaces = False\' API option. In that case, the
\'LegalVals\' attribute will contain only those namespaces you
pre-defined. All of these may- or may not have been used in the
configuration file, there is no way to tell.
 
 
.SS How The \*(TC Parser Validates The Initial Symbol Table
 
to be recorded. Similarly, if you put a value in \'LegalVals\' that
is outside the range of \'Min\' to \'Max\', \*(TC will accept
it quietly.
 
.SS The \'AllowNewVars\' Option
.SS The \'AllowNewVars\' API Option
 
By default, \*(TC lets the user define any new variables they
wish in a configuration file, merely by placing a line in the
file in this form:
 
.nf
Varname = Value
Varname = Value
.fi
 
However, you can disable this capability by calling the parser like
this:
 
.nf
retval = ParseConfig("myconfigfile", AllowNewVars=False)
retval = ParseConfig("myconfigfile", AllowNewVars=False)
.fi
 
This means that the configuration file can "reference"
any pre-defined variables, and even change their values
symbol table to the parser and you do not want any other variables
defined by the user. Why? There are several possible uses for
this option:
 
.nf
 
1) You know every configuration variable name the calling program
will use ahead of time. Disabling new variable names keeps
the configuration file from getting cluttered with variables
that the calling program will ignore anyway, thereby
keeping the file more readable.
 
2) You want to insulate your user from silent errors caused
by misspellings. Say your program looks for a configuration
variable called \'MyEmail\' but the user enters something
like \'myemail = foo@bar.com\'. \'MyEmail\' and \'myemail\'
are entirely different variables and only the former is
recognized by your calling program. By turning off new
variable creation, the user's inadvertent misspelling of
the desired variable name will be flagged as an error.
 
.fi
.IP \(bu 4
You know every configuration variable name the calling program
will use ahead of time. Disabling new variable names keeps
the configuration file from getting cluttered with variables
that the calling program will ignore anyway, thereby
keeping the file more readable.
 
.IP \(bu 4
You want to insulate your user from silent errors caused
by misspellings. Say your program looks for a configuration
variable called \'MyEmail\' but the user enters something
like \'myemail = foo@bar.com\'. \'MyEmail\' and \'myemail\'
are entirely different variables and only the former is
recognized by your calling program. By turning off new
variable creation, the user's inadvertent misspelling of
the desired variable name will be flagged as an error.
 
Note, however, that there is one big drawback to disabling new
variable creation. \*(TC processes the configuration file on a
line-by-line basis. No line "continuation" is supported. For really
example:
 
.nf
 
inter1 = Really, really, really, really, long argument #1
inter2 = Really, really, really, really, long argument #2
 
realvar = command [inter1] [inter2]
inter1 = Really, really, really, really, long argument #1
inter2 = Really, really, really, really, long argument #2
 
realvar = command [inter1] [inter2]
.fi
 
If you disable new variable creation you can't do this
anymore unless all the variables \'inter1\', \'inter2\',
and \'realvar\' are predefined in the initial symbol
and \'realvar\' are pre-defined in the initial symbol
table passed to the parser.
 
 
.SS The \'AllowNewNamespaces\' Option
.SS The \'AllowNewNamespaces\' API Option
 
By default, \*(TC supports the use of an arbitrary number of
lexical namespaces. They can be predefined in an initial
lexical namespaces. They can be pre-defined in an initial
symbol table passed to the parser and/or created in the configuration
file as desired. (The details are described in a later section
of this document.)
 
these pre-defined namespaces is available for use throughout the
configuration file even if \'AllowNewNamespaces\' is set to False.
 
 
.SS The \'Debug\' Option
.SS The \'Debug\' API Option
 
\*(TC has a fairly rich set of debugging features built into its
parser. It can provide some detail about each line parsed as well
as overall information about the parse. Be default, debugging is
turned off. To enable debugging, merely set \'Debug=True' in the
API call:
 
.nf
retval = ParseConfig("myconfigfile", Debug=True)
.fi
 
.SS The \'LiteralVars\' Option
retval = ParseConfig("myconfigfile", Debug=True)
.fi
 
.SS The \'LiteralVars\' API Option
 
\*(TC supports the inclusion of literal text anywhere in a
configuration file via the \'.literal\' directive. This
directive effectively tells the \*(TC parser to pass
 
Here is an example:
 
.nf
MyEmail = me@here.com # This defines variable MyEmail
MyEmail = me@here.com # This defines variable MyEmail
 
.literal
printf("[MyEmail]"); /* A C Statement */
.endliteral
literal blocks by setting \'LiteralVars=True\' in the
\'ParseConfig()\' call:
 
.nf
retval = ParseConfig("myconfigfile", LiteralVars=True)
retval = ParseConfig("myconfigfile", LiteralVars=True)
.fi
 
 
In this example, \*(TC would return:
nothing more than a container to hold return data. In the simplest
case, we can parse and extract results like this:
 
.nf
from tconfpy import *
 
retval = ParseConfig("myconfigfile", Debug=True)
from tconfpy import *
 
retval = ParseConfig("myconfigfile", Debug=True)
.fi
 
\'retval\' now contains the results of the parse:
 
.nf
retval.Errors - A Python list containing error messages. If this list
is empty, you can infer that there were no parsing
errors - i.e., The configuration file was OK.
 
retval.Warnings - A Python list containing warning messages. These
describe minor problems not fatal to the parse
process, but that you really ought to clean up in
the configuration file.
 
 
retval.SymTable - A Python dictionary which lists all the defined
symbols and their associated values. A "value"
in this case is always an object of type
tconfpy.VarDescriptor (as described above).
 
retval.Literals - As described below, the \*(TC configuration language
supports a \'.literal\' directive. This directive
allows the user to embed literal text anywhere in
the configuration file. This effectively makes
\*(TC useful as a preprocessor for any other
language or text. retval.Literals is a Python list
containing all literal text discovered during the parse.
 
retval.Debug - A Python list containing detailed debug information for
each line parsed as well as some brief summary
information about the parse. retval.Debug defaults
to an empty list and is only populated if you set
\'Debug=True\' in the API call that initiated the
parse (as in the example above).
.fi
.TP
.B retval.Errors
A Python list containing error messages. If this list is empty, you
can infer that there were no parsing errors - i.e., The configuration
file was OK.
 
.TP
.B retval.Warnings
A Python list containing warning messages. These describe minor
problems not fatal to the parse process, but that you really ought to
clean up in the configuration file.
 
 
.TP
.B retval.SymTable
 
A Python dictionary which lists all the defined symbols and their
associated values. A "value" in this case is always an object of type
tconfpy.VarDescriptor (as described above).
 
.TP
.B retval.Literals
 
As described below, the \*(TC configuration language supports a
\'.literal\' directive. This directive allows the user to embed
literal text anywhere in the configuration file. This effectively
makes \*(TC useful as a preprocessor for any other language or text.
retval.Literals is a Python list containing all literal text
discovered during the parse.
 
.TP
.B retval.Debug
 
A Python list containing detailed debug information for each line
parsed as well as some brief summary information about the parse.
retval.Debug defaults to an empty list and is only populated if you
set \'Debug=True\' in the API call that initiated the parse (as in the
example above).
 
 
.SH CONFIGURATION LANGUAGE REFERENCE
 
operators. However, there are some places where
whitespace matters:
 
.nf
- Variable names may not contain whitespace
 
- Directives must be followed by whitespace if
they take other arguments.
 
- When assigning a value to a string variable,
whitespace within the value on the right-hand-side
is preserved. Leading- and trailing whitespace
around the right-hand-side of the assignment is
ignored.
- Variable names may not contain whitespace
 
- Directives must be followed by whitespace if they
take other arguments.
 
- When assigning a value to a string variable,
whitespace within the value on the right-hand-side
is preserved. Leading- and trailing whitespace around
the right-hand-side of the assignment is ignored.
 
- Whitespace within both the left- and right-hand-side
arguments of a conditional comparison
(\'.if ... == / != ...\') is significant for purposes
of the comparison.
.fi
 
.IP \(bu 4
Case is always significant except when assigning a value to Booleans
.IP \(bu 4
Regardless of a variable's type, all variable references
return
.B a string representation of the variable's value!
This is done so that the variable's value can be used for
comparison testing and string substitution/concatenation.
In other words, variables are stored in their native type
in the symbol table that is returned to the calling program,
but they are treated as strings throughout the configuration
file.
 
 
 
.SS Variables And Variable References
This is done so that the variable's value can be used for comparison
testing and string substitution/concatenation. In other words,
variables are stored in their native type in the symbol table that is
returned to the calling program. However, they are treated as strings
during the parsing of the configuration file whenever they are used in
a comparison test or in a substitution.
 
.IP \(bu 4
Any line which does not conform to these rules and/or is not
in the proper format for one of the operations described below,
is considered an error.
 
 
.SS Creating Variables And Assigning A Value
 
The heart of a configuration file is a "variable". Variables are
stored in a "Symbol Table" which is returned to the calling program
once the configuration file has been processed. The calling program
 
Variables are assigned values like this:
 
.nf
MyVariable = Some string of text
MyVariable = Some string of text
.fi
 
If \'MyVariable\' is a new variable, \*(TC will create it on the spot.
If it already exists, \*(TC will first check and make sure that
\'Some string of text\' is a legal value for this variable.
If not, it will produce an error and refuse to change the current
value of \'MyVariable\'
 
You can get the value of any currently defined variable by
"referencing" it like this:
 
.nf
.... [MyVariable] ...
.fi
 
The brackets surrounding any name are what indicate that you want that
variable's value.
 
You can also get the value of any Environment Variable on your
system by naming the variable with a leading \'$\':
 
.nf
... [$USER] ... # Gets the value of the USER environment variable
.fi
 
However you cannot set the value of an environment variable:
 
.nf
$USER = me # This is not permitted
.fi
If it already exists, \*(TC will first check and make sure that \'Some
string of text\' is a legal value for this variable. If not, it will
produce an error and refuse to change the current value of
\'MyVariable\'.
 
Anytime you create a new variable, the first value assigned to
it is also considered its "default" value. This may (or may
not) be meaningful to the application program.
 
Variables which are newly-defined in a configuration file are
always understood to be
.B string
variables - i.e., They hold "strings" of text. However, it is
possible for the applications programmer to pre-define
variables with other types and place limitations on what values
the variable can take and/or how short or long a string variable
may be. (See the previous section,
.B PROGRAMMING USING THE \*(TC API
for all the gory details.)
 
 
.SS Variable Names
 
Variables can be named pretty much anything you like, with certain
restrictions:
 
Variable names cannot have the \'#\' character anywhere in them
because \*(TC sees that character as the beginning a comment.
 
.IP \(bu 4
You cannot have a variable with no name. This is illegal:
 
.nf
= String
.fi
 
 
 
.SS Predefined Variables
You cannot have a variable whose name is the empty string. This is
illegal:
 
.nf
= String
.fi
 
 
.SS Getting And Using The Value Of A Variable
 
You can get the value of any currently defined variable by
"referencing" it like this:
 
.nf
.... [MyVariable] ...
.fi
 
The brackets surrounding any name are what indicate that you want that
variable's value.
 
You can also get the value of any Environment Variable on your
system by naming the variable with a leading \'$\':
 
.nf
... [$USER] ... # Gets the value of the USER environment variable
.fi
 
However you cannot set the value of an environment variable:
 
.nf
$USER = me # This is not permitted
.fi
 
 
This ability to both set and retrieve variable content makes it easy
to combine variables through "substitution":
 
.nf
 
MYNAME = Mr. Tconfpy
MYAGE = 101
 
Greeting = Hello [MYNAME], you look great for someone [MYAGE]!
 
.fi
 
 
Several observations are worth noting here:
 
.IP \(bu 4
The substitution of variables takes place as soon as the parser
processes the line \'Greeting = ...\'. That is, variable substitution
happens as it is encountered in the configuration file. The only
exception to this is if an attempt is made to refer to an
undefined/non-existent variable. This generates an error.
 
.IP \(bu 4
Unless a variable as been marked as "Read Only" by the program,
you can continue to change its value as you go. Simply adding
another line at the end of our example above will change the value
of \'Greeting\' to something new:
 
.nf
Greeting = Generic Greeting Message
.fi
 
In other words, the last assignment statement for a given variable
"wins". This may seem sort of pointless, but it actually has great
utility. You can use the \'.include\' directive to get, say, a
"standard" configuration provided by the system administrator for a
particular application. You can then selectively override the
variables you want to change in your own configuration file.
 
.IP \(bu 4
The variable \'Greeting\' now contains the
.B string
"Hello Mr. Tconfpy, you look great for someone 101!" This is true even
if variable \'MYAGE\' has been defined by the calling program to be
an integer type. To repeat a previously-made point: All variable substitution
and comparison operations in a configuration file are done with
.B strings
regardless of the actual type of the variables involved.
 
 
.SS Lexical Namespaces
 
The discussion of variable assignment and reference needs to be
expanded now to include the idea of "lexical namespaces". Think of a
namespace as just being the current context for variables - a way to
group variables together. The idea is that a variable name or
reference is always relative to the current namespace. That is, the
"real" name of the variable is the concatenation of the current
namespace with the variable's name.
 
For instance, suppose the current namespace is "MYNS" and we see the
following in a configuration file:
 
.nf
Foo = Bar
Baz = [Foo]
.fi
 
What this
.B really
means to \*(TC is this:
 
.nf
MYNS.Foo = MYNS.Bar
MYNS.Baz = [MYNS.Foo]
.fi
 
In other words, \*(TC always prepends the current namespace (plus a
separating period) to any variable assignment or reference it finds in
a configuration. The one exception is when the namespace is "", in
which case \*(TC does not add anything to the variable names.
 
By default, the initial namespace is always "", but you can change this
as desired. Note, however, that the programmer can change this default
initial namespace if they wish. Also there is a programming option
(\'AllowNewNamespaces\', described in the previous section) which can limit
you to using only the namespaces the programmer has pre-defined. i.e., You
will not be able to create
.B new
namespaces of your own invention.
 
You can change to a new namespace anytime within the configuration file
a number of ways:
 
 
So how do you access variables in a namespace different than the one
you are in currently?
 
 
What's the point of all this namespace business anyway?
 
 
.SS Pre-Defined Variables
 
.SS The \'.include\' Directive
 
.SS Conditional Directives
 
.SS \'.literal\. - Using \*(TC As A Preprocessor For Other Languages
 
.SS Type And Value Enforcement
 
.SS Lexical Namespaces
 
.SH ADVANCED TOPICS
 
.SS Guaranteeing A Correct Base Configuration
For the first two installation methods, you must first download the
latest release from:
 
.nf
http://www.tundraware.com/Software/tconfpy/
http://www.tundraware.com/Software/tconfpy/
.fi
 
Then unpack the contents by issuing the following command:
 
.nf
tar -xzvf py-tconfpy-X.XXX.tar.gz (where X.XXX is the version number)
tar -xzvf py-tconfpy-X.XXX.tar.gz (where X.XXX is the version number)
.fi
 
Win32 users who do not have tar installed on their system can find
a Windows version of the program at:
 
Make sure you are logged in as root, then:
 
.nf
cd /usr/ports/devel/py-tconfpy
make install
cd /usr/ports/devel/py-tconfpy
make install
.fi
 
This is a fully-automated install that puts both code and
documentation where it belongs. After this command has completed
you'll find the license agreement and all the documentation (in the
various formats) in:
 
.nf
/usr/local/share/doc/py-tconfpy
/usr/local/share/doc/py-tconfpy
.fi
 
The 'man' pages will have been properly installed so either of these
commands will work: