.ds CP 2003-2004
.ds TC \'tconfpy\'
.TH TCONFPY 3 "TundraWare Inc."
.SH NAME tconfpy.py
Configuration File Support For Python Applications
.SH SYNOPSIS
It is common to provide an external "configuration file" when writing
sophisticated applications. This gives the end-user the ability to
easily change program options by editing that file.
\*(TC is a Python module for parsing such configuration files. \*(TC
understands and parses a configuration "language" which has a rich set
of string-substitution, variable name, conditional, and validation
features.
By using \*(TC, you unburden your program from the major responsibility
of configuration file parsing and validation, while providing your
users a rich set of configuration features.
.SH DOCUMENT ORGANIZATION
This document is divided into 4 major sections:
.B PROGRAMMING USING THE \*(TC API
discusses how to call the configuration file parser, the options
available when doing this, and what the parser returns. This is
the "Programmer's View" of the module and provides in-depth
descriptions of the API, data structures, and options available to
the programmer.
.B CONFIGURATION LANGUAGE REFERENCE
describes the syntax and semantics of the configuration language
recognized by \*(TC. This is the "User's View" of the package, but
both programmers and people writing configuration files will find this
helpful.
.B ADVANCED TOPICS
describes some ways to combine the various \*(TC features to
do some fairly nifty things.
.B INSTALLATION
explains how to install this package on various platforms. This
information can also be found in the \'READ-1ST.txt\' file distributed
with the package.
.SH PROGRAMMING USING THE \*(TC API
\*(TC is a Python module and thus available for use by any Python
program. This section discusses how to invoke the \*(TC parser, the
options available when doing so, and what the parser returns to
the calling program.
One small note is in order here. As a matter of coding style and
brevity, the code examples here assume the following Python import
syntax:
.nf
from tconfpy import *
.fi
If you prefer the more pedestrian:
.nf
import tconfpy
.fi
you will have to prepend all references to a \*(TC object with
\'tconfpy.\'. So \'retval=ParseConfig(...\' becomes
\'retval = tconfpy.ParseConfig(...\' and so on.
You will also find the test driver code provided in the \*(TC package
helpful as you read through the following sections. \'test-tc.py\' is
a utility to help you learn and exercise the \*(TC API. Perusing the
code therein is helpful as an example of the topics discussed below.
.SS API Overview
The \*(TC API consists of a single call. Only the configuration file
to be processed is a required parameter, all the others are optional
and default as described below:
.nf
from tconfpy import *
retval = ParseConfig(cfgfile,
InitialSymTable={},
AllowNewVars=True,
AllowNewNamespaces=True,
LiteralVars=True,
Debug=False
)
where:
.fi
.TP
.B cfgfile (Required Parameter - No Default)
The the name of a file containing configuration information
.TP
.B InitialSymTable (Default: {})
A pre-populated symbol table (a Python dictionary). As described
below, this must contain valid \'VarDescriptor\' entries for each
symbol in the table.
.TP
.B AllowNewVars (Default: True)
Allow the user to create new variables in the configuration file.
.TP
.B AllowNewNamespaces (Default: True)
Allow new namespaces to be created in the configuration file.
.TP
.B LiteralVars (Default: False)
If set to False this option enables variable substitutions within
\'.literal\' blocks of a configuration file. See the section in the
language reference below on \'.literal\' usage for details.
.TP
.B Debug (Default: False)
If set to True, \*(TC will provide detailed debugging information
about each line processed when it returns.
.TP
.B retval
An object of type \'tconfpy.RetObj\' used to return parsing results.
.SS The Initial Symbol Table API Option
The simplest way to parse a configuration file is just to call
the parser with the name of that file:
.nf
retval = ParseConfig("MyConfigFile")
.fi
Assuming your configuration file is valid, \'ParseConfig()\' will
return a symbol table populated with all the variables defined in the
file and their associated values. This symbol table will have
.B only
the symbols defined in that file (plus a few built-in and pre-defined
symbols needed internally by \*(TC).
However, the API provides a way for you to pass a "primed" symbol
table to the parser that contains pre-defined symbols/values of
your own choosing. Why on earth would you want to do this? There
are a number of reasons:
.IP \(bu 4
You may wish to write a configuration file which somehow depends
on a pre-defined variable that only the calling program can know:
.nf
.if [APPVERSION] == 1.0
# Set configuration for older application releases
.else
# Set configuration for newer releases
.endif
.fi
In this example, only the calling application can know its own
version, so it sets the variable APPVERSION in a symbol table
which is passed to \'ParseConfig()\'.
.IP \(bu 4
You may wish to "protect" certain variable names be creating
them ahead of time and marking them as "Read Only". This is
useful when you want a variable to be available for use
within a configuration file, but you do not want users to
be able to change its value. In this case, the variable can
be referenced in a string substitution or conditional test,
but cannot be changed.
.IP \(bu 4
You may want to place limits on what values can be assigned to
a particular variable. When a variable is newly defined in a
a configuration file, it just defaults to being a string
variable without any limits on its length or content. But
variables that are created by a program have access to the
variable's "descriptor". By setting various attribues
of the variable descriptor you can control variable type,
content, and range of values. In other words, you can have
\*(TC "validate" what values the user assigns to particular
variables. This substantially simplifies your application because
no invalid variable value will ever be returned from the parser.
.SS How To Create An Initial Symbol Table
A \*(TC "Symbol Table" is really nothing more than a Python
dictionary. The key for each dictionary entry is the variable's name
and the value is a \*(TC-specific object called a "variable
descriptor". Creating new variables in the symbol table involves
nothing more than this:
.nf
from tconfpy import *
# Create an empty symbol table
MySymTable = {}
# Create descriptor for new variable
MyVarDes = VarDescriptor()
# Code to fiddle with descriptor contents goes here
MyVarDes.Value = "MyVal"
# Now load the variable into the symbol table
MySymTable["MyVariableName"] = MyVarDes
# Repeat this process for all variables, then call the parser
retval = ParseConfig("MyConfigFile", InitialSymTable=MySymTable)
.fi
The heart of this whole business the \'VarDescriptor\' object. It
"describes" the value and properties of a variable. These
descriptor objects have the following attributes and defaults:
.nf
VarDescriptor.Value = ""
VarDescriptor.Writeable = True
VarDescriptor.Type = TYPE_STRING
VarDescriptor.Default = ""
VarDescriptor.LegalVals = []
VarDescriptor.Min = None
VarDescriptor.Max = None
.fi
When \*(TC encounters a new variable in a configuration file, it just
instantiates one of these descriptor objects with these defaults for
that variable. That is, variables newly-defined in a configuration
file are entered into the symbol table as string types, with an
initial value of "" and with no restriction on content or length.
But, when you create variables under program control to "prime" an
initial symbol table, you can modify the content of any of these
attributes for each variable. These descriptor attributes are what
\*(TC uses to validate subsequent attempts to change the variable's
value in the configuration file. In other words, modifying a
variable's descriptor tells \*(TC just what you'll accept as "legal"
values for that variable.
Each attribute has a specific role:
.TP
.B VarDescriptor.Value (Default: Empty String)
Holds the current value for the variable.
.TP
.B VarDescriptor.Writeable (Default: True)
Sets whether or not the user can change the variable's value. Setting
this attribute to False makes the variable
.B Read Only.
.TP
.B VarDescriptor.Type (Default: TYPE_STRING)
One of TYPE_BOOL, TYPE_COMPLEX, TYPE_FLOAT, TYPE_INT, or TYPE_STRING.
This defines the type of the variable. Each time \*(TC sees
a value being assigned to a variable in the configuration file, it
checks to see if that variable already exists in the symbol table.
If it does, the parser checks the value being assigned and makes sure
it matches the type declared for that variable. For example,
suppose you did this when defining the variable, \'foo\':
.nf
VarDescriptor.Type = TYPE_INT
.fi
Now suppose the user puts this in the configuration file:
.nf
foo = bar
.fi
This will cause a type mismatch error because \'bar\' cannot be coerced
into an integer type - it is a string.
As a general matter, for existing variables, \*(TC attempts to coerce
the right-hand-side of an assignment to the type declared for that
variable. The least fussy operation here is when the variable is
defined as TYPE_STRING because pretty much everything can be coerced
into a string. For example, here is how \'foo = 3+8j\' is treated for
different type declarations:
.nf
VarDescriptor.Type VarDescriptor.Value
------------------ -------------------
TYPE_BOOL Type Error
TYPE_COMPLEX 3+8j (A complex number)
TYPE_FLOAT Type Error
TYPE_INT Type Error
TYPE_STRING \'3+8j\' (A string)
.fi
This is why the default type for newly-defined variables in
the configuration file is TYPE_STRING: they can accept
pretty much
.B any
value.
.TP
.B VarDescriptor.Default (Default: Empty String)
This is a place to store the default value for a given variable. When
a variable is newly-defined in a configuration file, \*(TC places the
first value assigned to that variable into this attribute. For
variables already in the symbol table, \*TC does nothing to this
attribute. This attribute is not actually used by \*(TC for anything.
It is provided as a convenience so that the calling program can easily
"reset" every variable to its default value if desired.
.TP
.B VarDescriptor.LegalVals (Default: [])
Sometimes you want to limit a variable to a specific set of values.
That's what this attribute is for. \'LegalVals\' explictly lists
every legal value for the variable in question. If the list is
empty,then this validation check is skipped.
The exact semantics of LegalVals varies depending on
the type of the variable.
.nf
Variable Type What LegalVals Does
------------- -------------------
Boolean Nothing - Ignored
Integer, Float, Complex List of numeric values the
user can assign to this variable
Examples: [1, 2, 34]
[3.14, 2.73, 6.023e23]
[3.8-4j, 5+8j]
String List of Python regular expressions.
User must assign a value to this
variable that matches at least
one of these regular expressions.
Example: [r'a+.*', r'^AnExactString$']
.fi
The general semantic here is "If Legal Vals is not an empty list, the
user must assign a value that matches one of the items in LegalVals."
One special note applies to \'LegalVals\' for string variables. \*(TC
always assumes that this list contains Python regular expressions.
For validation, it grabs each entry in the list, attempts to compile
it as a regex, and checks to see if the value the user wants to set
matches. If you define an illegal regular expression here, \*(TC will
catch it and produce an appropriate error.
.TP
.B VarDescriptor.Min and VarDescriptor.Max (Default: None)
These set the minimum and maxium legal values for the variables,
but the semantics vary by variable type:
.nf
Variable Type What Min/Max Do
------------- ---------------
Boolean, Complex Nothing - Ignored
Integer, Float Set Minimum/Maxium allowed values.
String Set Minimum/Maximum string length
.fi
In all cases, if you want these tests skipped, set \'Min\' or \'Max\'
to the Python None.
.P
All these various validations are logically "ANDed" together.
i.e., A new value for a variable must be allowed
AND of the appropriate type AND one of the legal values AND
within the min/max range.
\*(TC makes no attempt to harmonize these validation
conditions with each other. If you specify a value in
\'LegalVals\' that is, say, lower than allowed by
\'Min\' you will always get an error when the user sets
the variable to that value: It passed the \'LegalVals\'
validation but failed it for \'Min\'.
.SS The Initial Symbol Table And Lexical Namespaces
The
.B CONFIGURATION LANGUAGE REFERENCE
section below discusses lexical namespaces in some detail from the
user's point-of-view. However, it is useful for the programmer
to understand how they are implemented.
\*(TC is written to use a pre-defined variable named \'NAMESPACE\' as
the place where the current namespace is kept. If you do not
define this variable in the initial symbol table passed to the parser,
\*(TC will create it automatically with an initial value of "".
From a programmer's perspective, there are are few important things
to know about namespaces and the \'NAMESPACE\' variable:
.IP \(bu 4
You can manually set the initial namespace to something other than "".
You do this by creating the \'NAMESPACE\' variable in the initial
symbol table passed to the parser, and setting the \'Value\' attribute
of its descriptor to whatever you want as the initial namespace. At
startup \*(TC will check this initial value to make sure it conforms
to the rules for properly formed names - i.e., It it will check for
blank space, a leading \'$\, the presence of square brackets, and so
on. If the initial namespace value you provide is illegal, \*(TC will
produce an error and reset the initial namespace to "".
.IP \(bu 4
Each time the user enters a new namespace, it is added to the
\'LegalVals\' attribute of the \'NAMESPACE\' variable descriptor.
This is done to keep a "history" of all namespaces encountered during
parsing.
.IP \(bu 4
The semantics of \'LegalVals\' are slightly different than usual
in the case of the \'NAMESPACE\' variable. As described above, the
presence of anything in \'LegalVals\' ordinarily means that the
variable in question is limited to one of those values. However,
in the case of \'NAMESPACE\'
.B this is only true if \'AllowNewNamespaces=False\' is passed to the API.
In other words, if \'AllowNewNamespaces=True\' (default), the user can
create new namespaces of their own choosing - i.e., They can create
namespaces not yet in \'LegalVals\'. In this case, \'LegalVals\'
is just a list of all the namespaces they created.
.IP \(bu 4
If you are inhibiting the creation of new namespaces (via the
\'AllowNewNamespaces = False\' option described below) you can
enumerate the namespaces the user
.B is
allowed to use by pre-defining them. You do this by adding each
permitted namespace to the \'LegalVals\' attribute of the
\'NAMESPACE\' variable descriptor in the initial symbol table.
.IP \(bu 4
When the call to \'ParseConfig()\' completes, the \'Value\'
attribute of the \'NAMESPACE\' variable descriptor will contain
the namespace that was in effect when the parse completed. i.e.,
It will contain the last namespace used.
.IP \(bu 4
Similarly, on parser completion, the \'LegalVals\' attribute of the
\'NAMESPACE\' variable descriptor will contain the name of all the
namespaces encountered during parsing. This is
.B not
true if you suppressed new namespace creation with the
\'AllowNewNamespaces = False\' API option. In that case, the
\'LegalVals\' attribute will contain only those namespaces you
pre-defined. Any or all of these may- or may not have been used in the
configuration file, there is no way to tell.
.SS How The \*(TC Parser Validates The Initial Symbol Table
When you pass an initial symbol table to the parser, \*(TC does some
basic validation that the table contents properly conform to the
\'VarDescriptor\' format and generates error messages if it finds
problems. However, the program does
.B not
check your specifications to see if they make sense. For instance
if you define an integer with a minimum value of 100 and a maximum
value of 50, \*(TC cheerfully accepts these limits even they they
are impossible. You'll just be unable to do anything with that
variable - any attempt to change its value will cause an error
to be recorded. Similarly, if you put a value in \'LegalVals\' that
is outside the range of \'Min\' to \'Max\', \*(TC will accept
it quietly.
.SS The \'AllowNewVars\' API Option
By default, \*(TC lets the user define any new variables they
wish in a configuration file, merely by placing a line in the
file in this form:
.nf
Varname = Value
.fi
However, you can disable this capability by calling the parser like
this:
.nf
retval = ParseConfig("myconfigfile", AllowNewVars=False)
.fi
This means that the configuration file can "reference"
any pre-defined variables, and even change their values
(if they are not Read-Only), but it cannot create
.B new
variables.
This feature is primarily intended for use when you pass an initial
symbol table to the parser and you do not want any other variables
defined by the user. Why? There are several possible uses for
this option:
.IP \(bu 4
You know every configuration variable name the calling program
will use ahead of time. Disabling new variable names keeps
the configuration file from getting cluttered with variables
that the calling program will ignore anyway, thereby
keeping the file more readable.
.IP \(bu 4
You want to insulate your user from silent errors caused
by misspellings. Say your program looks for a configuration
variable called \'MyEmail\' but the user enters something
like \'myemail = foo@bar.com\'. \'MyEmail\' and \'myemail\'
are entirely different variables and only the former is
recognized by your calling program. By turning off new
variable creation, the user's inadvertent misspelling of
the desired variable name will be flagged as an error.
Note, however, that there is one big drawback to disabling new
variable creation. \*(TC processes the configuration file on a
line-by-line basis. No line "continuation" is supported. For really
long variable values and ease of maintenance, it is sometimes helpful
to create "intermediate" variables what hold temporary values used to
construct a variable actually needed by the calling program. For
example:
.nf
inter1 = Really, really, really, really, long argument #1
inter2 = Really, really, really, really, long argument #2
realvar = command [inter1] [inter2]
.fi
If you disable new variable creation you can't do this
anymore unless all the variables \'inter1\', \'inter2\',
and \'realvar\' are pre-defined in the initial symbol
table passed to the parser.
.SS The \'AllowNewNamespaces\' API Option
By default, \*(TC supports the use of an arbitrary number of
lexical namespaces. They can be pre-defined in an initial
symbol table passed to the parser and/or created in the configuration
file as desired. (The details are described in a later section
of this document.)
There may be times, however, when you do not want users creating new
namespaces on their own. The reasons are much the same as for
preventing the creation of new variables in the option above:
Maintaining simplicity and clarity in the configuration file and
preventing "silent" errors due to misspellings.
In this case, call the API with \'AllowNewNamespaces=False\' and the
creation of new namespaces in the configuration file will be disabled.
Any attempt to create a new namespaces via either the
\'[new-ns-name]\' or \'NAMESPACE=new-ns-name\' methods will cause a
parse error to be generated.
It is important to realize that this option only disables the
creation of
.B new
namespaces. As discussed in the previous section on namespace
processing, it is possible to pass an initial symbol table to the
parser which has one or more pre-defined namespaces in it. Each of
these pre-defined namespaces is available for use throughout the
configuration file even if \'AllowNewNamespaces\' is set to False.
.SS The \'LiteralVars\' API Option
\*(TC supports the inclusion of literal text anywhere in a
configuration file via the \'.literal\' directive. This
directive effectively tells the \*(TC parser to pass
every line it encounters "literally" until it sees a
corresponding \'.endlinteral\' directive. By default,
\*(TC does
.B exactly
this. However, \*(TC has very powerful variable substitution
mechanisms. You may want to embed variable references in a
"literal" block and have them replaced by \*(TC.
Here is an example:
.nf
MyEmail = me@here.com # This defines variable MyEmail
.literal
printf("[MyEmail]"); /* A C Statement */
.endliteral
.fi
By default, \'ParseConfig()\' will leave everything within
the \'.literal\'/\'.endliteral\' block unchanged. In our
example, the string:
.nf
printf("[MyEmail]"); /* A C Statement */
.fi
would be in the list of literals returned by \'ParseConfig()\'.
However, we can ask \*(TC to do variable replacement
.B within
literal blocks by setting \'LiteralVars=True\' in the
\'ParseConfig()\' call:
.nf
retval = ParseConfig("myconfigfile", LiteralVars=True)
.fi
In this example, \*(TC would return:
.nf
printf("me@here.com"); /* A C Statement */
.fi
At first glance this seems only mildly useful, but it is
actually very handy. As described later in this document,
\*(TC has a rich set of conditional operators and string
sustitution facilities. You can use these features along
with literal block processing and variable substitution
within those blocks. This effectively lets you use
\*(TC as a preprocessor for
.B any
other language or text.
.SS The \'Debug\' API Option
\*(TC has a fairly rich set of debugging features built into its
parser. It can provide some detail about each line parsed as well
as overall information about the parse. Be default, debugging is
turned off. To enable debugging, merely set \'Debug=True' in the
API call:
.nf
retval = ParseConfig("myconfigfile", Debug=True)
.fi
.SS How \*(TC Processes Errors
As a general matter, when \*(TC encounters an error in the
configuration file currently being parsed, it does two things. First,
it adds a descriptive error message into the list of errors returned
to the calling program (see the next section). Secondly, in many
cases, noteably during conditional processing, it sets the parser
state so the block in which the error occurred is logically False.
This does not happen in every case, however. If you are having
problems with errors, enable the Debugging features of the package and
look at the debug output. It provides detailed information about what
caused the error and why.
.SS \*(TC Return Values
When \*(TC is finished processing the configuration file it returns an
object which contains the entire results of the parse. This includes
a symbol table, any relevant error or warning messages, debug information
(if you requested this via the API), and any "literal" lines encountred
in the configuration.
The return object is an instance of the class \'twander.RetObj\' which is
nothing more than a container to hold return data. In the simplest
case, we can parse and extract results like this:
.nf
from tconfpy import *
retval = ParseConfig("myconfigfile", Debug=True)
.fi
\'retval\' now contains the results of the parse:
.TP
.B retval.Errors
A Python list containing error messages. If this list is empty, you
can infer that there were no parsing errors - i.e., The configuration
file was OK.
.TP
.B retval.Warnings
A Python list containing warning messages. These describe minor
problems not fatal to the parse process, but that you really ought to
clean up in the configuration file.
.TP
.B retval.SymTable
A Python dictionary which lists all the defined symbols and their
associated values. A "value" in this case is always an object of type
tconfpy.VarDescriptor (as described above).
.TP
.B retval.Literals
As described below, the \*(TC configuration language supports a
\'.literal\' directive. This directive allows the user to embed
literal text anywhere in the configuration file. This effectively
makes \*(TC useful as a preprocessor for any other language or text.
retval.Literals is a Python list containing all literal text
discovered during the parse.
.TP
.B retval.Debug
A Python list containing detailed debug information for each line
parsed as well as some brief summary information about the parse.
retval.Debug defaults to an empty list and is only populated if you
set \'Debug=True\' in the API call that initiated the parse (as in the
example above).
.SH CONFIGURATION LANGUAGE REFERENCE
\*(TC recognizes a full-featured configuration language that includes
variable creation and value assignment, a full preprocessor with
conditionals, type and value enforcement, and lexical namespaces.
This section of the document describes that language and provides
examples of how each feature can be used.
.SS \*(TC Configuration Language Syntax
\*(TC supports a fairly simple and direct configuration
language syntax:
.IP \(bu 4
Each line is treated independently. There is no line
"continuation".
.IP \(bu 4
The \'#'\ can begin a comment anywhere on a line. This is done
blindly. If you need to embed this symbol somewhere within a variable
value, use the \'[HASH]\' variable reference.
.IP \(bu 4
Whitespace is (mostly) insignificant. Leading and trailing
whitespace is ignored, as is whitespace around comparison
operators. However, there are some places where
whitespace matters:
.nf
- Variable names may not contain whitespace
- Directives must be followed by whitespace if they
take other arguments.
- When assigning a value to a string variable,
whitespace within the value on the right-hand-side
is preserved. Leading- and trailing whitespace around
the right-hand-side of the assignment is ignored.
- Whitespace within both the left- and right-hand-side
arguments of a conditional comparison
(\'.if ... == / != ...\') is significant for purposes
of the comparison.
.fi
.IP \(bu 4
Case is always significant except when assigning a value to Booleans
(described in the section below entitled,
.B Some Notes On Boolean Variables
).
.IP \(bu 4
Regardless of a variable's type, all variable references
return
.B a string representation of the variable's value!
This is done so that the variable's value can be used for comparison
testing and string substitution/concatenation. In other words,
variables are stored in their native type in the symbol table that is
returned to the calling program. However, they are treated as strings
during the parsing of the configuration file whenever they are used in
a comparison test or in a substitution.
.IP \(bu 4
Any line which does not conform to these rules and/or is not
in the proper format for one of the operations described below,
is considered an error.
.SS Creating Variables And Assigning A Value
The heart of a configuration file is a "variable". Variables are
stored in a "Symbol Table" which is returned to the calling program
once the configuration file has been processed. The calling program
can pre-define any variables it wishes before processing a
configuration file. You can normally also define your own new
variables in the configuration file as desired (unless the programmer
has inhibited new variable creation).
Variables are assigned values like this:
.nf
MyVariable = Some string of text
.fi
If \'MyVariable\' is a new variable, \*(TC will create it on the spot.
If it already exists, \*(TC will first check and make sure that \'Some
string of text\' is a legal value for this variable. If not, it will
produce an error and refuse to change the current value of
\'MyVariable\'.
Anytime you create a new variable, the first value assigned to
it is also considered its "default" value. This may (or may
not) be meaningful to the application program.
Variables which are newly-defined in a configuration file are
always understood to be
.B string
variables - i.e., They hold "strings" of text. However, it is
possible for the applications programmer to pre-define
variables with other types and place limitations on what values
the variable can take and/or how short or long a string variable
may be. (See the previous section,
.B PROGRAMMING USING THE \*(TC API
for all the gory details.)
.SS Variable Names
Variables can be named pretty much anything you like, with certain
restrictions:
.IP \(bu 4
Variable names may not contain whitespace.
.IP \(bu 4
Variable names may not begin with the \'$\' character. The one exception
to this is when you are referencing the value of an environment
variable. References to environment variables begin with \'$\':
.nf
# A reference to an environment variable is legal
x = [$TERM]
# Attempting to create a new variable starting with \'$\' is illegal
$MYVAR = something
.fi
.IP \(bu 4
Variable names cannot have the \'#\' character anywhere in them
because \*(TC sees that character as the beginning a comment.
.IP \(bu 4
Variable names cannot begin with \'.\' character. \*(TC understands
a leading period in a variable name to be a "namespace escape".
This is discussed in a later section on lexical namespaces.
.IP \(bu 4
Variable names cannot contain the \'[\' or \']\' characters. These
are reserved symbols used to indicate a variable
.B reference.
.IP \(bu 4
You cannot have a variable whose name is the empty string. This is
illegal:
.nf
= String
.fi
.IP \(bu 4
The variable named \'NAMESPACE\' is not available for your own
use. \*(TC understands this variable to hold the current lexical
namespace as described later in this document. If you set it
to a new value, it will change the namespace, so be sure this is
what you wanted to do.
.SS Getting And Using The Value Of A Variable
You can get the value of any currently defined variable by
"referencing" it like this:
.nf
.... [MyVariable] ...
.fi
The brackets surrounding any name are what indicate that you want that
variable's value.
You can also get the value of any Environment Variable on your
system by naming the variable with a leading \'$\':
.nf
... [$USER] ... # Gets the value of the USER environment variable
.fi
However you cannot set the value of an environment variable:
.nf
$USER = me # This is not permitted
.fi
This ability to both set and retrieve variable content makes it easy
to combine variables through "substitution":
.nf
MYNAME = Mr. Tconfpy
MYAGE = 101
Greeting = Hello [MYNAME], you look great for someone [MYAGE]!
.fi
Several observations are worth noting here:
.IP \(bu 4
The substitution of variables takes place as soon as the parser
processes the line \'Greeting = ...\'. That is, variable substitution
happens as it is encountered in the configuration file. The only
exception to this is if an attempt is made to refer to an
undefined/non-existent variable. This generates an error.
.IP \(bu 4
The variable \'Greeting\' now contains the
.B string
"Hello Mr. Tconfpy, you look great for someone 101!" This is true even
if variable \'MYAGE\' has been defined by the calling program to be
an integer type. To repeat a previously-made point: All variable substitution
and comparison operations in a configuration file are done with
.B strings
regardless of the actual type of the variables involved.
.IP \(bu 4
Unless a variable as been marked as "Read Only" by the application
program, you can continue to change its value as you go. Simply
adding another line at the end of our example above will change the
value of \'Greeting\' to something new:
.nf
Greeting = Generic Greeting Message
.fi
In other words, the last assignment statement for a given variable
"wins". This may seem sort of pointless, but it actually has great
utility. You can use the \'.include\' directive to get, say, a
"standard" configuration provided by the system administrator for a
particular application. You can then selectively override the
variables you want to change in your own configuration file.
.SS Indirect Variable Assignment
The dereferencing of a variable's value can take place on either
the right- or
.B left-hand-side
of an assignment statement. This means so-called "indirect"
variable assignments are permitted:
.nf
CurrentTask = HouseCleaning
[CurrentTask] = Dad
.fi
To understand what this does you need to realize that before
\*(TC does anything with a statement in a configuration file, it
replaces every variable reference with its associated value
(or produces an error for references to non-existent variables).
So the second statement above is first converted to:
.nf
HouseCleaning = Dad
.fi
i.e., The value \'Dad\' is assigned to a (new) variable called
\'HouseCleaning\'. In other words, putting a variable reference
on the left-hand-side of an assignment like this allows you
to access another variable which is named "indirectly". You have
to be careful when doing this, though. Consider a similar, but
slightly different example:
.nf
CurrentTask = House Cleaning # This is fine
[CurrentTask] = Dad # Bad!
.fi
The reason this no longer works is that the indirect
reference causes the second line to parse to:
.nf
House Cleaning = Dad
.fi
This is illegal because whitespace is not permitted in variable names.
\*(TC will produce an error if it sees such a construct. As a general
matter, any variable you construct through this indirection method
must still conform to all the rules of variable naming: It cannot
contain whitespace, begin with \'$\', contain \'#\', \'[\', or \']\' and
so on.
Get into the habit of reading \'[something]\' as, "The current value
of \'something\'". See if you understand what the following does (if
you don't, try it out with \'test-tc.py\'):
.nf
foo = 1
bar = 2
[foo] = bar
[bar] = [foo]
.fi
You can get pretty creative with this since variable references
can occur pretty much anywhere in an assignment statement.
The only place they cannot appear is
.B within
another variable reference. That is, you cannot "nest" references:
.nf
# The Following Is Fine
FOO = Goodness
BAR = Me
Oh[FOO][BAR] = Goodness Gracious Me!
# But This Kind Of Nesting Attempt Causes An Error
[FOO[BAR]] = Something Or Other
.fi
.SS Introducing Lexical Namespaces
So far,the discussion of variables and references has conveniently
ignored the presence of another related \*(TC feature, "lexical
namespaces." Namespaces are a way to automatically group
related variables together. Suppose you wanted to describe
the options on your car in a configuration file. You might do
this:
.nf
MyCar.Brand = Ferrari
MyCar.Model = 250 GTO
MyCar.Color = Red
# And so on ...
.fi
You'll notice that every variable start with the "thing" that
each item has in common - they are features of \'MyCar\'.
We can simplify this considerably by introducing a lexical namespace:
.nf
[MyCar]
Brand = Ferrari
Model = 250 GTO
Color = Red
.fi
The first statement looks like a variable reference, but it is not.
.B A string inside square brackets by itself on a line introduces a namespace.
The first statement in this example sets the namespace to \'MyCar\'.
From that point forward until the namespace is changed again, every
variable assignment
.B and
reference is "relative" to the namespace. What this really means is
that \*(TC sticks the namspace plus a \'.\' in front of every
variable assigned or referenced. It does this automatically and
invisibly, so \'Brand\' is turned into \'MyCar.Brand\' and so on. You
can actually check this by loading the example above into a test
configuration file and running the \'test-tc\' program on it. You will
see the "fully qualified" variable names that actually were loaded
into the symbol table, each beginning with \'MyCar.\' and ending with
the variable name you specified.
Realize that this is entirely a naming "trick". \*(TC has no clue
what the namespace
.B means,
it just combines the current namespace with the variable name to
create the actual variable name that will be returned in the symbol
table.
You're likely scratching your head wondering why on earth this
feature present in \*(TC. There are several good reasons for it:
.IP \(bu 4
It reduces typing repetetive information throughout the configuration
file. In turn, this reduces the likelyhood of a typographical or
spelling error.
.IP \(bu 4
It helps visibly organize the configuration file. A namespace makes
it clear which variables are related to each other somehow. This is no
big deal in small configurations, but \*(TC was written with the idea
of supporting configuration files that might contain thousands or
even tens of thousands of entries.
.IP \(bu 4
It simplifies the application programmer's job. Say I want to write a
program that extracts all the information about your car from the
configuration file, but I don't know ahead of time how many things you
will describe. All I really have to know is that you are using
\'MyCar\' as the namespace for this information. My program can then
just scan the symbol table after the configuration file has been parsed,
looking for variables whose name begins with \'MyCar.\'. So if you
want to add other details about your auto like, say, \'Age\', \'Price\',
and so on, you can do so later
.B and the program does not have to be rewritten.
.IP \(bu 4
It helps enforce correct configuration files. By default, you can
introduce new namespaces into the configuration file any time you
like. However, as described in the previous section on the
\*(TC API, the application programmer can limit you to a pre-defined
set of legal namespaces (via the \'AllowNewNamespaces=False\' API
option). By doing this, the programmer is helping you avoid
incorrect configuration file entries by limiting just which
namespaces you can enter to reference or create variables.
.SS Rules For Using Lexical Namespace
Creating and using lexical namespaces is fairly straightforward,
but there are a few restrictions and rules:
.IP \(bu 4
The default initial namespace is the empty string, "". In this one
case, \*(TC does nothing to variables assigned or referenced. That's
why our early examples in the previous section worked. When we
assigned a value to a variable and then referenced that variable
value, we did so while in the so-called "root" namespace, "". When
the namespace is "", nothing is done to the variable names
Bear in mind that the programmer can change this default namespace to
something other than "" before the configuration file is ever
processed. If they do this, they would be well advised to let their
users know this fact.
.IP \(bu 4
There two ways to change to a new namespace:
.nf
[NewNameSpace] # Must appear on a line by itself or with a comment only
OR
NAMESPACE = NewNamespace
.fi
If, at any point, you want to return to the root namespace, you can use
one of these two methods:
.nf
[]
OR
NAMESPACE =
.fi
So, why are there two ways to do the same thing? The first way is the more
common, and the more readable way to do it. It appears on a line by itself and
makes it clear that the namespace is being changed. However, because variable
references cannot be "nested", you can only use strings of text here.
Suppose you want to change the namespace in a way that depends on the
value of another variable. For instance:
.nf
LOCATION = Timbuktu
NAMESPACE = [LOCATION]-East
.fi
In other words, the second form of a namespace change allows you to
employ the \*(TC string substitution and variable referencing
features. Bear in mind that \*(TC is case-sensitive so this
will not work as you expect:
.nf
Namespace = something
.fi
This just set the value of the variable \'Namespace\' to
\'something\' and has nothing whatsoever to do with lexical
namespaces.
.IP \(bu 4
Whichever method you use to change it,
.B the new namespace must follow all the same rules used for naming variables.
For example, both of the following will cause an error:
.nf
[$FOO]
OR
x = $FOO
NAMESPACE = [x]
.fi
.IP \(bu 4
By default, all variable assignments and references are
.B relative to the currently active namespace:
.nf
[MyNameSpace]
foo = 123 # Creates a variable called \'MyNameSpace.foo\'
x = bar # Means: \'MyNameSpace.x = MyNameSpace.bar\'
.fi
.IP \(bu 4
If you want to set or reference a variable in a namespace different
than the current namespace, you must use a so-called "absolute"
variable name. You do this by "escaping" the variable name. To escape the
name, begin it with a \'.\' and then use the
.B full name (including namespace)
of that variable. (This is called the "fully qualified variable
name".) For example:
.nf
[NS1] # Switch to the namespace \'NS1\'
foo = 14 # Creates \'NS1.foo\'
[NS2] # Switch to the \'NS2\' namespace
foo = [.NS1.foo] # Sets \'NS2.foo = 14\'
.fi
There is another clever way to do this without using the escape
character. \*(TC has no understanding whatsoever of what a
lexical namespace actually is. It does nothing more than
"glue" the current namespace to any variable names and references
in your configuration file. Internally, all variables are
named
.B relative to the root namespace.
This means that you can use the fully qualified variable name
without any escape character any time you are in the root
namespace:
.nf
[NS1] # Switch to the namespace \'NS1\'
foo = 14 # Creates \'NS1.foo\'
[] # Switch to the root namespace
foo = [NS1.foo] # Sets \'foo = 14\' - no escape needed
.fi
.IP \(bu 4
\*(TC keeps track of every new namespace you create and returns that
list to the calling program to use as it wishes. (See the section
above entitled,
.B The Initial Symbol Table And Lexical Namespaces
for details.)
.IP \(bu 4
Lexical namspaces are implemented by having \'NAMESPACE\' just be
nothing more than (yet) another variable in the symbol table. \*(TC
just understands that variable to be special - it treats it as the
repository of the current lexical namespace. This means you can use
the value of NAMESPACE in your own string substitutions:
.nf
MyVar = [NAMESPACE]-Isn't This Cool?
.fi
You can even use the current value of NAMESPACE when setting a new
namespace:
.nf
NAMESPACE = [NAMESPACE]-New
.fi
One final, but very important point is worth noting here. The
\'NAMESPACE\' variable itself is always understood to be
.B relative to the root namespace.
No matter what the current namespace actually is, \'[NAMESPACE]\' or
\'NAMESPACE = ...\' always set a variable by that name in the
root namespace. Similarly, when we use a variable reference to get
the current namespace value (as we did in the example above), \'NAMESPACE\'
is understood to be relative to the root namespace. That's why things
like this work:
.nf
[MyNewSpace]
x = 100 # MyNewSpace.x = 100
y = [NAMESPACE]-1 # MyNewSpace.y = MyNewSpace-1
NAMESPACE = NewSpace # .NAMESPACE = NewSpace
fi
.SS Predefined Variables
\*(TC predefines a number of variables. The \'NAMESPACE\' variable we
discussed in the previous section is one of them, but there are a number
of others of which you should be aware. Note that all predefined variables
.B are relative to the root namespace.
Except for the \'NAMESPACE\' variable, they are all Read Only and
cannot be modified in your configuration file.
The first group of predefined variables are called "System Variables".
As the name implies, they provide information about the system on
which you're running. These are primarily useful when doing
conditional tests (described later in this document). For example,
by doing conditional tests with System Variables you can have
one configuration file that works on both Unix and Windows operating
systems. The System Variables are:
.nf
Variable Name Contains
------------- --------
.MACHINENAME - The name of the computer on which you are running.
May also include full domain name, depending on system.
.OSDETAILS - Detailed information about the operating system in use.
.OSNAME - The name of the operating system in use.
.OSRELEASE - The version of the operating system in use.
.PLATFORM - The generic type of the operating system in use.
.PYTHONVERSION - The version of Python in use.
.fi
By combining these System Variables as well as the content of
selected Environment Variables, you can create complex conditional
configurations that "adapt" to the system on which a Python
application is running. For example:
.nf
.if [.MACHINENAME] == foo.bar.com
BKU = tar
.else
BKU = [$BACKUPPROGRAM]
.endif
.fi
The other kind of predefined variables are called "Reserved Variables".
\*(TC understands a number of symbols as part of its own language.
For example, the string \'#\' tells \*(TC to begin a comment until
end-of-line. There may be times, however, when
.B you
need these strings for your own use. In other words, you would like
to use one of the strings which comprise the \*(TC language for your
own purposes and have \*(TC ignore them. The Reserved Variables
give you a way to do this. The Reserved Variables are:
.nf
Variable Name Contains
------------- --------
DELIML [
DELIMR ]
DOLLAR $
ELSE .else
ENDIF .endif
ENDLITERAL .endliteral
EQUAL =
EQUIV ==
HASH #
IF .if
IFALL .ifall
IFANY .ifall
IFNONE .ifnone
INCLUDE .include
LITERAL .literal
NOTEQUIV !=
PERIOD .
.nf
For instance, suppose you wanted to include the \'#\' symbol
in the value of one of your variables. This will not work,
because \*(TC interprets it as the beginning of a comment,
which is not what you want:
.nf
MyJersey = Is #23
.fi
So, we use one of the Reserved Variables to get what we want:
.nf
MyJersey = Is [HASH]23
.fi
One word of warning, though. At the end of the day, you still have
to create variable names or namespace names that are legal. You
can't "sneak" illegal characters into these names using Reserved
Variables:
.nf
foo = [DOLLAR]MyNewNamespace # No problem
NAMESPACE = [foo] # No way - namespace cannot start with $
.fi
.SS Type And Value Enforcement
By default, any variable (or namespace) you create in a configuration file
is understood to just hold a string of characters. There are no limits
to what that string may contain, how long it is, and so on.
However, \*(TC gives the programmer considerable power to enforce variable
types and values, if they so choose. (See the section above entitled,
.B PROGRAMMING USING THE \*(TC API
for the details.) The programmer can set all kinds of limitations
about a variable's type, permissible values, and (in the case of
strings) how long or short it may be. The programmer does this by
defining these limitations for each variable of interest
.B prior to calling \*(TC to parse your configuration file.
In that case, when \*(TC actually processes the configuration file,
it "enforces" these restrictions any time you attempt to change the
value of one of these variables. If you try to assign a value that
fails one of these "validation" tests, \*(TC will produce an error
and leave the variable's value unchanged.
For instance, suppose the programmer has defined
variable "Foo" to be a floating point number, and that it must have
a value between -10.5 and 100.1. In that case:
.nf
Foo = 6.023E23 # Error - Value is out of range
Foo = MyGoodness # Error - Value must be a FP number, not a string
Foo = -2.387 # Good - Value is both FP an in range
.fi
.SS What Specific Validations Are Available?
The programmer has several different restrictions they can place on
a variable's value. You do not need to understand how they work,
merely what they are so that any error messages you see will make sense.
.IP \(bu 4
The programmer may declare any variable to be
.B Read Only.
This means you can still use references to that variable to extract
its value, but any attempt to change it value within the configuration
file will fail and produce an error.
.IP \(bu 4
The programmer may specify the variable's
.B type
as string (the default), integer, floating point, complex, or boolean.
.IP \(bu 4
The programmer may specify
.B the set of all legal values
that can be assigned to a variable. For instance, the programmer
might specify that the floating point variable \'Transcend\' can only
be set to either 3.14 or 2.73. Similarly, the programmer might
specify that the string variable \'Color\' can only ever be set to
\'Red\', \'Green\', or \'Blue\'. In fact, in the case of string
variables, the programmer can actually specify a set of patterns
(regular expressions) the value has to match. For instance, they can
demand that you only set a particular string variable to strings that
begin with \'a'\ and end with \'bob\'.
.IP \(bu 4
For integer and floating point variables, the programmer can specify
a legal
.B value range
for the variable. If you change the value of such a variable, that value
must be within the defined range or you'll get an error.
.IP \(bu 4
For string variables, the programmer can specify a minimum and maxium
.B length
for the strings you assign to the variable in question.
.IP \(bu 4
The programmer can limit you to
.B only being able to use existing variables.
(.i.e. The Predefined variables and any variables the programmer
has defined ahead of time.) In that case, any attempt to create a
new variable in the configuration file will fail and produce an error.
.IP \(bu 4
The programmer can limit you to
.B only being able to use namespaces they have defined ahead of time.
In that case, if you attempt to enter a namespace not on the list
the programmer created ahead of time will fail and produce an error.
.IP \(bu 4
The programmer can enable or prevent
.B the substitution of variable references in literal blocks
(see below). If they disable this option, something like \'[Foo]\'
is left unchanged within the literal block. i.e., It too, is treated
"literally".
.SS Notes On Variable Type/Value Enforcement
There are a few other things you should know about how \*(TC
enforces restrictions on variables:
.IP \(bu 4
Within the configuration file,
.B variable references are always converted to strings
regardless of the actual type of the variable in question. For
instance, suppose the programmer defines variable \'Foo\' to be
floating point. Then:
.nf
Foo = 1.23
Bar = Value is [Foo] # Creates a new *string* variable with the
# value: "Value is 1.23"
.fi
In other words, variable values are "coerced" into strings for
the purposes of substution and conditional testing within a
configuration file.
.IP \(bu 4
You cannot create anything but a string variable with no limitations
from within a configuration file. All validation features
.B require
the limitations to be specified by the calling program ahead of time.
.IP \(bu 4
Similarly, you cannot change any of the enforcement options from
within a configuration file. These features are only available
under program control, presumably by the application program that
is calling \*(TC.
.IP \(bu 4
There is no way to know what the limitations are on a particular
variable from within the configuration file. Programmers
who use these features should document the variable restrictions
they've employed as a part of the documentation for the application
in question.
.SS Some Further Notes On Boolean Variables
One last note here concerns Boolean variables. Booleans are actually
stored in the symbol table as the Python boolean values,
.B True
or
.B False.
However, \*(TC accepts user statements that set the value of the
boolean in a number of formats:
.nf
Boolean True Boolean False
------------ -------------
foo = 1 foo = 0
foo = True foo = False
foo = Yes foo = No
foo = On foo = Off
.fi
This is the one case where \*(TC is insensitive to case -
"tRUE", "TRUE", and "true" are all accepted, for example.
.B NOTE HOWEVER:
If the user wants to do a conditional test on the value of
a boolean they
.B must
observe case and test for either \'True\' or \'False\':
.nf
boolvar = No
.if [boolvar] == False # This works fine
.if [boolvar] == FALSE # This does not work - Case is not being observed
.if [boolvar] == Off # Neither does this - Only True and False can be tested
.fi
.SS The \'.include\' Directive
.SS Conditional Directives
.SS The \'.literal\. Directive
.SH ADVANCED TOPICS
.SS Guaranteeing A Correct Base Configuration
.SS Enforcing Mandatory Configurations
.SS Iterative Parsing
.SH INSTALLATION
There are three ways to install \*(TC depending on your preferences
and type of system. In each of these installation methods you must be
logged in with root authority on Unix-like systems or as the
Administrator on Win32 systems.
.SS Preparation - Getting And Extracting The Package
For the first two installation methods, you must first download the
latest release from:
.nf
http://www.tundraware.com/Software/tconfpy/
.fi
Then unpack the contents by issuing the following command:
.nf
tar -xzvf py-tconfpy-X.XXX.tar.gz (where X.XXX is the version number)
.fi
Win32 users who do not have tar installed on their system can find
a Windows version of the program at:
.nf
http://unxutils.sourceforge.net/
.fi
.SS Install Method #1 - All Systems (Semi-Automated)
Enter the directory created in the unpacking step above. Then issue
the following command:
.nf
python setup.py install
.fi
This will install the \*(TC module and compile it.
You will manually have to copy the 'test-tc.py' program to a directory
somewhere in your executable path. Similarly, copy the documentation
files to locations appropriate for your system.
.SS Install Method #2 - All Systems (Manual)
Enter the directory created in the unpacking step above. Then,
manually copy the tconfpy.py file to a directory somewhere in your
PYTHONPATH. The recommended location for Unix-like systems is:
.nf
.../pythonX.Y/site-packages
.fi
For Win32 systems, the recommended location is:
.nf
...\\PythonX.Y\\lib\\site-packages
.fi
Where X.Y is the Python release number.
You can pre-compile the \*(TC module by starting Python interactively
and then issuing the command:
.nf
import tconfpy
.fi
Manually copy the 'test-tc.py' program to a directory
somewhere in your executable path. Copy the documentation
files to locations appropriate for your system.
.SS Install Method #3 - FreeBSD Only (Fully-Automated)
Make sure you are logged in as root, then:
.nf
cd /usr/ports/devel/py-tconfpy
make install
.fi
This is a fully-automated install that puts both code and
documentation where it belongs. After this command has completed
you'll find the license agreement and all the documentation (in the
various formats) in:
.nf
/usr/local/share/doc/py-tconfpy
.fi
The 'man' pages will have been properly installed so either of these
commands will work:
.nf
man tconfpy
man test-tc
.fi
.SS Bundling \*(TC With Your Own Programs
If you write a program that depends on \*(TC you'll need to ensure
that the end-users have it installed on their systems. There are two
ways to do this:
1) Tell them to download and install the package as described above.
This is not recommended since you cannot rely on the technical
ability of end users to do this correctly.
2) Just include 'tconfpy.py' in your program distribution directory.
This ensures that the module is available to your program
regardless of what the end-user system has installed.
.SH THE \*(TC MAILING LIST
TundraWare Inc. maintains a mailing list to help you with your
\*(TC questions and bug reports. To join the list, send email
to
.B majordomo@tundraware.com
with a single line of text in the body (not the Subject line)
of the message:
.nf
subscribe tconfpy-users your-email-address-goes-here
.fi
You will be notified when your subscription has been approved. You
will also receive detailed information about how to use the list,
access archives of previous messages, unsubscribe, and so on.
.SH OTHER
\*(TC requires Python 2.3 or later.
.SH BUGS AND MISFEATURES
None known as of this release.
.SH COPYRIGHT AND LICENSING
\*(TC is Copyright(c) \*(CP TundraWare Inc. For terms of use, see
the tconfpy-license.txt file in the program distribution. If you
install \*(TC on a FreeBSD system using the 'ports' mechanism, you
will also find this file in /usr/local/share/doc/py-tconfpy.
.SH AUTHOR
.nf
Tim Daneliuk
tconfpy@tundraware.com
tundra