diff --git a/tsshbatch.rst b/tsshbatch.rst index e771129..979b0c6 100644 --- a/tsshbatch.rst +++ b/tsshbatch.rst @@ -107,7 +107,7 @@ tsshbatch.py -G "/foo/bar/baz /tmp" -i hostlist This copies ``/foo/bar/baz`` from every machine in - ``hostlistfile`` to the local ``/tmp/`` directory. + host file to the local ``/tmp/`` directory. Since all the files have the same name, they would overwrite each other if copied into the same directory. So, ``tsshbatch`` prepends the string @@ -157,7 +157,7 @@ This allows you to embed ``tsshbatch`` in external shell scripts like this:: - for server in $(tsshbatch.py -i devserverlist -uatserverlist -W) + for server in $(tsshbatch.py -i devserverlist -i uatserverlist -W) do ssh $server done @@ -190,20 +190,19 @@ -h Print help information. - -i hostfile(s) Specify which files to read to get a list of desired - hosts to target. This option may be repeated on - the command line and may also be followed with a - quoted list of such files. The following are - equivalent:: + -i hostfiles Specify which files to read to get a list of desired + hosts to target. This option may be repeated on the + command line and may also be followed with a quoted + list of such files. The following are equivalent:: tsshbatch.py -i devservers -i uatservers ... tsstbatch.py -i "devservers uatservers" ... - The ``-H`` and ``-i`` options can be freely - combined and repated on the command line to create - custom host lists made up of both known inventory - and specific, individual hosts. + The ``-H`` and ``-i`` options can be freely + combined and repated on the command line to create + custom host lists made up of both known inventory + and specific, individual hosts. -k Use ssh keys instead of name/password credentials. @@ -251,7 +250,7 @@ The last entry on the command line is optional and defines a command to run. ``tsshbatch`` will attempt to execute it on every host you've -specified either via ``-H`` or a ``hostlistfile``:: +specified either via ``-H`` or a host file:: tsshbatch.py -Hmyhost ls -al /etc @@ -268,53 +267,6 @@ "nothing to do" really isn't an error. -COMMAND FILES -------------- - -Command files are nothing more than files with a list of commands to -execute. Notice that this is not the same thing as a shell script. -There are no conditionals or other programming features. It's more-or-less -a "to do" list for each host. Command files may include other command -files, make use of variables, and even specify sudo commands. Their -basic form is: - - .include master_commands # optional, can be repeated - command1 # a command to run on each host - sudo foo # run foo with sudo promotion - -If you've specified a ``cmdfile`` containing the commands you want run via -the ``-f`` option, these commands will run *before* the command -you've defined on the command line. It is always the last command -run on each host. - -Command files may freely make use of comments and variable -definitions as described below. They also have a special -directive available, ``.notify`` which causes ``tsshbatch`` to -print descriptive messages to ``stdout`` as the commands in the file -are executed on a remote host. This is helpful when running long, -complex jobs:: - - # Example cmdfile - - .include /my/fine/options - .define __JOB_NAME__ - .notify starting JOB_NAME processing - /usr/local/__JOB_NAME__ - .notify starting phase 2 of __JOB_NAME__ - do_another_command - -Notifications are entirely optional *and do not run on the remote -host*. Think of them as runtime comments to help you see what's going -on. - -You can also specify file transfers within a command file. See the -section below on file transfers for all the details. - -You can put as many ``-f`` arguments as you wish on the command line -and the contents of these files will be run in the order they appeared -from left-to-right on the command line. - - ENVIRONMENT ----------- @@ -344,6 +296,158 @@ overriden with the ``-C`` option. +MANAGING INVENTORY: HOST FILES +------------------------------ + +Although you can specify a list of target hosts with one or more ``-H +"host host host"`` command line options, this gets cumbersome when you +have to manage a large inventory of machines. This is what "host files" +are intended to do. + +Host files are files that name a host, one per line. They may +contain comments, include other host files, and make reference to +previously defined variables. Here's an example, let's call it +``stage-servers``:: + + # Example staging host list + + .define __EXTERNAL__ = splat.com + + stage1.example.com + stage2.example.com + stage3.__EXTERNAL__ + +Say you'd like to get the uptime for each of these servers:: + + tsshbatch.py -x -i stage-servers uptime + +You can have more than one of these on a command line so this is +possible:: + + tsshbatch.py -x -i dev-servers -i stage-servers -i prod-servers uptime + +But ... that's kind of clumsy. Instead let's create a new host file +called ``all-servers`` like this:: + + # Master host list + + .include dev-servers + .include stage=servers + .include prod-servers + +Now our command looks like this:: + + tsshbatch.py -x -i all-servers uptime + +You can put as many ``-i`` arguments as you wish on the command line. +The contents of these files will be run in the order they appear from +left-to-right on the command line. + +You can organize your host files in any hierarchy you like by making +use of search paths, as described later in this document. + +The use of host files, the ``-H`` command line option, ``.includes``, +and variable substitution, give ``tsshbatch`` a very powerful way to +manage complex host inventories. There may be times when you'd like +to select a subset of that inventory but *run some other program with +those host names*. This is most commonly the case when you'd like to +interactively ``ssh`` into each host for some reason. + +That's what the ``-W`` command line option is for. ``tsshbatch`` computes +the inventory you've specified, spits it out as single line of text and exits, +thereby enabling things like this:: + + for server in $(tsshbatch.py -i dev-servers -i dr-servers -W) + do + ssh $server + done + +This allows you to do all your intentory management via ``tsshbatch`` +constructs, but make use of it with any other program of your +choosing. + + +MANAGING JOBS: COMMAND FILES +---------------------------- + +Although ``tsshbatch`` accepts command to be run on each host on its +own command line, this gets cumbersome for multi-step activities. The +``-f`` option allows you to specify one or more "command files" that +list the things you want done, in order. + +Command files may include other command files, make use of variables, +and even specify sudo commands. For example:: + + # This is a comment + .include master_commands # optional, can be repeated + command1 # a command to run on each host + sudo foo # run foo with sudo promotion on each host + +Notice that this is not the same thing as a shell script. There are +no conditionals or other programming features. The contents of a +command file are more-or-less a "to do" list for each host. If your +job requires the complexity of a "real" language, write script in +the language of your choice, and then make reference to that script +in a command file. + +If you've specified a command file containing the commands you want run +via the ``-f`` option, these commands will run *before* the command +you've defined on the command line. That one is always the last +command run on each host. + +Command files may freely make use of comments and variable definitions +as described below. They also have a special directive available, +``.notify``. This directive tells ``tsshbatch`` to print descriptive +messages to ``stdout`` as the commands in the file are executed on a +remote host. This is helpful when running long, complex jobs:: + + # Example cmdfile + + .include /my/fine/options + .define __JOB_NAME__ + .notify starting JOB_NAME processing + /usr/local/__JOB_NAME__ + .notify starting phase 2 of __JOB_NAME__ + do_another_command + +Notifications are entirely optional *and do not run on the remote +host*. Think of them as runtime comments emitted by ``tsshbatch`` to +help you know what's going on. + +You can also specify file transfers within a command file. See the +section below on file transfers for all the details. + +You can put as many ``-f`` arguments as you wish on the command line. +The contents of these files will be run in the order they appeare from +left-to-right on the command line. + +You can organize your command files in any hierarchy you like by making +use of search paths, as described later in this document. + + +SEARCHING YOUR HOST AND COMMAND FILES +------------------------------------- + +Over time, you will probably build up a large set of host files for +describing your inventory and host files for standard jobs you +run often. It's convenient to search through them quickly when +you're looking for something specific. + +The ``-L`` command line option just lists every host file and command +file ``tsshbatch`` knows about on all defined search paths. This is +handly if you want to examine your hierarchy of files. + +The ``-F string string ...`` command line option looks through all +your host *and* command files and returns the names of those that +contain any of the strings you've listed. The match is case +insensitive and literal - no regular expressions are supported. + +The ``-V string string ...`` command line option is the inverse +of ``-F``. It returns a list of host and command files that do +*not contain* the strings you specify. Again, the match is +a simple, literal case-insensitive test. + + FEATURES AND USE CASES ---------------------- @@ -640,10 +744,10 @@ tsshbatch.py -G"/etc/fstab ./tmp" -G"/etc/rc.conf ./tmp" ... -You may also put file transfer specifications into a ``cmdfile`` via +You may also put file transfer specifications into a command file via the ``.getfile`` and ``.putfile`` directives. This is handy when you have many to do and don't want to clutter up the command line. Each -must be on its own line in the ``cmdfile`` and in the same form as if +must be on its own line in the command file and in the same form as if it were provided on the command line:: .getfile /path/to/srcfile destdir # This will get a file @@ -651,13 +755,13 @@ File transfers are done in the order they appear. For instance, if you have a file transfer specification on the command line and then -make reference to a ``cmdfile`` with a file transfer specification in +make reference to a command file with a file transfer specification in it, the one on the command line gets done first. .. NOTE:: Keep in mind that ``tsshbatch`` always processes file transfers *before* executing any commands, no matter what - order they appear in the ``cmdfile``. If you have this in a - ``cmdfile``:: + order they appear in the command file. If you have this in a + command file:: echo "Test" .putfile "./myfile /foo/bar/baz/" @@ -666,7 +770,7 @@ gets run. This can be counterintuitive. It's therefore recommended that you put your file transfers into a single file, and ``.include`` it as the first thing in your - ``cmdfile`` to make it obvious that these will be run first. + command file to make it obvious that these will be run first. By default, ``tsshbatch`` aborts if any file transfer fails. This is @@ -694,7 +798,7 @@ Commenting ========== -Both the ``cmdfile`` and ``hostlistfile`` can be freely commented +Both the command file and host file can be freely commented using the ``#`` character. Everything from that character to the end of that line is ignored. Similarly, you can use whitespace freely, except in cases where it would change the syntax of a command or host @@ -705,7 +809,7 @@ ======== You may also include other files as you wish with the ``.include -filename`` directive anywhere in the ``cmdfile`` or ``hostlistfile``. +filename`` directive anywhere in the command file or host file. This is useful for breaking up long lists of things into smaller parts. For example, suppose you have three host lists, one for each major production areas of your network:: @@ -727,7 +831,7 @@ that way if you edited any of the underlying files, the ``hosts-all`` would reflect the change. -Similarly you can do the same thing with the ``cmdfile`` to group +Similarly you can do the same thing with the command file to group similar commands into separate files and include them. ``tsshbatch`` does not enforce a limit on how deeply nested @@ -743,8 +847,8 @@ references. Note, however, that references to builtin variables will fail unless you have overriden them. Why? Because builtins don't get defined until a host connection is attempted. This doesn't happen -until *after* all variable processing and file includes have been -done. So:: +until *after* all user defined variable processing and file includes +have been done. So:: .define MYINCLUDE = /some/fine/file # OK .include MYINCLUDE @@ -758,7 +862,7 @@ .define __HOSTNAME__ ! hostname # Override the builtin .include __HOSTNAME__ # OK -As a matter of keeping things simple, stick to User Defined variables +As a matter of keeping things simple, stick to user defined variables as part of an ``.include`` target. @@ -862,8 +966,8 @@ ``tsshbatch`` has three different kinds of variables: - *User Defined Variables* are the kind in the example above. - You, the user, define them as you wish in a ``cmdfile`` or - ``hostlistfile``. + You, the user, define them as you wish in a command file or + host file. - *Execution Variables* run any program or script of your choosing (on the same machine you're running ``tsshbatch``) and assign @@ -878,7 +982,7 @@ ========================================== User Defined and Execution Variables are defined in either a -``hostlistfile`` or ``cmdfile``. +host file or command file. Builtin Variables are defined within ``tsshbatch`` itself unless you override them. @@ -892,14 +996,14 @@ You'll get an output of ... ``secondfoo``! Why? Because before ``tsshbatch`` tries to run anything, it has to process all the -``cmdfiles``, ``hostlistfile``, and the command line content. So, +``cmdfiles``, host file, and the command line content. So, before we ever get around to doing an ``echo __FOO__`` on some host, the second definition of __FOO__ has been read in ... and last definition wins. Execution Variables are like User Defined Variables. They get processed a single time *at the time they're read in* from a -``cmdfile`` or ``hostlistfile``. +command file or host file. Builtin Variables get evaluated *every time ``tsshbatch`` prepares to connect to a new host* (unless you've overriden them). That way, the @@ -908,12 +1012,12 @@ Keep in mind that ``tsshbatch`` isn't a programming language. It's "variables" are simple string substitutions with "last one wins" semantics. There is no notion of scope, for example. If you define -the same variable in, say, a ``cmdfile`` and also in the -``hostlistfile``, the latter will "win". Why? Because +the same variable in, say, a command file and also in the +host file, the latter will "win". Why? Because ``hostlistfiles`` are always read in after any ``cmdfiles``. Finally, variable references in a definition are *ignored*. Say you -do this in a ``cmdfile``:: +do this in a command file:: .define __CLEVER __ = __REALLYCLEVER__ .define __REALLYCLEVER__ = Not That Smart @@ -933,7 +1037,7 @@ ``tsshbatch`` allows you to define variables which will then be used to replace matching strings in ``cmdfiles``, ``hostlistfiles``, and file transfer specifications. For example, suppose you have this in a -``hostlistfile``:: +host file:: .define DOMAIN=.my.own.domain.com @@ -947,7 +1051,7 @@ Similarly, you might want define ``MYCMD=some_long_string`` so you don't have to type ``some_long_string`` over and over again in a -``cmdfile``. +command file. There are some "gotchas" to this: @@ -976,7 +1080,7 @@ - Variables may be defined in either ``cmdfiles`` or ``hostlistfiles`` but they are *visible to any subsequent file that gets read*. For instance, ``cmdfiles`` are read before any - ``hostlistfiles``. Any variables you've defined in a ``cmdfile`` + ``hostlistfiles``. Any variables you've defined in a command file that happen to match a string in one of your hostnames will be substituted. @@ -1003,14 +1107,14 @@ For example, suppose you want create a file on many machines, and you want that file to be named based on who ran the ``tsshbatch`` job. -You might do this in a ``cmdfile``:: +You might do this in a command file:: .define __WHOAMI__ = ! whoami touch __WHOAMI__-Put_This_Here.txt So, if ID ``luser`` is running ``tsshbatch``, a file called ``luser-Put_This_Here.txt`` will be created (or have its timestamp -updated) on every machine in the ``hostlistfile`` or named with +updated) on every machine in the host file or named with ``-H``. Notice it is the ``!`` character that distinguishes an Execution @@ -1207,7 +1311,7 @@ The idea here is to use silent mode with the various variables described previously to customize your own reporting output. Imagine -you have this in a ``cmdfile`` and you run ``tsshbatch`` in silent +you have this in a command file and you run ``tsshbatch`` in silent mode:: .define __USER__ = ! echo $USER @@ -1229,12 +1333,12 @@ - Directives like ``.define`` and ``.include`` must be the first non-whitespace text on the left end of a line. If you do this in a - ``cmdfile``:: + command file:: foo .include bar ``tsshbatch`` thinks you want to run the command ``foo`` with an - argument of ``.include bar``. If you do it in a ``hostlistfile``, + argument of ``.include bar``. If you do it in a host file, the program thinks you're trying to contact a host called ``foo .include bar``. In neither case is this likely to be quite what you had in mind. Similarly, everything to the right of the directive is