/doc/pdcp.1.in

  1\." $Id$
  2.\"
  3.\"
  4.\"
  5.TH "@PACKAGE@" "1" "@host_os@" "@PDSH_VERSION@"
  6
  7.SH NAME
  8pdcp \- copy files to groups of hosts in parallel
  9.br
 10rpdcp \- (reverse pdcp) copy files from a group of hosts in parallel 
 11
 12.SH SYNOPSIS
 13\fBpdcp\fR [\fIoptions\fR]... src [src2...] dest
 14.br
 15\fBrpdcp\fR [\fIoptions\fR]... src [src2...] dir
 16
 17.SH DESCRIPTION
 18\fBpdcp\fR is a variant of the rcp(1) command.  Unlike rcp(1), which
 19copies files to a single remote host, \fBpdcp\fR can copy files to
 20multiple remote hosts in parallel.  However, \fBpdcp\fR does
 21not recognize files in the format ``rname@rhost:path,'' therefore all
 22source files must be on the local host machine.  Destination nodes must
 23be listed on the \fBpdcp\fR command line using a suitable target
 24nodelist option (See the \fIOPTIONS\fR section below).  Each destination
 25node listed must have \fBpdcp\fR installed for the copy to succeed.
 26.LP
 27When \fBpdcp\fR receives SIGINT (ctrl-C), it lists the status of current
 28threads.  A second SIGINT within one second terminates the program. Pending
 29threads may be canceled by issuing ctrl-Z within one second of ctrl-C.
 30Pending threads are those that have not yet been initiated, or are still
 31in the process of connecting to the remote host.
 32.LP 
 33Like pdsh(1), the functionality of \fBpdcp\fR may be supplemented by
 34dynamically loadable modules. In \fBpdcp\fR, the modules may provide
 35a new connect protocol (replacing the standard rsh(1) protocol), filtering
 36options (e.g. excluding hosts that are down), and/or host selection 
 37options (e.g. \fI-a\fR selects all nodes from a local config file). 
 38By default, \fBpdcp\fR requires at least one "rcmd" module to be
 39loaded (to provide the channel for remote copy).
 40
 41.SH "REVERSE PDCP"
 42\fBrpdcp\fR performs a reverse parallel copy.  Rather than copying files 
 43to remote hosts, files are retrieved from remote hosts and stored locally.  
 44All directories or files retrieved will be stored with their remote
 45hostname appended to the filename.  The destination file must be a
 46directory when this option is used. 
 47.LP
 48In other respects, \fBrpdcp\fR is exactly like \fBpdcp\fR, and further 
 49statements regarding \fBpdcp\fR in this manual also apply to \fBrpdcp\fR.
 50
 51.SH "RCMD MODULES"
 52The method by which \fBpdcp\fR connects to remote hosts may be
 53selected at runtime using the \fI-R\fR option (See \fIOPTIONS\fR below). 
 54This functionality is ultimately implemented via dynamically loadable
 55modules, and so the list of available options may be different
 56from installation to installation. A list of currently available rcmd 
 57modules is printed when using any of the \fI-h\fR, \fI-V\fR, or \fI-L\fR 
 58options. The default rcmd module will also be displayed with the
 59\fI-h\fR and \fI-V\fR options.
 60.LP
 61A list of \fIrcmd\fR modules currently distributed with \fBpdcp\fR     
 62follows.
 63.TP 8
 64rsh
 65Uses an internal, thread-safe implementation of BSD rcmd(3)
 66to run commands using the standard rsh(1) protocol.
 67.TP
 68ssh
 69Uses a variant of popen(3) to run multiple copies of the ssh(1)
 70command.
 71.TP
 72mrsh
 73This module uses the mrsh(1) protocol to execute jobs on remote hosts.
 74The mrsh protocol uses a credential based authentication, forgoing
 75the need to allocate reserved ports. In other aspects, it acts just
 76like rsh.
 77.TP
 78krb4
 79The krb4 module allows users to execute remote commands after
 80authenticating with kerberos. Of course, the remote rshd daemons
 81must be kerberized.
 82.TP
 83xcpu
 84The xcpu module uses the xcpu service to execute remote commands.
 85
 86.SH OPTIONS
 87The list of available \fBpdcp\fR options is determined at runtime
 88by supplementing the list of standard \fBpdcp\fR options with
 89any options provided by loaded \fIrcmd\fR and \fImisc\fR modules.
 90In some cases, options provided by modules may conflict with
 91each other. In these cases, the modules are incompatible and
 92the first module loaded wins.
 93
 94.SH "Standard target nodelist options"
 95.TP
 96\fB\-w\fR \fITARGETS,...\fR
 97Target and or filter the specified list of hosts. Do not use with any
 98other node selection options (e.g. \fI\-a\fR, \fI\-g\fR, if they are
 99available). No spaces are allowed in the comma-separated list.  Arguments in
100the \fITARGETS\fR list may include normal host names, a range of hosts
101in hostlist format (See \fBHOSTLIST EXPRESSIONS\fR), or a single `-'
102character to read the list of hosts on stdin.
103
104If a host or hostlist is preceded by a `-' character, this causes those
105hosts to be explicitly excluded. If the argument is preceded by a single `^'
106character, it is taken to be the path to file containing a list of hosts,
107one per line. If the item begins with a `/' character, it is taken  as a
108regular expression on which to filter the list of hosts (a regex argument
109may also be optionally trailed by another '/', e.g.  /node.*/). A regex or
110file name argument may also be preceeded by a minus `-' to exclude instead
111of include thoses hosts.
112
113A list of hosts may also be preceded by "user@" to specify a remote
114username other than the default, or "rcmd_type:" to specify an alternate
115rcmd connection type for these hosts. When used together, the rcmd type
116must be specified first, e.g. "ssh:user1@host0" would use ssh to connect
117to host0 as user "user1."
118
119
120.TP
121.I \fB-x\fR \fIhost,host,...\fR
122Exclude the specified hosts. May be specified in conjunction with
123other target node list options such as \fI\-a\fR and \fI\-g\fR (when
124available). Hostlists may also be specified to the \fI\-x\fR option
125(see the \fBHOSTLIST EXPRESSIONS\fR section below). Arguments to
126\fI-x\fR may also be preceeded by the filename (`^') and regex ('/')
127characters as described above, in which case the resulting hosts are excluded
128as if they had been given to \fB\-w\fR and preceeded with the minus `-'
129character.
130
131
132.SH "Standard pdcp options"
133.TP
134.I "-h"
135Output usage menu and quit. A list of available rcmd modules
136will be printed at the end of the usage message.
137.TP
138.I "-q"
139List option values and the target nodelist and exit without action.
140.TP
141.I "-b"
142Disable ctrl-C status feature so that a single ctrl-C kills parallel
143copy. (Batch Mode)
144.TP
145.I "-r"
146Copy directories recursively.
147.TP
148.I "-p"
149Preserve modification time and modes.
150.TP
151.I "-e PATH"
152Explicitly specify path to remote \fBpdcp\fR binary
153instead of using the locally executed path. Can also be set via
154the environment variable PDSH_REMOTE_PDCP_PATH.
155.TP
156.I "-l user"
157This option may be used to copy files as another user, subject to
158authorization. For BSD rcmd, this means the invoking user and system must
159be listed in the user\'s .rhosts file (even for root).
160.TP
161.I "-t seconds"
162Set the connect timeout. Default is @CONNECT_TIMEOUT@ seconds.
163.TP
164.I "-f number"
165Set the maximum number of simultaneous remote copies to \fInumber\fR.
166The default is @FANOUT@.
167.TP
168.I "-R name"
169Set rcmd module to \fIname\fR. This option may also be set via the
170PDSH_RCMD_TYPE environment variable. A list of available rcmd
171modules may be obtained via either the \fI-h\fR or \fI-L\fR options.
172.TP
173.I "-M name,..."
174When multiple \fBmisc\fR modules provide the same options to \fBpdsh\fR,
175the first module initialized "wins" and subsequent modules are not loaded.
176The \fI-M\fR option allows a list of modules to be specified that will be
177force-initialized before all others, in-effect ensuring that they load
178without conflict (unless they conflict with eachother). This option may
179also be set via the PDSH_MISC_MODULES environment variable.
180.TP
181.I "-L"
182List info on all loaded \fBpdcp\fR modules and quit.
183.TP
184.I "-d"
185Include more complete thread status when SIGINT is received, and display
186connect and command time statistics on stderr when done.
187.TP
188.I "-V"
189Output \fBpdcp\fR version information, along with list of currently
190loaded modules, and exit.
191
192
193.SH "HOSTLIST EXPRESSIONS"
194As noted in sections above, 
195.B pdcp 
196accepts ranges of hostnames in
197the general form: prefix[n-m,l-k,...], where n < m and l < k, etc.,
198as an alternative to explicit lists of hosts.  This form should not
199be confused with regular expression character classes (also denoted
200by ``[]''). For example, foo[19] does not represent foo1 or foo9, but
201rather represents a degenerate range: foo19. 
202
203This range syntax is meant
204only as a convenience on clusters with a prefixNN naming convention and
205specification of ranges should not be considered necessary -- the list
206foo1,foo9 could be specified as such, or by the range foo[1,9].
207
208Some examples of range usage follow:
209
210.nf
211
212Copy /etc/hosts to foo01,foo02,...,foo05
213    pdcp -w foo[01-05] /etc/hosts /etc
214
215Copy /etc/hosts to foo7,foo9,foo10
216    pdcp -w foo[7,9-10] /etc/hosts /etc
217
218Copy /etc/hosts to foo0,foo4,foo5
219    pdcp -w foo[0-5] -x foo[1-3] /etc/hosts /etc
220
221.fi
222
223As a reminder to the reader, some shells will interpret brackets ('['
224and ']') for pattern matching.  Depending on your shell, it may be
225necessary to enclose ranged lists within quotes.  For example, in
226tcsh, the first example above should be executed as:
227
228    pdcp -w "foo[01-05]" /etc/hosts /etc
229
230.SH "ORIGIN"
231Pdsh/pdcp was originally a rewrite of IBM dsh(1) by Jim Garlick
232<garlick@llnl.gov> on LLNL's ASCI Blue-Pacific IBM SP system.  
233It is now also used on Linux clusters at LLNL.
234
235.SH "LIMITATIONS"
236When using 
237.B ssh 
238for remote execution, stderr of ssh to be folded in with that of the
239remote command.  When invoked by pdcp, it is not possible for ssh to
240prompt for confirmation if a host key changes, prompt for passwords if
241RSA keys are not configured properly, etc..  Finally, the connect
242timeout is only adjustable with ssh when the underlying ssh implementation
243supports it, and pdsh has been built to use the correct option.
244
245.SH "SEE ALSO"
246pdsh(1)