/doc/pdcp.1.in

https://code.google.com/ · Autoconf · 246 lines · 219 code · 27 blank · 0 comment · 12 complexity · b245c7907b08b9499132ae3718ea40b9 MD5 · raw file

  1. \." $Id$
  2. .\"
  3. .\"
  4. .\"
  5. .TH "@PACKAGE@" "1" "@host_os@" "@PDSH_VERSION@"
  6. .SH NAME
  7. pdcp \- copy files to groups of hosts in parallel
  8. .br
  9. rpdcp \- (reverse pdcp) copy files from a group of hosts in parallel
  10. .SH SYNOPSIS
  11. \fBpdcp\fR [\fIoptions\fR]... src [src2...] dest
  12. .br
  13. \fBrpdcp\fR [\fIoptions\fR]... src [src2...] dir
  14. .SH DESCRIPTION
  15. \fBpdcp\fR is a variant of the rcp(1) command. Unlike rcp(1), which
  16. copies files to a single remote host, \fBpdcp\fR can copy files to
  17. multiple remote hosts in parallel. However, \fBpdcp\fR does
  18. not recognize files in the format ``rname@rhost:path,'' therefore all
  19. source files must be on the local host machine. Destination nodes must
  20. be listed on the \fBpdcp\fR command line using a suitable target
  21. nodelist option (See the \fIOPTIONS\fR section below). Each destination
  22. node listed must have \fBpdcp\fR installed for the copy to succeed.
  23. .LP
  24. When \fBpdcp\fR receives SIGINT (ctrl-C), it lists the status of current
  25. threads. A second SIGINT within one second terminates the program. Pending
  26. threads may be canceled by issuing ctrl-Z within one second of ctrl-C.
  27. Pending threads are those that have not yet been initiated, or are still
  28. in the process of connecting to the remote host.
  29. .LP
  30. Like pdsh(1), the functionality of \fBpdcp\fR may be supplemented by
  31. dynamically loadable modules. In \fBpdcp\fR, the modules may provide
  32. a new connect protocol (replacing the standard rsh(1) protocol), filtering
  33. options (e.g. excluding hosts that are down), and/or host selection
  34. options (e.g. \fI-a\fR selects all nodes from a local config file).
  35. By default, \fBpdcp\fR requires at least one "rcmd" module to be
  36. loaded (to provide the channel for remote copy).
  37. .SH "REVERSE PDCP"
  38. \fBrpdcp\fR performs a reverse parallel copy. Rather than copying files
  39. to remote hosts, files are retrieved from remote hosts and stored locally.
  40. All directories or files retrieved will be stored with their remote
  41. hostname appended to the filename. The destination file must be a
  42. directory when this option is used.
  43. .LP
  44. In other respects, \fBrpdcp\fR is exactly like \fBpdcp\fR, and further
  45. statements regarding \fBpdcp\fR in this manual also apply to \fBrpdcp\fR.
  46. .SH "RCMD MODULES"
  47. The method by which \fBpdcp\fR connects to remote hosts may be
  48. selected at runtime using the \fI-R\fR option (See \fIOPTIONS\fR below).
  49. This functionality is ultimately implemented via dynamically loadable
  50. modules, and so the list of available options may be different
  51. from installation to installation. A list of currently available rcmd
  52. modules is printed when using any of the \fI-h\fR, \fI-V\fR, or \fI-L\fR
  53. options. The default rcmd module will also be displayed with the
  54. \fI-h\fR and \fI-V\fR options.
  55. .LP
  56. A list of \fIrcmd\fR modules currently distributed with \fBpdcp\fR
  57. follows.
  58. .TP 8
  59. rsh
  60. Uses an internal, thread-safe implementation of BSD rcmd(3)
  61. to run commands using the standard rsh(1) protocol.
  62. .TP
  63. ssh
  64. Uses a variant of popen(3) to run multiple copies of the ssh(1)
  65. command.
  66. .TP
  67. mrsh
  68. This module uses the mrsh(1) protocol to execute jobs on remote hosts.
  69. The mrsh protocol uses a credential based authentication, forgoing
  70. the need to allocate reserved ports. In other aspects, it acts just
  71. like rsh.
  72. .TP
  73. krb4
  74. The krb4 module allows users to execute remote commands after
  75. authenticating with kerberos. Of course, the remote rshd daemons
  76. must be kerberized.
  77. .TP
  78. xcpu
  79. The xcpu module uses the xcpu service to execute remote commands.
  80. .SH OPTIONS
  81. The list of available \fBpdcp\fR options is determined at runtime
  82. by supplementing the list of standard \fBpdcp\fR options with
  83. any options provided by loaded \fIrcmd\fR and \fImisc\fR modules.
  84. In some cases, options provided by modules may conflict with
  85. each other. In these cases, the modules are incompatible and
  86. the first module loaded wins.
  87. .SH "Standard target nodelist options"
  88. .TP
  89. \fB\-w\fR \fITARGETS,...\fR
  90. Target and or filter the specified list of hosts. Do not use with any
  91. other node selection options (e.g. \fI\-a\fR, \fI\-g\fR, if they are
  92. available). No spaces are allowed in the comma-separated list. Arguments in
  93. the \fITARGETS\fR list may include normal host names, a range of hosts
  94. in hostlist format (See \fBHOSTLIST EXPRESSIONS\fR), or a single `-'
  95. character to read the list of hosts on stdin.
  96. If a host or hostlist is preceded by a `-' character, this causes those
  97. hosts to be explicitly excluded. If the argument is preceded by a single `^'
  98. character, it is taken to be the path to file containing a list of hosts,
  99. one per line. If the item begins with a `/' character, it is taken as a
  100. regular expression on which to filter the list of hosts (a regex argument
  101. may also be optionally trailed by another '/', e.g. /node.*/). A regex or
  102. file name argument may also be preceeded by a minus `-' to exclude instead
  103. of include thoses hosts.
  104. A list of hosts may also be preceded by "user@" to specify a remote
  105. username other than the default, or "rcmd_type:" to specify an alternate
  106. rcmd connection type for these hosts. When used together, the rcmd type
  107. must be specified first, e.g. "ssh:user1@host0" would use ssh to connect
  108. to host0 as user "user1."
  109. .TP
  110. .I \fB-x\fR \fIhost,host,...\fR
  111. Exclude the specified hosts. May be specified in conjunction with
  112. other target node list options such as \fI\-a\fR and \fI\-g\fR (when
  113. available). Hostlists may also be specified to the \fI\-x\fR option
  114. (see the \fBHOSTLIST EXPRESSIONS\fR section below). Arguments to
  115. \fI-x\fR may also be preceeded by the filename (`^') and regex ('/')
  116. characters as described above, in which case the resulting hosts are excluded
  117. as if they had been given to \fB\-w\fR and preceeded with the minus `-'
  118. character.
  119. .SH "Standard pdcp options"
  120. .TP
  121. .I "-h"
  122. Output usage menu and quit. A list of available rcmd modules
  123. will be printed at the end of the usage message.
  124. .TP
  125. .I "-q"
  126. List option values and the target nodelist and exit without action.
  127. .TP
  128. .I "-b"
  129. Disable ctrl-C status feature so that a single ctrl-C kills parallel
  130. copy. (Batch Mode)
  131. .TP
  132. .I "-r"
  133. Copy directories recursively.
  134. .TP
  135. .I "-p"
  136. Preserve modification time and modes.
  137. .TP
  138. .I "-e PATH"
  139. Explicitly specify path to remote \fBpdcp\fR binary
  140. instead of using the locally executed path. Can also be set via
  141. the environment variable PDSH_REMOTE_PDCP_PATH.
  142. .TP
  143. .I "-l user"
  144. This option may be used to copy files as another user, subject to
  145. authorization. For BSD rcmd, this means the invoking user and system must
  146. be listed in the user\'s .rhosts file (even for root).
  147. .TP
  148. .I "-t seconds"
  149. Set the connect timeout. Default is @CONNECT_TIMEOUT@ seconds.
  150. .TP
  151. .I "-f number"
  152. Set the maximum number of simultaneous remote copies to \fInumber\fR.
  153. The default is @FANOUT@.
  154. .TP
  155. .I "-R name"
  156. Set rcmd module to \fIname\fR. This option may also be set via the
  157. PDSH_RCMD_TYPE environment variable. A list of available rcmd
  158. modules may be obtained via either the \fI-h\fR or \fI-L\fR options.
  159. .TP
  160. .I "-M name,..."
  161. When multiple \fBmisc\fR modules provide the same options to \fBpdsh\fR,
  162. the first module initialized "wins" and subsequent modules are not loaded.
  163. The \fI-M\fR option allows a list of modules to be specified that will be
  164. force-initialized before all others, in-effect ensuring that they load
  165. without conflict (unless they conflict with eachother). This option may
  166. also be set via the PDSH_MISC_MODULES environment variable.
  167. .TP
  168. .I "-L"
  169. List info on all loaded \fBpdcp\fR modules and quit.
  170. .TP
  171. .I "-d"
  172. Include more complete thread status when SIGINT is received, and display
  173. connect and command time statistics on stderr when done.
  174. .TP
  175. .I "-V"
  176. Output \fBpdcp\fR version information, along with list of currently
  177. loaded modules, and exit.
  178. .SH "HOSTLIST EXPRESSIONS"
  179. As noted in sections above,
  180. .B pdcp
  181. accepts ranges of hostnames in
  182. the general form: prefix[n-m,l-k,...], where n < m and l < k, etc.,
  183. as an alternative to explicit lists of hosts. This form should not
  184. be confused with regular expression character classes (also denoted
  185. by ``[]''). For example, foo[19] does not represent foo1 or foo9, but
  186. rather represents a degenerate range: foo19.
  187. This range syntax is meant
  188. only as a convenience on clusters with a prefixNN naming convention and
  189. specification of ranges should not be considered necessary -- the list
  190. foo1,foo9 could be specified as such, or by the range foo[1,9].
  191. Some examples of range usage follow:
  192. .nf
  193. Copy /etc/hosts to foo01,foo02,...,foo05
  194. pdcp -w foo[01-05] /etc/hosts /etc
  195. Copy /etc/hosts to foo7,foo9,foo10
  196. pdcp -w foo[7,9-10] /etc/hosts /etc
  197. Copy /etc/hosts to foo0,foo4,foo5
  198. pdcp -w foo[0-5] -x foo[1-3] /etc/hosts /etc
  199. .fi
  200. As a reminder to the reader, some shells will interpret brackets ('['
  201. and ']') for pattern matching. Depending on your shell, it may be
  202. necessary to enclose ranged lists within quotes. For example, in
  203. tcsh, the first example above should be executed as:
  204. pdcp -w "foo[01-05]" /etc/hosts /etc
  205. .SH "ORIGIN"
  206. Pdsh/pdcp was originally a rewrite of IBM dsh(1) by Jim Garlick
  207. <garlick@llnl.gov> on LLNL's ASCI Blue-Pacific IBM SP system.
  208. It is now also used on Linux clusters at LLNL.
  209. .SH "LIMITATIONS"
  210. When using
  211. .B ssh
  212. for remote execution, stderr of ssh to be folded in with that of the
  213. remote command. When invoked by pdcp, it is not possible for ssh to
  214. prompt for confirmation if a host key changes, prompt for passwords if
  215. RSA keys are not configured properly, etc.. Finally, the connect
  216. timeout is only adjustable with ssh when the underlying ssh implementation
  217. supports it, and pdsh has been built to use the correct option.
  218. .SH "SEE ALSO"
  219. pdsh(1)