/README.QsNet

https://code.google.com/ · Unknown · 49 lines · 38 code · 11 blank · 0 comment · 0 complexity · c875bb9030a266495a389ccae54496cc MD5 · raw file 

    

  1. Running Elan MPI jobs on QsNet Clusters
    2. 

  2. ---------------------------------------
    3. 

  3. 

  4. If built with --with-qshell or --with-mqshell, pdsh may be used to run
    5. 

  5. MPI jobs on a QsNet interconnect.  This option requires that you have
    6. 

  6. the Elan user space libraries installed (qsnetlibs or qswelan-r RPM for
    7. 

  7. Linux) and that your kernel be patched to run the 'elan3' or 'elan4' and
    8. 

  8. 'rms' device drivers. Pdsh can run independently of the RMS product (the
    9. 

  9. 'rms' kernel module, which is used by pdsh, is a distinct facility from
    10. 

  10. the RMS product).
    11. 

  11. 

  12. Quadrics has provided a PDSH FAQ which may answer some common questions
    13. 

  13. about getting the qshell module to run MPI jobs. Please see
    14. 

  14. 

  15.   http://web1.quadrics.com/twiki/bin/view/FAQs/SetupPDSH
    16. 

  16. 

  17. 

  18. rms pdsh module
    19. 

  19. ---------------------------------------
    20. 

  20. Pdsh can also be run via the Quadrics RMS 'allocate' command such that
    21. 

  21. allocate takes care of the node reservations and passes a batch ID through
    22. 

  22. to pdsh via the RMS_RESOURCEID environment variable.  Pdsh retrieves
    23. 

  23. the list of allocated nodes out of the RMS database using the rmsquery
    24. 

  24. command. This functionality is provided by the "rms" pdsh module
    25. 

  25. (--with-rms).
    26. 

  26. 

  27. slurm pdsh module
    28. 

  28. ---------------------------------------
    29. 

  29. Similar to the rms pdsh module, the slurm module allows pdsh to target
    30. 

  30. nodes based on SLURM allocations, either targetting an already running job
    31. 

  31. or by running under ``srun --allocate'' The SLURM jobid can be passed to
    32. 

  32. pdsh using the `-j' option provided by the module, or via the SLURM_JOBID
    33. 

  33. environment variable, which is set by --allocate.
    34. 

    35. 

    36. 

  36. The `/etc/elanhosts' config file
    37. 

  37. ---------------------------------------
    38. 

  38. Pdsh uses a simple config file, /etc/elanhosts, to describe hosts
    39. 

  39. containing Elan adapters (and on which Elan MPI jobs may be run). The
    40. 

  40. config file is also used by the daemons qshd and mqshd to initialize
    41. 

  41. the Elan network error resolver thread. Parsing of the /etc/elanhosts
    42. 

  42. file is accomplished by using the libelanhosts(3) library, upon which
    43. 

  43. pdsh depends (when building for QsNet). See the elanhosts(5)
    44. 

  44. man page for a description of the /etc/elanhosts file format.
    45. 

  45. 

  46. The libelanhosts package may be obtained from
    47. 

  47. 

  48.   ftp://ftp.llnl.gov/pub/linux/libelanhosts/
    49.