PageRenderTime 39ms CodeModel.GetById 18ms RepoModel.GetById 0ms app.codeStats 0ms

/README.md

http://github.com/jamwt/Angel
Markdown | 332 lines | 258 code | 74 blank | 0 comment | 0 complexity | 0ef7635d7b8b2d84c1bddfb04c69942a MD5 | raw file
Possible License(s): BSD-3-Clause
  1. Angel
  2. =====
  3. [![Build Status](https://travis-ci.org/MichaelXavier/Angel.png?branch=master)](https://travis-ci.org/MichaelXavier/Angel)
  4. `angel` is a daemon that runs and monitors other processes. It
  5. is similar to djb's `daemontools` or the Ruby project `god`.
  6. It's goals are to keep a set of services running, and to facilitate
  7. the easy configuration and restart of those services.
  8. Maintainers Wanted
  9. ------------------
  10. I do not actively use Angel anymore and don't have much time to work
  11. on it. If you are an invested user in Angel, you shold have a say in
  12. the direction of the project. Let me know in a Github issue and I will
  13. happily add you to the project.
  14. Motivation
  15. ----------
  16. The author is a long-time user of `daemontools` due to its reliability
  17. and simplicity; however, `daemontools` is quirky and follows many
  18. unusual conventions.
  19. `angel` is an attempt to recreate `daemontools`'s capabilities (though
  20. not the various bundled utility programs which are still quite useful)
  21. in a more intuitive and modern unix style.
  22. Functionality
  23. -------------
  24. `angel` is driven by a configuration file that contains a list of
  25. program specifications to run. `angel` assumes every program listed in
  26. the specification file should be running at all times.
  27. `angel` starts each program, and optionally sets the program's stdout
  28. and stderr to some file(s) which have been opened in append mode
  29. (or pipes stdout and stderr to some logger process); at
  30. this point, the program is said to be "supervised".
  31. If the program dies for any reason, `angel` waits a specified number
  32. of seconds (default, 5), then restarts the program.
  33. The `angel` process itself will respond to a HUP signal by
  34. re-processing its configuration file, and synchronizing the run
  35. states with the new configuration. Specifically:
  36. * If a new program has been added to the file, it is started and
  37. supervised
  38. * If a program's specification has changed (command line path,
  39. stdin/stdout path, delay time, etc) that supervised child
  40. process will be sent a TERM signal, and as a consequence of
  41. normal supervision, will be restarted with the updated spec
  42. * If a program has been removed from the configuration file,
  43. the corresponding child process will be sent a TERM signal;
  44. when it dies, supervision of the process will end, and
  45. therefore, it will not be restarted
  46. Safety and Reliability
  47. ----------------------
  48. Because of `angel`'s role in policing the behavior of other
  49. daemons, it has been written to be very reliable:
  50. * It is written in Haskell, which boasts a combination of
  51. strong, static typing and purity-by-default that lends
  52. itself to very low bug counts
  53. * It uses multiple, simple, independent lightweight threads
  54. with specific roles, ownership, and interfaces
  55. * It uses STM for mutex-free state synchronization between
  56. these threads
  57. * It falls back to polling behavior to ensure eventual
  58. synchronization between configuration state and run
  59. state, just in case odd timing issues should make
  60. event-triggered changes fail
  61. * It simply logs errors and keeps running the last good
  62. configuration if it runs into problems on configuration
  63. reloads
  64. * It has logged hundreds of thousands of uptime-hours
  65. since 2010-07 supervising all the daemons that power
  66. http://bu.mp without a single memory leak or crash
  67. Building
  68. --------
  69. 1. Install the haskell-platform (or somehow, ghc 7.6 +
  70. cabal-install)
  71. 2. Run `cabal install` in the project root (this directory)
  72. 3. Either add the ~/.cabal/bin file to your $PATH or copy
  73. the `angel` executable to /usr/local/bin
  74. Notes:
  75. * Angel is recommended to be built on GHC 7.6 and newer.
  76. Configuration and Usage Example
  77. -------------------------------
  78. The `angel` executable takes a path to an angel configuration
  79. file.
  80. angel --help
  81. angel - Process management and supervision daemon
  82. Usage: angel CONFIG_FILE [-u USER] [-v VERBOSITY]
  83. Available options:
  84. -h,--help Show this help text
  85. -u USER Execute as this user
  86. -v VERBOSITY Verbosity from 0-2 (default: 2)
  87. If the -u option is specified on the command line, it will take precedence over
  88. any configuration command in the configuration file.
  89. `angel`'s configuration system is based on Bryan O'Sullivan's `configurator`
  90. package. A full description of the format can be found here:
  91. http://hackage.haskell.org/packages/archive/configurator/0.1.0.0/doc/html/Data-Configurator.html
  92. A basic configuration file might look like this:
  93. #user is optional with a default of the current user
  94. user = "alice"
  95. watch-date {
  96. exec = "watch date"
  97. }
  98. ls {
  99. exec = "ls"
  100. stdout = "/tmp/ls_log"
  101. stderr = "/tmp/ls_log"
  102. delay = 7
  103. termgrace = off
  104. }
  105. workers {
  106. directory = "/path/to/worker"
  107. exec = "run_worker"
  108. count = 30
  109. pidfile = "/path/to/pidfile.pid"
  110. env {
  111. FOO = "BAR"
  112. BAR = "BAZ"
  113. }
  114. termgrace = 10
  115. }
  116. By adding a "user" configuration command at the top level of the
  117. configuration it is possible to specify the user Angel will be executed as.
  118. Each of the programs listed in the specification file will also be executed
  119. as this user. This option is only read on first start up, and is not re-read
  120. if the configuration file changes.
  121. The user configuration command is ignored if a user is specified on the
  122. command line via the -u option.
  123. Angel will run as the invoking user if no user configuration command
  124. is specified.
  125. Each program that should be supervised starts a `program-id` block:
  126. watch-date {
  127. Then, a series of corresponding configuration commands follow:
  128. * `exec` is the exact command line to run (required)
  129. * `stdout` is a path to a file where the program's standard output
  130. should be appended (optional, defaults to /dev/null)
  131. * `stderr` is a path to a file where the program's standard error
  132. should be appended (optional, defaults to /dev/null)
  133. * `delay` is the number of seconds (integer) `angel` should wait
  134. after the program dies before attempting to start it again
  135. (optional, defaults to 5)
  136. * `directory` is the current working directory of the newly
  137. executed program (optional, defaults to angel's cwd)
  138. * `logger` is another process that should be launched to handle
  139. logging. The `exec` process will then have its stdout and stderr
  140. piped into stdin of this logger. Recommended log
  141. rotation daemons include [clog](https://github.com/jamwt/clog)
  142. or [multilog](http://cr.yp.to/daemontools.html). *Note that
  143. if you use a logger process, it is a configuration error
  144. to specify either stdout or stderr as well.*
  145. * `count` is an optional argument to specify the number of processes to spawn.
  146. For instance, if you specified a count of 2, it will spawn the program
  147. twice, internally as `workers-1` and `workers-2`, for example. Note that
  148. `count` will inject the environment variable `ANGEL_PROCESS_NUMBER` into the
  149. child process' environment variable.
  150. * `pidfile` is an optional argument to specify where a pidfile should be
  151. created. If you don't specify an absolute path, it will use the running
  152. directory of angel. When combined with the `count` option, specifying a
  153. pidfile of `worker.pid`, it will generate `worker-1.pid`, `worker-2.pid`,
  154. etc. If you don't specify a `pidfile` directive, then `angel` will *not*
  155. create a pidfile
  156. * `env` is a nested config of string key/value pairs. Non-string values are
  157. invalid.
  158. * `termgrace` is an optional number of seconds to wait between
  159. sending a SIGTERM and a SIGKILL to a program when it needs to shut
  160. down. Any positive number will be interpreted as seconds. `0`,
  161. `off`, or omission will be interpreted as disabling the feature and
  162. only a sigterm will be sent. This is useful for processes that must
  163. not be brought down forcefully to avoid corruption of data or other
  164. ill effects.
  165. Assuming the above configuration was in a file called "example.conf",
  166. here's what a shell session might look like:
  167. jamie@choo:~/random/angel$ angel example.conf
  168. [2010/08/24 15:21:22] {main} Angel started
  169. [2010/08/24 15:21:22] {main} Using config file: example.conf
  170. [2010/08/24 15:21:22] {process-monitor} Must kill=0, must start=2
  171. [2010/08/24 15:21:22] {- program: watch-date -} START
  172. [2010/08/24 15:21:22] {- program: watch-date -} RUNNING
  173. [2010/08/24 15:21:22] {- program: ls -} START
  174. [2010/08/24 15:21:22] {- program: ls -} RUNNING
  175. [2010/08/24 15:21:22] {- program: ls -} ENDED
  176. [2010/08/24 15:21:22] {- program: ls -} WAITING
  177. [2010/08/24 15:21:29] {- program: ls -} RESTART
  178. [2010/08/24 15:21:29] {- program: ls -} START
  179. [2010/08/24 15:21:29] {- program: ls -} RUNNING
  180. [2010/08/24 15:21:29] {- program: ls -} ENDED
  181. [2010/08/24 15:21:29] {- program: ls -} WAITING
  182. .. etc
  183. You can see that when the configuration is parsed, the process-monitor
  184. notices that two programs need to be started. A supervisor is started
  185. in a lightweight thread for each, and starts logging with the context
  186. `program: <program-id>`.
  187. pp
  188. `watch-date` starts up and runs. Since `watch` is a long-running process
  189. it just keeps running in the background.
  190. `ls`, meanwhile, runs and immediately ends, of course; then, the WAITING
  191. state is entered until `delay` seconds pass. Finally, the RESTART event
  192. is triggered and it is started again, ad naseum.
  193. Now, let's see what happens if we modify the config file to look like this:
  194. #watch-date {
  195. # exec = "watch date"
  196. #}
  197. ls {
  198. exec = "ls"
  199. stdout = "/tmp/ls_log"
  200. stderr = "/tmp/ls_log"
  201. delay = 7
  202. }
  203. .. and then send HUP to angel.
  204. [2010/08/24 15:33:59] {config-monitor} HUP caught, reloading config
  205. [2010/08/24 15:33:59] {process-monitor} Must kill=1, must start=0
  206. [2010/08/24 15:33:59] {- program: watch-date -} ENDED
  207. [2010/08/24 15:33:59] {- program: watch-date -} QUIT
  208. [2010/08/24 15:34:03] {- program: ls -} RESTART
  209. [2010/08/24 15:34:03] {- program: ls -} START
  210. [2010/08/24 15:34:03] {- program: ls -} RUNNING
  211. [2010/08/24 15:34:03] {- program: ls -} ENDED
  212. [2010/08/24 15:34:03] {- program: ls -} WAITING
  213. As you can see, the config monitor reloaded on HUP, and then the
  214. process monitor marked the watch-date process for killing. TERM
  215. was sent to the child process, and then the supervisor loop QUIT
  216. because the watch-date program no longer had a config entry.
  217. This also works for when you specify count. Incrementing/decrementing the count
  218. will intelligently shut down excess processes and spin new ones up.
  219. Advanced Configuration
  220. ----------------------
  221. The `configurator` package supports `import` statements, as
  222. well as environment variable expansion. Using collections
  223. of configuration files and host-based or service-based
  224. environment variables, efficient, templated `angel`
  225. configurations can be had.
  226. Testing
  227. -------
  228. If you prefer to stick with haskell tools, use cabal to build the package.
  229. You can run the test suite with
  230. ```
  231. cabal test
  232. ```
  233. FAQ
  234. ---
  235. **Can I have multiple programs logging to the same file?**
  236. Yes, angel `dup()`s file descriptors and makes effort to safely
  237. allow concurrent writes by child programs; you should DEFINITELY
  238. make sure your child program is doing stdout/stderr writes in
  239. line-buffered mode so this doesn't result in a complete interleaved
  240. mess in the log file.
  241. **Will angel restart programs for me?**
  242. No; the design is just to send your programs TERM, then `angel` will
  243. restart them. `angel` tries to work in harmony with traditional
  244. Unix process management conventions.
  245. **How can I take a service down without wiping out its configuration?**
  246. Specify a `count` of 0 for the process. That will kill any running processes
  247. but still let you keep it in the config file.
  248. CHANGELOG
  249. ---------
  250. See [changelog.md](changelog.md)
  251. Author
  252. ------
  253. Original Author: Jamie Turner <jamie@jamwt.com>
  254. Current Maintainer: Michael Xavier <michael@michaelxavier.net>
  255. Thanks to Bump Technologies, Inc. (http://bu.mp) for sponsoring some
  256. of the work on angel.
  257. And, of course, thanks to all Angel's contributors:
  258. https://github.com/MichaelXavier/Angel/contributors