PageRenderTime 29ms CodeModel.GetById 1ms app.highlight 24ms RepoModel.GetById 1ms app.codeStats 0ms

/README.md

http://github.com/jamwt/Angel
Markdown | 332 lines | 258 code | 74 blank | 0 comment | 0 complexity | 0ef7635d7b8b2d84c1bddfb04c69942a MD5 | raw file
  1Angel
  2=====
  3[![Build Status](https://travis-ci.org/MichaelXavier/Angel.png?branch=master)](https://travis-ci.org/MichaelXavier/Angel)
  4
  5`angel` is a daemon that runs and monitors other processes.  It
  6is similar to djb's `daemontools` or the Ruby project `god`.
  7
  8It's goals are to keep a set of services running, and to facilitate
  9the easy configuration and restart of those services.
 10
 11
 12Maintainers Wanted
 13------------------
 14I do not actively use Angel anymore and don't have much time to work
 15on it. If you are an invested user in Angel, you shold have a say in
 16the direction of the project. Let me know in a Github issue and I will
 17happily add you to the project.
 18
 19
 20Motivation
 21----------
 22
 23The author is a long-time user of `daemontools` due to its reliability
 24and simplicity; however, `daemontools` is quirky and follows many
 25unusual conventions.
 26
 27`angel` is an attempt to recreate `daemontools`'s capabilities (though
 28not the various bundled utility programs which are still quite useful)
 29in a more intuitive and modern unix style.
 30
 31
 32Functionality
 33-------------
 34
 35`angel` is driven by a configuration file that contains a list of
 36program specifications to run.  `angel` assumes every program listed in
 37the specification file should be running at all times.
 38
 39`angel` starts each program, and optionally sets the program's stdout
 40and stderr to some file(s) which have been opened in append mode
 41(or pipes stdout and stderr to some logger process); at
 42this point, the program is said to be "supervised".
 43
 44If the program dies for any reason, `angel` waits a specified number
 45of seconds (default, 5), then restarts the program.
 46
 47The `angel` process itself will respond to a HUP signal by
 48re-processing its configuration file, and synchronizing the run
 49states with the new configuration.  Specifically:
 50
 51 * If a new program has been added to the file, it is started and
 52   supervised
 53 * If a program's specification has changed (command line path,
 54   stdin/stdout path, delay time, etc) that supervised child
 55   process will be sent a TERM signal, and as a consequence of
 56   normal supervision, will be restarted with the updated spec
 57 * If a program has been removed from the configuration file,
 58   the corresponding child process will be sent a TERM signal;
 59   when it dies, supervision of the process will end, and
 60   therefore, it will not be restarted
 61
 62Safety and Reliability
 63----------------------
 64
 65Because of `angel`'s role in policing the behavior of other
 66daemons, it has been written to be very reliable:
 67
 68 * It is written in Haskell, which boasts a combination of
 69   strong, static typing and purity-by-default that lends
 70   itself to very low bug counts
 71 * It uses multiple, simple, independent lightweight threads
 72   with specific roles, ownership, and interfaces
 73 * It uses STM for mutex-free state synchronization between
 74   these threads
 75 * It falls back to polling behavior to ensure eventual
 76   synchronization between configuration state and run
 77   state, just in case odd timing issues should make
 78   event-triggered changes fail
 79 * It simply logs errors and keeps running the last good
 80   configuration if it runs into problems on configuration
 81   reloads
 82 * It has logged hundreds of thousands of uptime-hours
 83   since 2010-07 supervising all the daemons that power
 84   http://bu.mp without a single memory leak or crash
 85
 86Building
 87--------
 88
 89 1. Install the haskell-platform (or somehow, ghc 7.6 +
 90    cabal-install)
 91 2. Run `cabal install` in the project root (this directory)
 92 3. Either add the ~/.cabal/bin file to your $PATH or copy
 93    the `angel` executable to /usr/local/bin
 94
 95Notes:
 96
 97 * Angel is recommended to be built on GHC 7.6 and newer.
 98
 99Configuration and Usage Example
100-------------------------------
101
102The `angel` executable takes a path to an angel configuration
103file.
104
105    angel --help
106    angel - Process management and supervision daemon
107
108    Usage: angel CONFIG_FILE [-u USER] [-v VERBOSITY]
109
110    Available options:
111      -h,--help                Show this help text
112      -u USER                  Execute as this user
113      -v VERBOSITY             Verbosity from 0-2 (default: 2)
114
115If the -u option is specified on the command line, it will take precedence over
116any configuration command in the configuration file.
117
118`angel`'s configuration system is based on Bryan O'Sullivan's `configurator`
119package.  A full description of the format can be found here:
120
121http://hackage.haskell.org/packages/archive/configurator/0.1.0.0/doc/html/Data-Configurator.html
122
123A basic configuration file might look like this:
124
125    #user is optional with a default of the current user
126    user = "alice"
127
128    watch-date {
129        exec = "watch date"
130    }
131
132    ls {
133        exec = "ls"
134        stdout = "/tmp/ls_log"
135        stderr = "/tmp/ls_log"
136        delay = 7
137        termgrace = off
138    }
139
140    workers {
141        directory = "/path/to/worker"
142        exec      = "run_worker"
143        count     = 30
144        pidfile   = "/path/to/pidfile.pid"
145        env {
146          FOO = "BAR"
147          BAR = "BAZ"
148        }
149        termgrace = 10
150    }
151
152By adding a "user" configuration command at the top level of the
153configuration it is possible to specify the user Angel will be executed as.
154Each of the programs listed in the specification file will also be executed
155as this user. This option is only read on first start up, and is not re-read
156if the configuration file changes.
157
158The user configuration command is ignored if a user is specified on the
159command line via the -u option.
160
161Angel will run as the invoking user if no user configuration command
162is specified.
163
164Each program that should be supervised starts a `program-id` block:
165
166    watch-date {
167
168Then, a series of corresponding configuration commands follow:
169
170 * `exec` is the exact command line to run (required)
171 * `stdout` is a path to a file where the program's standard output
172    should be appended (optional, defaults to /dev/null)
173 * `stderr` is a path to a file where the program's standard error
174    should be appended (optional, defaults to /dev/null)
175 * `delay` is the number of seconds (integer) `angel` should wait
176   after the program dies before attempting to start it again
177   (optional, defaults to 5)
178 * `directory` is the current working directory of the newly
179   executed program (optional, defaults to angel's cwd)
180 * `logger` is another process that should be launched to handle
181   logging.  The `exec` process will then have its stdout and stderr
182   piped into stdin of this logger.  Recommended log
183   rotation daemons include [clog](https://github.com/jamwt/clog)
184   or [multilog](http://cr.yp.to/daemontools.html). *Note that
185   if you use a logger process, it is a configuration error
186   to specify either stdout or stderr as well.*
187 * `count` is an optional argument to specify the number of processes to spawn.
188   For instance, if you specified a count of 2, it will spawn the program
189   twice, internally as `workers-1` and `workers-2`, for example. Note that
190   `count` will inject the environment variable `ANGEL_PROCESS_NUMBER` into the
191   child process' environment variable.
192 * `pidfile` is an optional argument to specify where a pidfile should be
193   created. If you don't specify an absolute path, it will use the running
194   directory of angel. When combined with the `count` option, specifying a
195   pidfile of `worker.pid`, it will generate `worker-1.pid`, `worker-2.pid`,
196   etc. If you don't specify a `pidfile` directive, then `angel` will *not*
197   create a pidfile
198 * `env` is a nested config of string key/value pairs. Non-string values are
199   invalid.
200 * `termgrace` is an optional number of seconds to wait between
201   sending a SIGTERM and a SIGKILL to a program when it needs to shut
202   down. Any positive number will be interpreted as seconds. `0`,
203   `off`, or omission will be interpreted as disabling the feature and
204   only a sigterm will be sent. This is useful for processes that must
205   not be brought down forcefully to avoid corruption of data or other
206   ill effects.
207
208Assuming the above configuration was in a file called "example.conf",
209here's what a shell session might look like:
210
211    jamie@choo:~/random/angel$ angel example.conf
212    [2010/08/24 15:21:22] {main} Angel started
213    [2010/08/24 15:21:22] {main} Using config file: example.conf
214    [2010/08/24 15:21:22] {process-monitor} Must kill=0, must start=2
215    [2010/08/24 15:21:22] {- program: watch-date -} START
216    [2010/08/24 15:21:22] {- program: watch-date -} RUNNING
217    [2010/08/24 15:21:22] {- program: ls -} START
218    [2010/08/24 15:21:22] {- program: ls -} RUNNING
219    [2010/08/24 15:21:22] {- program: ls -} ENDED
220    [2010/08/24 15:21:22] {- program: ls -} WAITING
221    [2010/08/24 15:21:29] {- program: ls -} RESTART
222    [2010/08/24 15:21:29] {- program: ls -} START
223    [2010/08/24 15:21:29] {- program: ls -} RUNNING
224    [2010/08/24 15:21:29] {- program: ls -} ENDED
225    [2010/08/24 15:21:29] {- program: ls -} WAITING
226
227.. etc
228
229You can see that when the configuration is parsed, the process-monitor
230notices that two programs need to be started.  A supervisor is started
231in a lightweight thread for each, and starts logging with the context
232`program: <program-id>`.
233pp
234`watch-date` starts up and runs.  Since `watch` is a long-running process
235it just keeps running in the background.
236
237`ls`, meanwhile, runs and immediately ends, of course; then, the WAITING
238state is entered until `delay` seconds pass.  Finally, the RESTART event
239is triggered and it is started again, ad naseum.
240
241Now, let's see what happens if we modify the config file to look like this:
242
243    #watch-date {
244    #    exec = "watch date"
245    #}
246
247    ls {
248        exec = "ls"
249        stdout = "/tmp/ls_log"
250        stderr = "/tmp/ls_log"
251        delay = 7
252    }
253
254.. and then send HUP to angel.
255
256    [2010/08/24 15:33:59] {config-monitor} HUP caught, reloading config
257    [2010/08/24 15:33:59] {process-monitor} Must kill=1, must start=0
258    [2010/08/24 15:33:59] {- program: watch-date -} ENDED
259    [2010/08/24 15:33:59] {- program: watch-date -} QUIT
260    [2010/08/24 15:34:03] {- program: ls -} RESTART
261    [2010/08/24 15:34:03] {- program: ls -} START
262    [2010/08/24 15:34:03] {- program: ls -} RUNNING
263    [2010/08/24 15:34:03] {- program: ls -} ENDED
264    [2010/08/24 15:34:03] {- program: ls -} WAITING
265
266As you can see, the config monitor reloaded on HUP, and then the
267process monitor marked the watch-date process for killing.  TERM
268was sent to the child process, and then the supervisor loop QUIT
269because the watch-date program no longer had a config entry.
270
271This also works for when you specify count. Incrementing/decrementing the count
272will intelligently shut down excess processes and spin new ones up.
273
274Advanced Configuration
275----------------------
276
277The `configurator` package supports `import` statements, as
278well as environment variable expansion.  Using collections
279of configuration files and host-based or service-based
280environment variables, efficient, templated `angel`
281configurations can be had.
282
283Testing
284-------
285If you prefer to stick with haskell tools, use cabal to build the package.
286
287
288You can run the test suite with
289
290```
291cabal test
292```
293
294FAQ
295---
296
297**Can I have multiple programs logging to the same file?**
298
299Yes, angel `dup()`s file descriptors and makes effort to safely
300allow concurrent writes by child programs; you should DEFINITELY
301make sure your child program is doing stdout/stderr writes in
302line-buffered mode so this doesn't result in a complete interleaved
303mess in the log file.
304
305**Will angel restart programs for me?**
306
307No; the design is just to send your programs TERM, then `angel` will
308restart them.  `angel` tries to work in harmony with traditional
309Unix process management conventions.
310
311**How can I take a service down without wiping out its configuration?**
312
313Specify a `count` of 0 for the process. That will kill any running processes
314but still let you keep it in the config file.
315
316CHANGELOG
317---------
318
319See [changelog.md](changelog.md)
320
321Author
322------
323
324Original Author: Jamie Turner <jamie@jamwt.com>
325Current Maintainer: Michael Xavier <michael@michaelxavier.net>
326
327Thanks to Bump Technologies, Inc. (http://bu.mp) for sponsoring some
328of the work on angel.
329
330And, of course, thanks to all Angel's contributors:
331
332https://github.com/MichaelXavier/Angel/contributors