/README.md
Markdown | 332 lines | 258 code | 74 blank | 0 comment | 0 complexity | 0ef7635d7b8b2d84c1bddfb04c69942a MD5 | raw file
Possible License(s): BSD-3-Clause
- Angel
- =====
- [](https://travis-ci.org/MichaelXavier/Angel)
- `angel` is a daemon that runs and monitors other processes. It
- is similar to djb's `daemontools` or the Ruby project `god`.
- It's goals are to keep a set of services running, and to facilitate
- the easy configuration and restart of those services.
- Maintainers Wanted
- ------------------
- I do not actively use Angel anymore and don't have much time to work
- on it. If you are an invested user in Angel, you shold have a say in
- the direction of the project. Let me know in a Github issue and I will
- happily add you to the project.
- Motivation
- ----------
- The author is a long-time user of `daemontools` due to its reliability
- and simplicity; however, `daemontools` is quirky and follows many
- unusual conventions.
- `angel` is an attempt to recreate `daemontools`'s capabilities (though
- not the various bundled utility programs which are still quite useful)
- in a more intuitive and modern unix style.
- Functionality
- -------------
- `angel` is driven by a configuration file that contains a list of
- program specifications to run. `angel` assumes every program listed in
- the specification file should be running at all times.
- `angel` starts each program, and optionally sets the program's stdout
- and stderr to some file(s) which have been opened in append mode
- (or pipes stdout and stderr to some logger process); at
- this point, the program is said to be "supervised".
- If the program dies for any reason, `angel` waits a specified number
- of seconds (default, 5), then restarts the program.
- The `angel` process itself will respond to a HUP signal by
- re-processing its configuration file, and synchronizing the run
- states with the new configuration. Specifically:
- * If a new program has been added to the file, it is started and
- supervised
- * If a program's specification has changed (command line path,
- stdin/stdout path, delay time, etc) that supervised child
- process will be sent a TERM signal, and as a consequence of
- normal supervision, will be restarted with the updated spec
- * If a program has been removed from the configuration file,
- the corresponding child process will be sent a TERM signal;
- when it dies, supervision of the process will end, and
- therefore, it will not be restarted
- Safety and Reliability
- ----------------------
- Because of `angel`'s role in policing the behavior of other
- daemons, it has been written to be very reliable:
- * It is written in Haskell, which boasts a combination of
- strong, static typing and purity-by-default that lends
- itself to very low bug counts
- * It uses multiple, simple, independent lightweight threads
- with specific roles, ownership, and interfaces
- * It uses STM for mutex-free state synchronization between
- these threads
- * It falls back to polling behavior to ensure eventual
- synchronization between configuration state and run
- state, just in case odd timing issues should make
- event-triggered changes fail
- * It simply logs errors and keeps running the last good
- configuration if it runs into problems on configuration
- reloads
- * It has logged hundreds of thousands of uptime-hours
- since 2010-07 supervising all the daemons that power
- http://bu.mp without a single memory leak or crash
- Building
- --------
- 1. Install the haskell-platform (or somehow, ghc 7.6 +
- cabal-install)
- 2. Run `cabal install` in the project root (this directory)
- 3. Either add the ~/.cabal/bin file to your $PATH or copy
- the `angel` executable to /usr/local/bin
- Notes:
- * Angel is recommended to be built on GHC 7.6 and newer.
- Configuration and Usage Example
- -------------------------------
- The `angel` executable takes a path to an angel configuration
- file.
- angel --help
- angel - Process management and supervision daemon
- Usage: angel CONFIG_FILE [-u USER] [-v VERBOSITY]
- Available options:
- -h,--help Show this help text
- -u USER Execute as this user
- -v VERBOSITY Verbosity from 0-2 (default: 2)
- If the -u option is specified on the command line, it will take precedence over
- any configuration command in the configuration file.
- `angel`'s configuration system is based on Bryan O'Sullivan's `configurator`
- package. A full description of the format can be found here:
- http://hackage.haskell.org/packages/archive/configurator/0.1.0.0/doc/html/Data-Configurator.html
- A basic configuration file might look like this:
- #user is optional with a default of the current user
- user = "alice"
- watch-date {
- exec = "watch date"
- }
- ls {
- exec = "ls"
- stdout = "/tmp/ls_log"
- stderr = "/tmp/ls_log"
- delay = 7
- termgrace = off
- }
- workers {
- directory = "/path/to/worker"
- exec = "run_worker"
- count = 30
- pidfile = "/path/to/pidfile.pid"
- env {
- FOO = "BAR"
- BAR = "BAZ"
- }
- termgrace = 10
- }
- By adding a "user" configuration command at the top level of the
- configuration it is possible to specify the user Angel will be executed as.
- Each of the programs listed in the specification file will also be executed
- as this user. This option is only read on first start up, and is not re-read
- if the configuration file changes.
- The user configuration command is ignored if a user is specified on the
- command line via the -u option.
- Angel will run as the invoking user if no user configuration command
- is specified.
- Each program that should be supervised starts a `program-id` block:
- watch-date {
- Then, a series of corresponding configuration commands follow:
- * `exec` is the exact command line to run (required)
- * `stdout` is a path to a file where the program's standard output
- should be appended (optional, defaults to /dev/null)
- * `stderr` is a path to a file where the program's standard error
- should be appended (optional, defaults to /dev/null)
- * `delay` is the number of seconds (integer) `angel` should wait
- after the program dies before attempting to start it again
- (optional, defaults to 5)
- * `directory` is the current working directory of the newly
- executed program (optional, defaults to angel's cwd)
- * `logger` is another process that should be launched to handle
- logging. The `exec` process will then have its stdout and stderr
- piped into stdin of this logger. Recommended log
- rotation daemons include [clog](https://github.com/jamwt/clog)
- or [multilog](http://cr.yp.to/daemontools.html). *Note that
- if you use a logger process, it is a configuration error
- to specify either stdout or stderr as well.*
- * `count` is an optional argument to specify the number of processes to spawn.
- For instance, if you specified a count of 2, it will spawn the program
- twice, internally as `workers-1` and `workers-2`, for example. Note that
- `count` will inject the environment variable `ANGEL_PROCESS_NUMBER` into the
- child process' environment variable.
- * `pidfile` is an optional argument to specify where a pidfile should be
- created. If you don't specify an absolute path, it will use the running
- directory of angel. When combined with the `count` option, specifying a
- pidfile of `worker.pid`, it will generate `worker-1.pid`, `worker-2.pid`,
- etc. If you don't specify a `pidfile` directive, then `angel` will *not*
- create a pidfile
- * `env` is a nested config of string key/value pairs. Non-string values are
- invalid.
- * `termgrace` is an optional number of seconds to wait between
- sending a SIGTERM and a SIGKILL to a program when it needs to shut
- down. Any positive number will be interpreted as seconds. `0`,
- `off`, or omission will be interpreted as disabling the feature and
- only a sigterm will be sent. This is useful for processes that must
- not be brought down forcefully to avoid corruption of data or other
- ill effects.
- Assuming the above configuration was in a file called "example.conf",
- here's what a shell session might look like:
- jamie@choo:~/random/angel$ angel example.conf
- [2010/08/24 15:21:22] {main} Angel started
- [2010/08/24 15:21:22] {main} Using config file: example.conf
- [2010/08/24 15:21:22] {process-monitor} Must kill=0, must start=2
- [2010/08/24 15:21:22] {- program: watch-date -} START
- [2010/08/24 15:21:22] {- program: watch-date -} RUNNING
- [2010/08/24 15:21:22] {- program: ls -} START
- [2010/08/24 15:21:22] {- program: ls -} RUNNING
- [2010/08/24 15:21:22] {- program: ls -} ENDED
- [2010/08/24 15:21:22] {- program: ls -} WAITING
- [2010/08/24 15:21:29] {- program: ls -} RESTART
- [2010/08/24 15:21:29] {- program: ls -} START
- [2010/08/24 15:21:29] {- program: ls -} RUNNING
- [2010/08/24 15:21:29] {- program: ls -} ENDED
- [2010/08/24 15:21:29] {- program: ls -} WAITING
- .. etc
- You can see that when the configuration is parsed, the process-monitor
- notices that two programs need to be started. A supervisor is started
- in a lightweight thread for each, and starts logging with the context
- `program: <program-id>`.
- pp
- `watch-date` starts up and runs. Since `watch` is a long-running process
- it just keeps running in the background.
- `ls`, meanwhile, runs and immediately ends, of course; then, the WAITING
- state is entered until `delay` seconds pass. Finally, the RESTART event
- is triggered and it is started again, ad naseum.
- Now, let's see what happens if we modify the config file to look like this:
- #watch-date {
- # exec = "watch date"
- #}
- ls {
- exec = "ls"
- stdout = "/tmp/ls_log"
- stderr = "/tmp/ls_log"
- delay = 7
- }
- .. and then send HUP to angel.
- [2010/08/24 15:33:59] {config-monitor} HUP caught, reloading config
- [2010/08/24 15:33:59] {process-monitor} Must kill=1, must start=0
- [2010/08/24 15:33:59] {- program: watch-date -} ENDED
- [2010/08/24 15:33:59] {- program: watch-date -} QUIT
- [2010/08/24 15:34:03] {- program: ls -} RESTART
- [2010/08/24 15:34:03] {- program: ls -} START
- [2010/08/24 15:34:03] {- program: ls -} RUNNING
- [2010/08/24 15:34:03] {- program: ls -} ENDED
- [2010/08/24 15:34:03] {- program: ls -} WAITING
- As you can see, the config monitor reloaded on HUP, and then the
- process monitor marked the watch-date process for killing. TERM
- was sent to the child process, and then the supervisor loop QUIT
- because the watch-date program no longer had a config entry.
- This also works for when you specify count. Incrementing/decrementing the count
- will intelligently shut down excess processes and spin new ones up.
- Advanced Configuration
- ----------------------
- The `configurator` package supports `import` statements, as
- well as environment variable expansion. Using collections
- of configuration files and host-based or service-based
- environment variables, efficient, templated `angel`
- configurations can be had.
- Testing
- -------
- If you prefer to stick with haskell tools, use cabal to build the package.
- You can run the test suite with
- ```
- cabal test
- ```
- FAQ
- ---
- **Can I have multiple programs logging to the same file?**
- Yes, angel `dup()`s file descriptors and makes effort to safely
- allow concurrent writes by child programs; you should DEFINITELY
- make sure your child program is doing stdout/stderr writes in
- line-buffered mode so this doesn't result in a complete interleaved
- mess in the log file.
- **Will angel restart programs for me?**
- No; the design is just to send your programs TERM, then `angel` will
- restart them. `angel` tries to work in harmony with traditional
- Unix process management conventions.
- **How can I take a service down without wiping out its configuration?**
- Specify a `count` of 0 for the process. That will kill any running processes
- but still let you keep it in the config file.
- CHANGELOG
- ---------
- See [changelog.md](changelog.md)
- Author
- ------
- Original Author: Jamie Turner <jamie@jamwt.com>
- Current Maintainer: Michael Xavier <michael@michaelxavier.net>
- Thanks to Bump Technologies, Inc. (http://bu.mp) for sponsoring some
- of the work on angel.
- And, of course, thanks to all Angel's contributors:
- https://github.com/MichaelXavier/Angel/contributors