/README.md
Markdown | 332 lines | 258 code | 74 blank | 0 comment | 0 complexity | 0ef7635d7b8b2d84c1bddfb04c69942a MD5 | raw file
1Angel 2===== 3[](https://travis-ci.org/MichaelXavier/Angel) 4 5`angel` is a daemon that runs and monitors other processes. It 6is similar to djb's `daemontools` or the Ruby project `god`. 7 8It's goals are to keep a set of services running, and to facilitate 9the easy configuration and restart of those services. 10 11 12Maintainers Wanted 13------------------ 14I do not actively use Angel anymore and don't have much time to work 15on it. If you are an invested user in Angel, you shold have a say in 16the direction of the project. Let me know in a Github issue and I will 17happily add you to the project. 18 19 20Motivation 21---------- 22 23The author is a long-time user of `daemontools` due to its reliability 24and simplicity; however, `daemontools` is quirky and follows many 25unusual conventions. 26 27`angel` is an attempt to recreate `daemontools`'s capabilities (though 28not the various bundled utility programs which are still quite useful) 29in a more intuitive and modern unix style. 30 31 32Functionality 33------------- 34 35`angel` is driven by a configuration file that contains a list of 36program specifications to run. `angel` assumes every program listed in 37the specification file should be running at all times. 38 39`angel` starts each program, and optionally sets the program's stdout 40and stderr to some file(s) which have been opened in append mode 41(or pipes stdout and stderr to some logger process); at 42this point, the program is said to be "supervised". 43 44If the program dies for any reason, `angel` waits a specified number 45of seconds (default, 5), then restarts the program. 46 47The `angel` process itself will respond to a HUP signal by 48re-processing its configuration file, and synchronizing the run 49states with the new configuration. Specifically: 50 51 * If a new program has been added to the file, it is started and 52 supervised 53 * If a program's specification has changed (command line path, 54 stdin/stdout path, delay time, etc) that supervised child 55 process will be sent a TERM signal, and as a consequence of 56 normal supervision, will be restarted with the updated spec 57 * If a program has been removed from the configuration file, 58 the corresponding child process will be sent a TERM signal; 59 when it dies, supervision of the process will end, and 60 therefore, it will not be restarted 61 62Safety and Reliability 63---------------------- 64 65Because of `angel`'s role in policing the behavior of other 66daemons, it has been written to be very reliable: 67 68 * It is written in Haskell, which boasts a combination of 69 strong, static typing and purity-by-default that lends 70 itself to very low bug counts 71 * It uses multiple, simple, independent lightweight threads 72 with specific roles, ownership, and interfaces 73 * It uses STM for mutex-free state synchronization between 74 these threads 75 * It falls back to polling behavior to ensure eventual 76 synchronization between configuration state and run 77 state, just in case odd timing issues should make 78 event-triggered changes fail 79 * It simply logs errors and keeps running the last good 80 configuration if it runs into problems on configuration 81 reloads 82 * It has logged hundreds of thousands of uptime-hours 83 since 2010-07 supervising all the daemons that power 84 http://bu.mp without a single memory leak or crash 85 86Building 87-------- 88 89 1. Install the haskell-platform (or somehow, ghc 7.6 + 90 cabal-install) 91 2. Run `cabal install` in the project root (this directory) 92 3. Either add the ~/.cabal/bin file to your $PATH or copy 93 the `angel` executable to /usr/local/bin 94 95Notes: 96 97 * Angel is recommended to be built on GHC 7.6 and newer. 98 99Configuration and Usage Example 100------------------------------- 101 102The `angel` executable takes a path to an angel configuration 103file. 104 105 angel --help 106 angel - Process management and supervision daemon 107 108 Usage: angel CONFIG_FILE [-u USER] [-v VERBOSITY] 109 110 Available options: 111 -h,--help Show this help text 112 -u USER Execute as this user 113 -v VERBOSITY Verbosity from 0-2 (default: 2) 114 115If the -u option is specified on the command line, it will take precedence over 116any configuration command in the configuration file. 117 118`angel`'s configuration system is based on Bryan O'Sullivan's `configurator` 119package. A full description of the format can be found here: 120 121http://hackage.haskell.org/packages/archive/configurator/0.1.0.0/doc/html/Data-Configurator.html 122 123A basic configuration file might look like this: 124 125 #user is optional with a default of the current user 126 user = "alice" 127 128 watch-date { 129 exec = "watch date" 130 } 131 132 ls { 133 exec = "ls" 134 stdout = "/tmp/ls_log" 135 stderr = "/tmp/ls_log" 136 delay = 7 137 termgrace = off 138 } 139 140 workers { 141 directory = "/path/to/worker" 142 exec = "run_worker" 143 count = 30 144 pidfile = "/path/to/pidfile.pid" 145 env { 146 FOO = "BAR" 147 BAR = "BAZ" 148 } 149 termgrace = 10 150 } 151 152By adding a "user" configuration command at the top level of the 153configuration it is possible to specify the user Angel will be executed as. 154Each of the programs listed in the specification file will also be executed 155as this user. This option is only read on first start up, and is not re-read 156if the configuration file changes. 157 158The user configuration command is ignored if a user is specified on the 159command line via the -u option. 160 161Angel will run as the invoking user if no user configuration command 162is specified. 163 164Each program that should be supervised starts a `program-id` block: 165 166 watch-date { 167 168Then, a series of corresponding configuration commands follow: 169 170 * `exec` is the exact command line to run (required) 171 * `stdout` is a path to a file where the program's standard output 172 should be appended (optional, defaults to /dev/null) 173 * `stderr` is a path to a file where the program's standard error 174 should be appended (optional, defaults to /dev/null) 175 * `delay` is the number of seconds (integer) `angel` should wait 176 after the program dies before attempting to start it again 177 (optional, defaults to 5) 178 * `directory` is the current working directory of the newly 179 executed program (optional, defaults to angel's cwd) 180 * `logger` is another process that should be launched to handle 181 logging. The `exec` process will then have its stdout and stderr 182 piped into stdin of this logger. Recommended log 183 rotation daemons include [clog](https://github.com/jamwt/clog) 184 or [multilog](http://cr.yp.to/daemontools.html). *Note that 185 if you use a logger process, it is a configuration error 186 to specify either stdout or stderr as well.* 187 * `count` is an optional argument to specify the number of processes to spawn. 188 For instance, if you specified a count of 2, it will spawn the program 189 twice, internally as `workers-1` and `workers-2`, for example. Note that 190 `count` will inject the environment variable `ANGEL_PROCESS_NUMBER` into the 191 child process' environment variable. 192 * `pidfile` is an optional argument to specify where a pidfile should be 193 created. If you don't specify an absolute path, it will use the running 194 directory of angel. When combined with the `count` option, specifying a 195 pidfile of `worker.pid`, it will generate `worker-1.pid`, `worker-2.pid`, 196 etc. If you don't specify a `pidfile` directive, then `angel` will *not* 197 create a pidfile 198 * `env` is a nested config of string key/value pairs. Non-string values are 199 invalid. 200 * `termgrace` is an optional number of seconds to wait between 201 sending a SIGTERM and a SIGKILL to a program when it needs to shut 202 down. Any positive number will be interpreted as seconds. `0`, 203 `off`, or omission will be interpreted as disabling the feature and 204 only a sigterm will be sent. This is useful for processes that must 205 not be brought down forcefully to avoid corruption of data or other 206 ill effects. 207 208Assuming the above configuration was in a file called "example.conf", 209here's what a shell session might look like: 210 211 jamie@choo:~/random/angel$ angel example.conf 212 [2010/08/24 15:21:22] {main} Angel started 213 [2010/08/24 15:21:22] {main} Using config file: example.conf 214 [2010/08/24 15:21:22] {process-monitor} Must kill=0, must start=2 215 [2010/08/24 15:21:22] {- program: watch-date -} START 216 [2010/08/24 15:21:22] {- program: watch-date -} RUNNING 217 [2010/08/24 15:21:22] {- program: ls -} START 218 [2010/08/24 15:21:22] {- program: ls -} RUNNING 219 [2010/08/24 15:21:22] {- program: ls -} ENDED 220 [2010/08/24 15:21:22] {- program: ls -} WAITING 221 [2010/08/24 15:21:29] {- program: ls -} RESTART 222 [2010/08/24 15:21:29] {- program: ls -} START 223 [2010/08/24 15:21:29] {- program: ls -} RUNNING 224 [2010/08/24 15:21:29] {- program: ls -} ENDED 225 [2010/08/24 15:21:29] {- program: ls -} WAITING 226 227.. etc 228 229You can see that when the configuration is parsed, the process-monitor 230notices that two programs need to be started. A supervisor is started 231in a lightweight thread for each, and starts logging with the context 232`program: <program-id>`. 233pp 234`watch-date` starts up and runs. Since `watch` is a long-running process 235it just keeps running in the background. 236 237`ls`, meanwhile, runs and immediately ends, of course; then, the WAITING 238state is entered until `delay` seconds pass. Finally, the RESTART event 239is triggered and it is started again, ad naseum. 240 241Now, let's see what happens if we modify the config file to look like this: 242 243 #watch-date { 244 # exec = "watch date" 245 #} 246 247 ls { 248 exec = "ls" 249 stdout = "/tmp/ls_log" 250 stderr = "/tmp/ls_log" 251 delay = 7 252 } 253 254.. and then send HUP to angel. 255 256 [2010/08/24 15:33:59] {config-monitor} HUP caught, reloading config 257 [2010/08/24 15:33:59] {process-monitor} Must kill=1, must start=0 258 [2010/08/24 15:33:59] {- program: watch-date -} ENDED 259 [2010/08/24 15:33:59] {- program: watch-date -} QUIT 260 [2010/08/24 15:34:03] {- program: ls -} RESTART 261 [2010/08/24 15:34:03] {- program: ls -} START 262 [2010/08/24 15:34:03] {- program: ls -} RUNNING 263 [2010/08/24 15:34:03] {- program: ls -} ENDED 264 [2010/08/24 15:34:03] {- program: ls -} WAITING 265 266As you can see, the config monitor reloaded on HUP, and then the 267process monitor marked the watch-date process for killing. TERM 268was sent to the child process, and then the supervisor loop QUIT 269because the watch-date program no longer had a config entry. 270 271This also works for when you specify count. Incrementing/decrementing the count 272will intelligently shut down excess processes and spin new ones up. 273 274Advanced Configuration 275---------------------- 276 277The `configurator` package supports `import` statements, as 278well as environment variable expansion. Using collections 279of configuration files and host-based or service-based 280environment variables, efficient, templated `angel` 281configurations can be had. 282 283Testing 284------- 285If you prefer to stick with haskell tools, use cabal to build the package. 286 287 288You can run the test suite with 289 290``` 291cabal test 292``` 293 294FAQ 295--- 296 297**Can I have multiple programs logging to the same file?** 298 299Yes, angel `dup()`s file descriptors and makes effort to safely 300allow concurrent writes by child programs; you should DEFINITELY 301make sure your child program is doing stdout/stderr writes in 302line-buffered mode so this doesn't result in a complete interleaved 303mess in the log file. 304 305**Will angel restart programs for me?** 306 307No; the design is just to send your programs TERM, then `angel` will 308restart them. `angel` tries to work in harmony with traditional 309Unix process management conventions. 310 311**How can I take a service down without wiping out its configuration?** 312 313Specify a `count` of 0 for the process. That will kill any running processes 314but still let you keep it in the config file. 315 316CHANGELOG 317--------- 318 319See [changelog.md](changelog.md) 320 321Author 322------ 323 324Original Author: Jamie Turner <jamie@jamwt.com> 325Current Maintainer: Michael Xavier <michael@michaelxavier.net> 326 327Thanks to Bump Technologies, Inc. (http://bu.mp) for sponsoring some 328of the work on angel. 329 330And, of course, thanks to all Angel's contributors: 331 332https://github.com/MichaelXavier/Angel/contributors