PageRenderTime 51ms CodeModel.GetById 20ms RepoModel.GetById 0ms app.codeStats 0ms

/collectors/node.d.plugin/README.md

https://gitlab.com/centminmod-github-mirror/netdata
Markdown | 218 lines | 153 code | 65 blank | 0 comment | 0 complexity | 31f1aeeddd5cd39c48da250ee2501777 MD5 | raw file
  1. # node.d.plugin
  2. `node.d.plugin` is a netdata external plugin. It is an **orchestrator** for data collection modules written in `node.js`.
  3. 1. It runs as an independent process `ps fax` shows it
  4. 2. It is started and stopped automatically by netdata
  5. 3. It communicates with netdata via a unidirectional pipe (sending data to the netdata daemon)
  6. 4. Supports any number of data collection **modules**
  7. 5. Allows each **module** to have one or more data collection **jobs**
  8. 6. Each **job** is collecting one or more metrics from a single data source
  9. # Motivation
  10. Node.js is perfect for asynchronous operations. It is very fast and quite common (actually the whole web is based on it).
  11. Since data collection is not a CPU intensive task, node.js is an ideal solution for it.
  12. `node.d.plugin` is a netdata plugin that provides an abstraction layer to allow easy and quick development of data
  13. collectors in node.js. It also manages all its data collectors (placed in `/usr/libexec/netdata/node.d`) using a single
  14. instance of node, thus lowering the memory footprint of data collection.
  15. Of course, there can be independent plugins written in node.js (placed in `/usr/libexec/netdata/plugins`).
  16. These will have to be developed using the guidelines of **[External Plugins](../plugins.d/)**.
  17. To run `node.js` plugins you need to have `node` installed in your system.
  18. In some older systems, the package named `node` is not node.js. It is a terminal emulation program called `ax25-node`.
  19. In this case the node.js package may be referred as `nodejs`. Once you install `nodejs`, we suggest to link
  20. `/usr/bin/nodejs` to `/usr/bin/node`, so that typing `node` in your terminal, opens node.js.
  21. For more information check the **[[Installation]]** guide.
  22. ## configuring `node.d.plugin`
  23. `node.d.plugin` can work even without any configuration. Its default configuration file is
  24. [/etc/netdata/node.d.conf](node.d.conf) (to edit it on your system run `/etc/netdata/edit-config node.d.conf`).
  25. ## configuring `node.d.plugin` modules
  26. `node.d.plugin` modules accept configuration in `JSON` format.
  27. Unfortunately, `JSON` files do not accept comments. So, the best way to describe them is to have markdown text files
  28. with instructions.
  29. `JSON` has a very strict formatting. If you get errors from netdata at `/var/log/netdata/error.log` that a certain
  30. configuration file cannot be loaded, we suggest to verify it at [http://jsonlint.com/](http://jsonlint.com/).
  31. The files in this directory, provide usable examples for configuring each `node.d.plugin` module.
  32. ## debugging modules written for node.d.plugin
  33. To test `node.d.plugin` modules, which are placed in `/usr/libexec/netdata/node.d`, you can run `node.d.plugin` by hand,
  34. like this:
  35. ```sh
  36. # become user netdata
  37. sudo su -s /bin/sh netdata
  38. # run the plugin in debug mode
  39. /usr/libexec/netdata/plugins.d/node.d.plugin debug 1 X Y Z
  40. ```
  41. `node.d.plugin` will run in `debug` mode (lots of debug info), with an update frequency of `1` second, evaluating only
  42. the collector scripts `X` (i.e. `/usr/libexec/netdata/node.d/X.node.js`), `Y` and `Z`.
  43. You can define zero or more modules. If none is defined, `node.d.plugin` will evaluate all modules available.
  44. Keep in mind that if your configs are not in `/etc/netdata`, you should do the following before running `node.d.plugin`:
  45. ```sh
  46. export NETDATA_USER_CONFIG_DIR="/path/to/etc/netdata"
  47. ```
  48. ---
  49. ## developing `node.d.plugin` modules
  50. Your data collection module should be split in 3 parts:
  51. - a function to fetch the data from its source. `node.d.plugin` already can fetch data from web sources,
  52. so you don't need to do anything about it for http.
  53. - a function to process the fetched/manipulate the data fetched. This function will make a number of calls
  54. to create charts and dimensions and pass the collected values to netdata.
  55. This is the only function you need to write for collecting http JSON data.
  56. - a `configure` and an `update` function, which take care of your module configuration and data refresh
  57. respectively. You can use the supplied ones.
  58. Your module will automatically be able to process any number of servers, with different settings (even different
  59. data collection frequencies). You will write just the work needed for one and `node.d.plugin` will do the rest.
  60. For each server you are going to fetch data from, you will have to create a `service` (more later).
  61. ### writing the data collection module
  62. To provide a module called `mymodule`, you have create the file `/usr/libexec/netdata/node.d/mymodule.node.js`, with this structure:
  63. ```js
  64. // the processor is needed only
  65. // if you need a custom processor
  66. // other than http
  67. netdata.processors.myprocessor = {
  68. name: 'myprocessor',
  69. process: function(service, callback) {
  70. /* do data collection here */
  71. callback(data);
  72. }
  73. };
  74. // this is the mymodule definition
  75. var mymodule = {
  76. processResponse: function(service, data) {
  77. /* send information to the netdata server here */
  78. },
  79. configure: function(config) {
  80. var eligible_services = 0;
  81. if(typeof(config.servers) === 'undefined' || config.servers.length === 0) {
  82. /*
  83. * create a service using internal defaults;
  84. * this is used for auto-detecting the settings
  85. * if possible
  86. */
  87. netdata.service({
  88. name: 'a name for this service',
  89. update_every: this.update_every,
  90. module: this,
  91. processor: netdata.processors.myprocessor,
  92. // any other information your processor needs
  93. }).execute(this.processResponse);
  94. eligible_services++;
  95. }
  96. else {
  97. /*
  98. * create a service for each server in the
  99. * configuration file
  100. */
  101. var len = config.servers.length;
  102. while(len--) {
  103. var server = config.servers[len];
  104. netdata.service({
  105. name: server.name,
  106. update_every: server.update_every,
  107. module: this,
  108. processor: netdata.processors.myprocessor,
  109. // any other information your processor needs
  110. }).execute(this.processResponse);
  111. eligible_services++;
  112. }
  113. }
  114. return eligible_services;
  115. },
  116. update: function(service, callback) {
  117. /*
  118. * this function is called when each service
  119. * created by the configure function, needs to
  120. * collect updated values.
  121. *
  122. * You normally will not need to change it.
  123. */
  124. service.execute(function(service, data) {
  125. mymodule.processResponse(service, data);
  126. callback();
  127. });
  128. },
  129. };
  130. module.exports = mymodule;
  131. ```
  132. #### configure(config)
  133. `configure(config)` is called just once, when `node.d.plugin` starts.
  134. The config file will contain the contents of `/etc/netdata/node.d/mymodule.conf`.
  135. This file should have the following format:
  136. ```js
  137. {
  138. "enable_autodetect": false,
  139. "update_every": 5,
  140. "servers": [ { /* server 1 */ }, { /* server 2 */ } ]
  141. }
  142. ```
  143. If the config file `/etc/netdata/node.d/mymodule.conf` does not give a `enable_autodetect` or `update_every`, these
  144. will be added by `node.d.plugin`. So you module will always have them.
  145. The configuration file `/etc/netdata/node.d/mymodule.conf` may contain whatever else is needed for `mymodule`.
  146. #### processResponse(data)
  147. `data` may be `null` or whatever the processor specified in the `service` returned.
  148. The `service` object defines a set of functions to allow you send information to the netdata core about:
  149. 1. Charts and dimension definitions
  150. 2. Updated values, from the collected values
  151. ---
  152. *FIXME: document an operational node.d.plugin data collector - the best example is the
  153. [snmp collector](snmp/snmp.node.js)*