/share/man/man4/siftr.4

https://bitbucket.org/freebsd/freebsd-head/ · Forth · 761 lines · 761 code · 0 blank · 0 comment · 39 complexity · 2a756cffe2480702b2d67f0d1493e6be MD5 · raw file

  1. .\"
  2. .\" Copyright (c) 2010 The FreeBSD Foundation
  3. .\" All rights reserved.
  4. .\"
  5. .\" Portions of this software were developed at the Centre for Advanced
  6. .\" Internet Architectures, Swinburne University of Technology, Melbourne,
  7. .\" Australia by Lawrence Stewart under sponsorship from the FreeBSD
  8. .\" Foundation.
  9. .\"
  10. .\" Redistribution and use in source and binary forms, with or without
  11. .\" modification, are permitted provided that the following conditions
  12. .\" are met:
  13. .\" 1. Redistributions of source code must retain the above copyright
  14. .\" notice, this list of conditions, and the following disclaimer,
  15. .\" without modification, immediately at the beginning of the file.
  16. .\" 2. The name of the author may not be used to endorse or promote products
  17. .\" derived from this software without specific prior written permission.
  18. .\"
  19. .\" THIS SOFTWARE IS PROVIDED BY THE AUTHOR AND CONTRIBUTORS ``AS IS'' AND
  20. .\" ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
  21. .\" IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE
  22. .\" ARE DISCLAIMED. IN NO EVENT SHALL THE AUTHOR OR CONTRIBUTORS BE LIABLE FOR
  23. .\" ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
  24. .\" DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS
  25. .\" OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION)
  26. .\" HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
  27. .\" LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY
  28. .\" OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF
  29. .\" SUCH DAMAGE.
  30. .\"
  31. .\" $FreeBSD$
  32. .\"
  33. .Dd November 12, 2010
  34. .Dt SIFTR 4
  35. .Os
  36. .Sh NAME
  37. .Nm SIFTR
  38. .Nd Statistical Information For TCP Research
  39. .Sh SYNOPSIS
  40. To load
  41. the driver
  42. as a module at run-time, run the following command as root:
  43. .Bd -literal -offset indent
  44. kldload siftr
  45. .Ed
  46. .Pp
  47. Alternatively, to load
  48. the driver
  49. as a module at boot time, add the following line into the
  50. .Xr loader.conf 5
  51. file:
  52. .Bd -literal -offset indent
  53. siftr_load="YES"
  54. .Ed
  55. .Sh DESCRIPTION
  56. The
  57. .Nm
  58. .Po
  59. .Em S Ns tatistical
  60. .Em I Ns nformation
  61. .Em F Ns or
  62. .Em T Ns CP
  63. .Em R Ns esearch
  64. .Pc
  65. kernel module logs a range of statistics on active TCP connections to
  66. a log file.
  67. It provides the ability to make highly granular measurements of TCP connection
  68. state, aimed at system administrators, developers and researchers.
  69. .Ss Compile-time Configuration
  70. The default operation of
  71. .Nm
  72. is to capture IPv4 TCP/IP packets.
  73. .Nm
  74. can be configured to support IPv4 and IPv6 by uncommenting:
  75. .Bd -literal -offset indent
  76. CFLAGS+=-DSIFTR_IPV6
  77. .Ed
  78. .Pp
  79. in
  80. .Aq sys/modules/siftr/Makefile
  81. and recompiling.
  82. .Pp
  83. In the IPv4-only (default) mode, standard dotted decimal notation (e.g.
  84. "136.186.229.95") is used to format IPv4 addresses for logging.
  85. In IPv6 mode, standard dotted decimal notation is used to format IPv4 addresses,
  86. and standard colon-separated hex notation (see RFC 4291) is used to format IPv6
  87. addresses for logging. Note that SIFTR uses uncompressed notation to format IPv6
  88. addresses.
  89. For example, the address "fe80::20f:feff:fea2:531b" would be logged as
  90. "fe80:0:0:0:20f:feff:fea2:531b".
  91. .Ss Run-time Configuration
  92. .Nm
  93. utilises the
  94. .Xr sysctl 8
  95. interface to export its configuration variables to user-space.
  96. The following variables are available:
  97. .Bl -tag -offset indent -width Va
  98. .It Va net.inet.siftr.enabled
  99. controls whether the module performs its
  100. measurements or not.
  101. By default, the value is set to 0, which means the module
  102. will not be taking any measurements.
  103. Having the module loaded with
  104. .Va net.inet.siftr.enabled
  105. set to 0 will have no impact on the performance of the network stack, as the
  106. packet filtering hooks are only inserted when
  107. .Va net.inet.siftr.enabled
  108. is set to 1.
  109. .El
  110. .Bl -tag -offset indent -width Va
  111. .It Va net.inet.siftr.ppl
  112. controls how many inbound/outbound packets for a given TCP connection will cause
  113. a log message to be generated for the connection.
  114. By default, the value is set to 1, which means the module will log a message for
  115. every packet of every TCP connection.
  116. The value can be set to any integer in the range [1,2^32], and can be changed at
  117. any time, even while the module is enabled.
  118. .El
  119. .Bl -tag -offset indent -width Va
  120. .It Va net.inet.siftr.logfile
  121. controls the path to the file that the module writes its log messages to.
  122. By default, the file /var/log/siftr.log is used.
  123. The path can be changed at any time, even while the module is enabled.
  124. .El
  125. .Bl -tag -offset indent -width Va
  126. .It Va net.inet.siftr.genhashes
  127. controls whether a hash is generated for each TCP packet seen by
  128. .Nm .
  129. By default, the value is set to 0, which means no hashes are generated.
  130. The hashes are useful to correlate which TCP packet triggered the generation of
  131. a particular log message, but calculating them adds additional computational
  132. overhead into the fast path.
  133. .El
  134. .Ss Log Format
  135. A typical
  136. .Nm
  137. log file will contain 3 different types of log message.
  138. All messages are written in plain ASCII text.
  139. .Pp
  140. Note: The
  141. .Qq \e
  142. present in the example log messages in this section indicates a
  143. line continuation and is not part of the actual log message.
  144. .Pp
  145. The first type of log message is written to the file when the module is
  146. enabled and starts collecting data from the running kernel. The text below
  147. shows an example module enable log. The fields are tab delimited key-value
  148. pairs which describe some basic information about the system.
  149. .Bd -literal -offset indent
  150. enable_time_secs=1238556193 enable_time_usecs=462104 \\
  151. siftrver=1.2.2 hz=1000 tcp_rtt_scale=32 \\
  152. sysname=FreeBSD sysver=604000 ipmode=4
  153. .Ed
  154. .Pp
  155. Field descriptions are as follows:
  156. .Bl -tag -offset indent -width Va
  157. .It Va enable_time_secs
  158. time at which the module was enabled, in seconds since the UNIX epoch.
  159. .El
  160. .Bl -tag -offset indent -width Va
  161. .It Va enable_time_usecs
  162. time at which the module was enabled, in microseconds since enable_time_secs.
  163. .El
  164. .Bl -tag -offset indent -width Va
  165. .It Va siftrver
  166. version of
  167. .Nm .
  168. .El
  169. .Bl -tag -offset indent -width Va
  170. .It Va hz
  171. tick rate of the kernel in ticks per second.
  172. .El
  173. .Bl -tag -offset indent -width Va
  174. .It Va tcp_rtt_scale
  175. smoothed RTT estimate scaling factor.
  176. .El
  177. .Bl -tag -offset indent -width Va
  178. .It Va sysname
  179. operating system name.
  180. .El
  181. .Bl -tag -offset indent -width Va
  182. .It Va sysver
  183. operating system version.
  184. .El
  185. .Bl -tag -offset indent -width Va
  186. .It Va ipmode
  187. IP mode as defined at compile time.
  188. An ipmode of "4" means IPv6 is not supported and IP addresses are logged in
  189. regular dotted quad format.
  190. An ipmode of "6" means IPv6 is supported, and IP addresses are logged in dotted
  191. quad or hex format, as described in the
  192. .Qq Compile-time Configuration
  193. subsection.
  194. .El
  195. .Pp
  196. The second type of log message is written to the file when a data log message
  197. is generated.
  198. The text below shows an example data log triggered by an IPv4
  199. TCP/IP packet.
  200. The data is CSV formatted.
  201. .Bd -literal -offset indent
  202. o,0xbec491a5,1238556193.463551,172.16.7.28,22,172.16.2.5,55931, \\
  203. 1073725440,172312,6144,66560,66608,8,1,4,1448,936,1,996,255, \\
  204. 33304,208,66608,0,208,0
  205. .Ed
  206. .Pp
  207. Field descriptions are as follows:
  208. .Bl -tag -offset indent -width Va
  209. .It Va 1
  210. Direction of packet that triggered the log message.
  211. Either
  212. .Qq i
  213. for in, or
  214. .Qq o
  215. for out.
  216. .El
  217. .Bl -tag -offset indent -width Va
  218. .It Va 2
  219. Hash of the packet that triggered the log message.
  220. .El
  221. .Bl -tag -offset indent -width Va
  222. .It Va 3
  223. Time at which the packet that triggered the log message was processed by
  224. the
  225. .Xr pfil 9
  226. hook function, in seconds and microseconds since the UNIX epoch.
  227. .El
  228. .Bl -tag -offset indent -width Va
  229. .It Va 4
  230. The IPv4 or IPv6 address of the local host, in dotted quad (IPv4 packet)
  231. or colon-separated hex (IPv6 packet) notation.
  232. .El
  233. .Bl -tag -offset indent -width Va
  234. .It Va 5
  235. The TCP port that the local host is communicating via.
  236. .El
  237. .Bl -tag -offset indent -width Va
  238. .It Va 6
  239. The IPv4 or IPv6 address of the foreign host, in dotted quad (IPv4 packet)
  240. or colon-separated hex (IPv6 packet) notation.
  241. .El
  242. .Bl -tag -offset indent -width Va
  243. .It Va 7
  244. The TCP port that the foreign host is communicating via.
  245. .El
  246. .Bl -tag -offset indent -width Va
  247. .It Va 8
  248. The slow start threshold for the flow, in bytes.
  249. .El
  250. .Bl -tag -offset indent -width Va
  251. .It Va 9
  252. The current congestion window for the flow, in bytes.
  253. .El
  254. .Bl -tag -offset indent -width Va
  255. .It Va 10
  256. The current bandwidth-controlled window for the flow, in bytes.
  257. .El
  258. .Bl -tag -offset indent -width Va
  259. .It Va 11
  260. The current sending window for the flow, in bytes.
  261. The post scaled value is reported, except during the initial handshake (first
  262. few packets), during which time the unscaled value is reported.
  263. .El
  264. .Bl -tag -offset indent -width Va
  265. .It Va 12
  266. The current receive window for the flow, in bytes.
  267. The post scaled value is always reported.
  268. .El
  269. .Bl -tag -offset indent -width Va
  270. .It Va 13
  271. The current window scaling factor for the sending window.
  272. .El
  273. .Bl -tag -offset indent -width Va
  274. .It Va 14
  275. The current window scaling factor for the receiving window.
  276. .El
  277. .Bl -tag -offset indent -width Va
  278. .It Va 15
  279. The current state of the TCP finite state machine, as defined
  280. in
  281. .Aq Pa netinet/tcp_fsm.h .
  282. .El
  283. .Bl -tag -offset indent -width Va
  284. .It Va 16
  285. The maximum segment size for the flow, in bytes.
  286. .El
  287. .Bl -tag -offset indent -width Va
  288. .It Va 17
  289. The current smoothed RTT estimate for the flow, in units of TCP_RTT_SCALE * HZ,
  290. where TCP_RTT_SCALE is a define found in tcp_var.h, and HZ is the kernel's tick
  291. timer.
  292. Divide by TCP_RTT_SCALE * HZ to get the RTT in secs. TCP_RTT_SCALE and HZ are
  293. reported in the enable log message.
  294. .El
  295. .Bl -tag -offset indent -width Va
  296. .It Va 18
  297. SACK enabled indicator. 1 if SACK enabled, 0 otherwise.
  298. .El
  299. .Bl -tag -offset indent -width Va
  300. .It Va 19
  301. The current state of the TCP flags for the flow.
  302. See
  303. .Aq Pa netinet/tcp_var.h
  304. for information about the various flags.
  305. .El
  306. .Bl -tag -offset indent -width Va
  307. .It Va 20
  308. The current retransmission timeout length for the flow, in units of HZ, where HZ
  309. is the kernel's tick timer.
  310. Divide by HZ to get the timeout length in seconds. HZ is reported in the
  311. enable log message.
  312. .El
  313. .Bl -tag -offset indent -width Va
  314. .It Va 21
  315. The current size of the socket send buffer in bytes.
  316. .El
  317. .Bl -tag -offset indent -width Va
  318. .It Va 22
  319. The current number of bytes in the socket send buffer.
  320. .El
  321. .Bl -tag -offset indent -width Va
  322. .It Va 23
  323. The current size of the socket receive buffer in bytes.
  324. .El
  325. .Bl -tag -offset indent -width Va
  326. .It Va 24
  327. The current number of bytes in the socket receive buffer.
  328. .El
  329. .Bl -tag -offset indent -width Va
  330. .It Va 25
  331. The current number of unacknowledged bytes in-flight.
  332. Bytes acknowledged via SACK are not excluded from this count.
  333. .El
  334. .Bl -tag -offset indent -width Va
  335. .It Va 26
  336. The current number of segments in the reassembly queue.
  337. .El
  338. .Pp
  339. The third type of log message is written to the file when the module is disabled
  340. and ceases collecting data from the running kernel.
  341. The text below shows an example module disable log.
  342. The fields are tab delimited key-value pairs which provide statistics about
  343. operations since the module was most recently enabled.
  344. .Bd -literal -offset indent
  345. disable_time_secs=1238556197 disable_time_usecs=933607 \\
  346. num_inbound_tcp_pkts=356 num_outbound_tcp_pkts=627 \\
  347. total_tcp_pkts=983 num_inbound_skipped_pkts_malloc=0 \\
  348. num_outbound_skipped_pkts_malloc=0 num_inbound_skipped_pkts_mtx=0 \\
  349. num_outbound_skipped_pkts_mtx=0 num_inbound_skipped_pkts_tcb=0 \\
  350. num_outbound_skipped_pkts_tcb=0 num_inbound_skipped_pkts_icb=0 \\
  351. num_outbound_skipped_pkts_icb=0 total_skipped_tcp_pkts=0 \\
  352. flow_list=172.16.7.28;22-172.16.2.5;55931,
  353. .Ed
  354. .Pp
  355. Field descriptions are as follows:
  356. .Bl -tag -offset indent -width Va
  357. .It Va disable_time_secs
  358. Time at which the module was disabled, in seconds since the UNIX epoch.
  359. .El
  360. .Bl -tag -offset indent -width Va
  361. .It Va disable_time_usecs
  362. Time at which the module was disabled, in microseconds since disable_time_secs.
  363. .El
  364. .Bl -tag -offset indent -width Va
  365. .It Va num_inbound_tcp_pkts
  366. Number of TCP packets that traversed up the network stack.
  367. This only includes inbound TCP packets during the periods when
  368. .Nm
  369. was enabled.
  370. .El
  371. .Bl -tag -offset indent -width Va
  372. .It Va num_outbound_tcp_pkts
  373. Number of TCP packets that traversed down the network stack.
  374. This only includes outbound TCP packets during the periods when
  375. .Nm
  376. was enabled.
  377. .El
  378. .Bl -tag -offset indent -width Va
  379. .It Va total_tcp_pkts
  380. The summation of num_inbound_tcp_pkts and num_outbound_tcp_pkts.
  381. .El
  382. .Bl -tag -offset indent -width Va
  383. .It Va num_inbound_skipped_pkts_malloc
  384. Number of inbound packets that were not processed because of failed malloc() calls.
  385. .El
  386. .Bl -tag -offset indent -width Va
  387. .It Va num_outbound_skipped_pkts_malloc
  388. Number of outbound packets that were not processed because of failed malloc() calls.
  389. .El
  390. .Bl -tag -offset indent -width Va
  391. .It Va num_inbound_skipped_pkts_mtx
  392. Number of inbound packets that were not processed because of failure to add the
  393. packet to the packet processing queue.
  394. .El
  395. .Bl -tag -offset indent -width Va
  396. .It Va num_outbound_skipped_pkts_mtx
  397. Number of outbound packets that were not processed because of failure to add the
  398. packet to the packet processing queue.
  399. .El
  400. .Bl -tag -offset indent -width Va
  401. .It Va num_inbound_skipped_pkts_tcb
  402. Number of inbound packets that were not processed because of failure to find the
  403. TCP control block associated with the packet.
  404. .El
  405. .Bl -tag -offset indent -width Va
  406. .It Va num_outbound_skipped_pkts_tcb
  407. Number of outbound packets that were not processed because of failure to find
  408. the TCP control block associated with the packet.
  409. .El
  410. .Bl -tag -offset indent -width Va
  411. .It Va num_inbound_skipped_pkts_icb
  412. Number of inbound packets that were not processed because of failure to find the
  413. IP control block associated with the packet.
  414. .El
  415. .Bl -tag -offset indent -width Va
  416. .It Va num_outbound_skipped_pkts_icb
  417. Number of outbound packets that were not processed because of failure to find
  418. the IP control block associated with the packet.
  419. .El
  420. .Bl -tag -offset indent -width Va
  421. .It Va total_skipped_tcp_pkts
  422. The summation of all skipped packet counters.
  423. .El
  424. .Bl -tag -offset indent -width Va
  425. .It Va flow_list
  426. A CSV list of TCP flows that triggered data log messages to be generated since
  427. the module was loaded.
  428. Each flow entry in the CSV list is
  429. formatted as
  430. .Qq local_ip;local_port-foreign_ip;foreign_port .
  431. If there are no entries in the list (i.e., no data log messages were generated),
  432. the value will be blank.
  433. If there is at least one entry in the list, a trailing comma will always be
  434. present.
  435. .El
  436. .Pp
  437. The total number of data log messages found in the log file for a module
  438. enable/disable cycle should equate to total_tcp_pkts - total_skipped_tcp_pkts.
  439. .Sh IMPLEMENTATION NOTES
  440. .Nm
  441. hooks into the network stack using the
  442. .Xr pfil 9
  443. interface.
  444. In its current incarnation, it hooks into the AF_INET/AF_INET6 (IPv4/IPv6)
  445. .Xr pfil 9
  446. filtering points, which means it sees packets at the IP layer of the network
  447. stack.
  448. This means that TCP packets inbound to the stack are intercepted before
  449. they have been processed by the TCP layer.
  450. Packets outbound from the stack are intercepted after they have been processed
  451. by the TCP layer.
  452. .Pp
  453. The diagram below illustrates how
  454. .Nm
  455. inserts itself into the stack.
  456. .Bd -literal -offset indent
  457. ----------------------------------
  458. Upper Layers
  459. ----------------------------------
  460. ^ |
  461. | |
  462. | |
  463. | v
  464. TCP in TCP out
  465. ----------------------------------
  466. ^ |
  467. |________ _________|
  468. | |
  469. | v
  470. ---------
  471. | SIFTR |
  472. ---------
  473. ^ |
  474. ________| |__________
  475. | |
  476. | v
  477. IPv{4/6} in IPv{4/6} out
  478. ----------------------------------
  479. ^ |
  480. | |
  481. | v
  482. Layer 2 in Layer 2 out
  483. ----------------------------------
  484. Physical Layer
  485. ----------------------------------
  486. .Ed
  487. .Pp
  488. .Nm
  489. uses the
  490. .Xr alq 9
  491. interface to manage writing data to disk.
  492. .Pp
  493. At first glance, you might mistakenly think that
  494. .Nm
  495. extracts information from
  496. individual TCP packets.
  497. This is not the case.
  498. .Nm
  499. uses TCP packet events (inbound and outbound) for each TCP flow originating from
  500. the system to trigger a dump of the state of the TCP control block for that
  501. flow.
  502. With the PPL set to 1, we are in effect sampling each TCP flow's control block
  503. state as frequently as flow packets enter/leave the system.
  504. For example, setting PPL to 2 halves the sampling rate i.e., every second flow
  505. packet (inbound OR outbound) causes a dump of the control block state.
  506. .Pp
  507. The distinction between interrogating individual packets versus interrogating the
  508. control block is important, because
  509. .Nm
  510. does not remove the need for packet capturing tools like
  511. .Xr tcpdump 1 .
  512. .Nm
  513. allows you to correlate and observe the cause-and-affect relationship between
  514. what you see on the wire (captured using a tool like
  515. .Xr tcpdump 1 Ns )
  516. and changes in the TCP control block corresponding to the flow of interest.
  517. It is therefore useful to use
  518. .Nm
  519. and a tool like
  520. .Xr tcpdump 1
  521. to gather the necessary data to piece together the complete picture.
  522. Use of either tool on its own will not be able to provide all of the necessary
  523. data.
  524. .Pp
  525. As a result of needing to interrogate the TCP control block, certain packets
  526. during the lifecycle of a connection are unable to trigger a
  527. .Nm
  528. log message.
  529. The initial handshake takes place without the existence of a control block and
  530. the final ACK is exchanged when the connection is in the TIMEWAIT state.
  531. .Pp
  532. .Nm
  533. was designed to minimise the delay introduced to packets traversing the network
  534. stack.
  535. This design called for a highly optimised and minimal hook function that
  536. extracted the minimal details necessary whilst holding the packet up, and
  537. passing these details to another thread for actual processing and logging.
  538. .Pp
  539. This multithreaded design does introduce some contention issues when accessing
  540. the data structure shared between the threads of operation.
  541. When the hook function tries to place details in the structure, it must first
  542. acquire an exclusive lock.
  543. Likewise, when the processing thread tries to read details from the structure,
  544. it must also acquire an exclusive lock to do so.
  545. If one thread holds the lock, the other must wait before it can obtain it.
  546. This does introduce some additional bounded delay into the kernel's packet
  547. processing code path.
  548. .Pp
  549. In some cases (e.g., low memory, connection termination), TCP packets that enter
  550. the
  551. .Nm
  552. .Xr pfil 9
  553. hook function will not trigger a log message to be generated.
  554. .Nm
  555. refers to this outcome as a
  556. .Qq skipped packet .
  557. Note that
  558. .Nm
  559. always ensures that packets are allowed to continue through the stack, even if
  560. they could not successfully trigger a data log message.
  561. .Nm
  562. will therefore not introduce any packet loss for TCP/IP packets traversing the
  563. network stack.
  564. .Ss Important Behaviours
  565. The behaviour of a log file path change whilst the module is enabled is as
  566. follows:
  567. .Bl -enum
  568. .It
  569. Attempt to open the new file path for writing.
  570. If this fails, the path change will fail and the existing path will continue to
  571. be used.
  572. .It
  573. Assuming the new path is valid and opened successfully:
  574. .Bl -dash
  575. .It
  576. Flush all pending log messages to the old file path.
  577. .It
  578. Close the old file path.
  579. .It
  580. Switch the active log file pointer to point at the new file path.
  581. .It
  582. Commence logging to the new file.
  583. .El
  584. .El
  585. .Pp
  586. During the time between the flush of pending log messages to the old file and
  587. commencing logging to the new file, new log messages will still be generated and
  588. buffered.
  589. As soon as the new file path is ready for writing, the accumulated log messages
  590. will be written out to the file.
  591. .Sh EXAMPLES
  592. To enable the module's operations, run the following command as root:
  593. sysctl net.inet.siftr.enabled=1
  594. .Pp
  595. To change the granularity of log messages such that 1 log message is
  596. generated for every 10 TCP packets per connection, run the following
  597. command as root:
  598. sysctl net.inet.siftr.ppl=10
  599. .Pp
  600. To change the log file location to /tmp/siftr.log, run the following
  601. command as root:
  602. sysctl net.inet.siftr.logfile=/tmp/siftr.log
  603. .Sh SEE ALSO
  604. .Xr tcpdump 1 ,
  605. .Xr tcp 4 ,
  606. .Xr sysctl 8 ,
  607. .Xr alq 9 ,
  608. .Xr pfil 9
  609. .Sh ACKNOWLEDGEMENTS
  610. Development of this software was made possible in part by grants from the
  611. Cisco University Research Program Fund at Community Foundation Silicon Valley,
  612. and the FreeBSD Foundation.
  613. .Sh HISTORY
  614. .Nm
  615. first appeared in
  616. .Fx 7.4
  617. and
  618. .Fx 8.2 .
  619. .Pp
  620. .Nm
  621. was first released in 2007 by Lawrence Stewart and James Healy whilst working on
  622. the NewTCP research project at Swinburne University of Technology's Centre for
  623. Advanced Internet Architectures, Melbourne, Australia, which was made possible
  624. in part by a grant from the Cisco University Research Program Fund at Community
  625. Foundation Silicon Valley.
  626. More details are available at:
  627. .Pp
  628. http://caia.swin.edu.au/urp/newtcp/
  629. .Pp
  630. Work on
  631. .Nm
  632. v1.2.x was sponsored by the FreeBSD Foundation as part of
  633. the
  634. .Qq Enhancing the FreeBSD TCP Implementation
  635. project 2008-2009.
  636. More details are available at:
  637. .Pp
  638. http://www.freebsdfoundation.org/
  639. .Pp
  640. http://caia.swin.edu.au/freebsd/etcp09/
  641. .Sh AUTHORS
  642. .An -nosplit
  643. .Nm
  644. was written by
  645. .An Lawrence Stewart Aq lstewart@FreeBSD.org
  646. and
  647. .An James Healy Aq jimmy@deefa.com .
  648. .Pp
  649. This manual page was written by
  650. .An Lawrence Stewart Aq lstewart@FreeBSD.org .
  651. .Sh BUGS
  652. Current known limitations and any relevant workarounds are outlined below:
  653. .Bl -dash
  654. .It
  655. The internal queue used to pass information between the threads of operation is
  656. currently unbounded.
  657. This allows
  658. .Nm
  659. to cope with bursty network traffic, but sustained high packet-per-second
  660. traffic can cause exhaustion of kernel memory if the processing thread cannot
  661. keep up with the packet rate.
  662. .It
  663. If using
  664. .Nm
  665. on a machine that is also running other modules utilising the
  666. .Xr pfil 9
  667. framework e.g.
  668. .Xr dummynet 4 ,
  669. .Xr ipfw 8 ,
  670. .Xr pf 4 Ns ,
  671. the order in which you load the modules is important.
  672. You should kldload the other modules first, as this will ensure TCP packets
  673. undergo any necessary manipulations before
  674. .Nm
  675. .Qq sees
  676. and processes them.
  677. .It
  678. There is a known, harmless lock order reversal warning between the
  679. .Xr pfil 9
  680. mutex and tcbinfo TCP lock reported by
  681. .Xr witness 4
  682. when
  683. .Nm
  684. is enabled in a kernel compiled with
  685. .Xr witness 4
  686. support.
  687. .It
  688. There is no way to filter which TCP flows you wish to capture data for.
  689. Post processing is required to separate out data belonging to particular flows
  690. of interest.
  691. .It
  692. The module does not detect deletion of the log file path.
  693. New log messages will simply be lost if the log file being used by
  694. .Nm
  695. is deleted whilst the module is set to use the file.
  696. Switching to a new log file using the
  697. .Em net.inet.siftr.logfile
  698. variable will create the new file and allow log messages to begin being written
  699. to disk again.
  700. The new log file path must differ from the path to the deleted file.
  701. .It
  702. The hash table used within the code is sized to hold 65536 flows. This is not a
  703. hard limit, because chaining is used to handle collisions within the hash table
  704. structure.
  705. However, we suspect (based on analogies with other hash table performance data)
  706. that the hash table look up performance (and therefore the module's packet
  707. processing performance) will degrade in an exponential manner as the number of
  708. unique flows handled in a module enable/disable cycle approaches and surpasses
  709. 65536.
  710. .It
  711. There is no garbage collection performed on the flow hash table.
  712. The only way currently to flush it is to disable
  713. .Nm .
  714. .It
  715. The PPL variable applies to packets that make it into the processing thread,
  716. not total packets received in the hook function.
  717. Packets are skipped before the PPL variable is applied, which means there may be
  718. a slight discrepancy in the triggering of log messages.
  719. For example, if PPL was set to 10, and the 8th packet since the last log message
  720. is skipped, the 11th packet will actually trigger the log message to be
  721. generated.
  722. This is discussed in greater depth in CAIA technical report 070824A.
  723. .It
  724. At the time of writing, there was no simple way to hook into the TCP layer
  725. to intercept packets.
  726. .Nm Ap s
  727. use of IP layer hook points means all IP
  728. traffic will be processed by the
  729. .Nm
  730. .Xr pfil 9
  731. hook function, which introduces minor, but nonetheless unnecessary packet delay
  732. and processing overhead on the system for non-TCP packets as well.
  733. Hooking in at the IP layer is also not ideal from the data gathering point of
  734. view.
  735. Packets traversing up the stack will be intercepted and cause a log message
  736. generation BEFORE they have been processed by the TCP layer, which means we
  737. cannot observe the cause-and-affect relationship between inbound events and the
  738. corresponding TCP control block as precisely as could be.
  739. Ideally,
  740. .Nm
  741. should intercept packets after they have been processed by the TCP layer i.e.
  742. intercept packets coming up the stack after they have been processed by
  743. tcp_input(), and intercept packets coming down the stack after they have been
  744. processed by tcp_output().
  745. The current code still gives satisfactory granularity though, as inbound events
  746. tend to trigger outbound events, allowing the cause-and-effect to be observed
  747. indirectly by capturing the state on outbound events as well.
  748. .It
  749. The
  750. .Qq inflight bytes
  751. value logged by
  752. .Nm
  753. does not take into account bytes that have been
  754. .No SACK Ap ed
  755. by the receiving host.
  756. .It
  757. Packet hash generation does not currently work for IPv6 based TCP packets.
  758. .It
  759. Compressed notation is not used for IPv6 address representation.
  760. This consumes more bytes than is necessary in log output.
  761. .El