/tests/sympolymathesy/content/journal/2020/Follow-Up on Command-Line Finding and Filtering.md

https://github.com/chriskrycho/lightning-rs · Markdown · 46 lines · 36 code · 10 blank · 0 comment · 0 complexity · e34f2d0440e49f74e42e8856eac83ddd MD5 · raw file 

    

  1. ---
    2. 

  2. title: Follow-Up on Command-Line Finding and Filtering
    3. 

  3. subtitle: >
    4. 

  4.     A simpler solution that doesn’t require <code>tr</code>… if you have GNU utils or other alternatives.
    5. 

  5. date: 2020-05-04T09:15:00-0600
    6. 

  6. updated: 2020-07-26T12:05:00-0600
    7. 

  7. summary: >
    8. 

  8.     You can use a variant flag with GNU grep and ripgrep to filter with null characters.
    9. 

  9. tags:
    10. 

  10.     - things I learned
    11. 

  11.     - command line
    12. 

  12.     - software development
    13. 

  13. qualifiers:
    14. 

  14.     audience: >
    15. 

  15.         90% myself in the future, when I (inevitably) ask this question again—but also anyone else who hits this particular question about command-line invocations.
    16. 

  16.     epistemic: >
    17. 

  17.         Slightly higher than [the *previous* post on the subject](https://v5.chriskrycho.com/journal/find-grep-xargs-newlines-null/), courtesy of the requested reader feedback!
    18. 

  18. ---
    19. 

  19. 

  20. In my [previous post](https://v5.chriskrycho.com/journal/find-grep-xargs-newlines-null/), I used the `tr` utility to deal with needing to transform newlines into null characters. However, as I hoped when I put a request for a better way to do it in my <b>Epistemic Status</b> qualifier, a reader emailed me with a better solution!
    21. 

  21. 

  22. If you’re using the GNU version of `grep`, it has a `--null-data` (shortened as `-z`) flag which makes grep treat its input as null-character-separated. You can combine that with the `-print0` flag to `find` to get the same results as I got with `tr` (presumably with better performance because it doesn’t require doing the replacement in another tool):
    23. 

  23. 

  24. ```sh
    25. 

  25. $ find notes -name ".md" -print0 |\
    26. 

  26.   grep --null-data "notes/2020" |\
    27. 

  27.   xargs -0 wc -w
    28. 

  28. ```
    29. 

  29. 

  30. This reminded me that [ripgrep] has the same feature, with the same `--null-data` flag. Similarly, [fd] has a `--print0` (`-0`) option. You can combine *these* and (if you like) [cw][cw][^cw] to get the same effect:
    31. 

  31. 

  32. ```sh
    33. 

  33. $ fd notes --print0 ".md" notes |\
    34. 

  34.   rg --null-data 'notes/2020' |\
    35. 

  35.   xargs -0 cw -w
    36. 

  36. ```
    37. 

  37. 

  38. Huzzah for versions of tools that understand these things and make this simpler than the solution I posted yesterday (and thanks to my reader for sending in that note)!
    39. 

  39. 

  40. 

  41. 

  42. [^cw]: `cw` is nice because with especially large sets of data, the fact that you can invoke across threads becomes very handy. If I word-count *all* of my notes with it (currently 667 files and just shy of 150,000 words), using 4 threads instead of 1 (the default, and all you get with `wc`) takes about 6–8 milliseconds off the run time. Not important at *this* scale… but if you’re dealing with *very* large amounts of data, it might be.
    43. 

  43. 

  44. [ripgrep]: https://github.com/BurntSushi/ripgrep
    45. 

  45. [fd]: https://github.com/sharkdp/fd
    46. 

  46. [cw]: https://github.com/Freaky/cw
    47.