PageRenderTime 35ms CodeModel.GetById 31ms app.highlight 2ms RepoModel.GetById 0ms app.codeStats 0ms

Unknown | 454 lines | 407 code | 47 blank | 0 comment | 0 complexity | 838d3ab236f9a439040e794e9fa8be24 MD5 | raw file
Possible License(s): MPL-2.0-no-copyleft-exception, BSD-3-Clause, LGPL-2.0, LGPL-2.1, BSD-2-Clause, 0BSD, JSON, AGPL-1.0, GPL-2.0
  2.TH bzip2 1
  4bzip2, bunzip2 \- a block-sorting file compressor, v1.0.6
  6bzcat \- decompresses files to stdout
  8bzip2recover \- recovers data from damaged bzip2 files
 11.ll +8
 12.B bzip2
 13.RB [ " \-cdfkqstvzVL123456789 " ]
 15.I "filenames \&..."
 17.ll -8
 19.B bunzip2
 20.RB [ " \-fkvsVL " ]
 22.I "filenames \&..."
 25.B bzcat
 26.RB [ " \-s " ]
 28.I "filenames \&..."
 31.B bzip2recover
 32.I "filename"
 35.I bzip2
 36compresses files using the Burrows-Wheeler block sorting
 37text compression algorithm, and Huffman coding.  Compression is
 38generally considerably better than that achieved by more conventional
 39LZ77/LZ78-based compressors, and approaches the performance of the PPM
 40family of statistical compressors.
 42The command-line options are deliberately very similar to 
 43those of 
 44.I GNU gzip, 
 45but they are not identical.
 47.I bzip2
 48expects a list of file names to accompany the
 49command-line flags.  Each file is replaced by a compressed version of
 50itself, with the name "original_name.bz2".  
 51Each compressed file
 52has the same modification date, permissions, and, when possible,
 53ownership as the corresponding original, so that these properties can
 54be correctly restored at decompression time.  File name handling is
 55naive in the sense that there is no mechanism for preserving original
 56file names, permissions, ownerships or dates in filesystems which lack
 57these concepts, or have serious file name length restrictions, such as
 60.I bzip2
 62.I bunzip2
 63will by default not overwrite existing
 64files.  If you want this to happen, specify the \-f flag.
 66If no file names are specified,
 67.I bzip2
 68compresses from standard
 69input to standard output.  In this case,
 70.I bzip2
 71will decline to
 72write compressed output to a terminal, as this would be entirely
 73incomprehensible and therefore pointless.
 75.I bunzip2
 77.I bzip2 \-d) 
 78decompresses all
 79specified files.  Files which were not created by 
 80.I bzip2
 81will be detected and ignored, and a warning issued.  
 82.I bzip2
 83attempts to guess the filename for the decompressed file 
 84from that of the compressed file as follows:
 86       filename.bz2    becomes   filename
 87     becomes   filename
 88       filename.tbz2   becomes   filename.tar
 89       filename.tbz    becomes   filename.tar
 90       anyothername    becomes   anyothername.out
 92If the file does not end in one of the recognised endings, 
 93.I .bz2, 
 94.I .bz, 
 95.I .tbz2
 97.I .tbz, 
 98.I bzip2 
 99complains that it cannot
100guess the name of the original file, and uses the original name
102.I .out
105As with compression, supplying no
106filenames causes decompression from 
107standard input to standard output.
109.I bunzip2 
110will correctly decompress a file which is the
111concatenation of two or more compressed files.  The result is the
112concatenation of the corresponding uncompressed files.  Integrity
113testing (\-t) 
114of concatenated 
115compressed files is also supported.
117You can also compress or decompress files to the standard output by
118giving the \-c flag.  Multiple files may be compressed and
119decompressed like this.  The resulting outputs are fed sequentially to
120stdout.  Compression of multiple files 
121in this manner generates a stream
122containing multiple compressed file representations.  Such a stream
123can be decompressed correctly only by
124.I bzip2 
125version 0.9.0 or
126later.  Earlier versions of
127.I bzip2
128will stop after decompressing
129the first file in the stream.
131.I bzcat
133.I bzip2 -dc) 
134decompresses all specified files to
135the standard output.
137.I bzip2
138will read arguments from the environment variables
139.I BZIP2
141.I BZIP,
142in that order, and will process them
143before any arguments read from the command line.  This gives a 
144convenient way to supply default arguments.
146Compression is always performed, even if the compressed 
147file is slightly
148larger than the original.  Files of less than about one hundred bytes
149tend to get larger, since the compression mechanism has a constant
150overhead in the region of 50 bytes.  Random data (including the output
151of most file compressors) is coded at about 8.05 bits per byte, giving
152an expansion of around 0.5%.
154As a self-check for your protection, 
157uses 32-bit CRCs to
158make sure that the decompressed version of a file is identical to the
159original.  This guards against corruption of the compressed data, and
160against undetected bugs in
161.I bzip2
162(hopefully very unlikely).  The
163chances of data corruption going undetected is microscopic, about one
164chance in four billion for each file processed.  Be aware, though, that
165the check occurs upon decompression, so it can only tell you that
166something is wrong.  It can't help you 
167recover the original uncompressed
168data.  You can use 
169.I bzip2recover
170to try to recover data from
171damaged files.
173Return values: 0 for a normal exit, 1 for environmental problems (file
174not found, invalid flags, I/O errors, &c), 2 to indicate a corrupt
175compressed file, 3 for an internal consistency error (eg, bug) which
177.I bzip2
178to panic.
182.B \-c --stdout
183Compress or decompress to standard output.
185.B \-d --decompress
186Force decompression.  
187.I bzip2, 
188.I bunzip2 
190.I bzcat 
192really the same program, and the decision about what actions to take is
193done on the basis of which name is used.  This flag overrides that
194mechanism, and forces 
195.I bzip2
196to decompress.
198.B \-z --compress
199The complement to \-d: forces compression, regardless of the
200invocation name.
202.B \-t --test
203Check integrity of the specified file(s), but don't decompress them.
204This really performs a trial decompression and throws away the result.
206.B \-f --force
207Force overwrite of output files.  Normally,
208.I bzip2 
209will not overwrite
210existing output files.  Also forces 
211.I bzip2 
212to break hard links
213to files, which it otherwise wouldn't do.
215bzip2 normally declines to decompress files which don't have the
216correct magic header bytes.  If forced (-f), however, it will pass
217such files through unmodified.  This is how GNU gzip behaves.
219.B \-k --keep
220Keep (don't delete) input files during compression
221or decompression.
223.B \-s --small
224Reduce memory usage, for compression, decompression and testing.  Files
225are decompressed and tested using a modified algorithm which only
226requires 2.5 bytes per block byte.  This means any file can be
227decompressed in 2300k of memory, albeit at about half the normal speed.
229During compression, \-s selects a block size of 200k, which limits
230memory use to around the same figure, at the expense of your compression
231ratio.  In short, if your machine is low on memory (8 megabytes or
232less), use \-s for everything.  See MEMORY MANAGEMENT below.
234.B \-q --quiet
235Suppress non-essential warning messages.  Messages pertaining to
236I/O errors and other critical events will not be suppressed.
238.B \-v --verbose
239Verbose mode -- show the compression ratio for each file processed.
240Further \-v's increase the verbosity level, spewing out lots of
241information which is primarily of interest for diagnostic purposes.
243.B \-L --license -V --version
244Display the software version, license terms and conditions.
246.B \-1 (or \-\-fast) to \-9 (or \-\-best)
247Set the block size to 100 k, 200 k ..  900 k when compressing.  Has no
248effect when decompressing.  See MEMORY MANAGEMENT below.
249The \-\-fast and \-\-best aliases are primarily for GNU gzip 
250compatibility.  In particular, \-\-fast doesn't make things
251significantly faster.  
252And \-\-best merely selects the default behaviour.
254.B \--
255Treats all subsequent arguments as file names, even if they start
256with a dash.  This is so you can handle files with names beginning
257with a dash, for example: bzip2 \-- \-myfilename.
259.B \--repetitive-fast --repetitive-best
260These flags are redundant in versions 0.9.5 and above.  They provided
261some coarse control over the behaviour of the sorting algorithm in
262earlier versions, which was sometimes useful.  0.9.5 and above have an
263improved algorithm which renders these flags irrelevant.
266.I bzip2 
267compresses large files in blocks.  The block size affects
268both the compression ratio achieved, and the amount of memory needed for
269compression and decompression.  The flags \-1 through \-9
270specify the block size to be 100,000 bytes through 900,000 bytes (the
271default) respectively.  At decompression time, the block size used for
272compression is read from the header of the compressed file, and
273.I bunzip2
274then allocates itself just enough memory to decompress
275the file.  Since block sizes are stored in compressed files, it follows
276that the flags \-1 to \-9 are irrelevant to and so ignored
277during decompression.
279Compression and decompression requirements, 
280in bytes, can be estimated as:
282       Compression:   400k + ( 8 x block size )
284       Decompression: 100k + ( 4 x block size ), or
285                      100k + ( 2.5 x block size )
287Larger block sizes give rapidly diminishing marginal returns.  Most of
288the compression comes from the first two or three hundred k of block
289size, a fact worth bearing in mind when using
290.I bzip2
291on small machines.
292It is also important to appreciate that the decompression memory
293requirement is set at compression time by the choice of block size.
295For files compressed with the default 900k block size,
296.I bunzip2
297will require about 3700 kbytes to decompress.  To support decompression
298of any file on a 4 megabyte machine, 
299.I bunzip2
300has an option to
301decompress using approximately half this amount of memory, about 2300
302kbytes.  Decompression speed is also halved, so you should use this
303option only where necessary.  The relevant flag is -s.
305In general, try and use the largest block size memory constraints allow,
306since that maximises the compression achieved.  Compression and
307decompression speed are virtually unaffected by block size.
309Another significant point applies to files which fit in a single block
310-- that means most files you'd encounter using a large block size.  The
311amount of real memory touched is proportional to the size of the file,
312since the file is smaller than a block.  For example, compressing a file
31320,000 bytes long with the flag -9 will cause the compressor to
314allocate around 7600k of memory, but only touch 400k + 20000 * 8 = 560
315kbytes of it.  Similarly, the decompressor will allocate 3700k but only
316touch 100k + 20000 * 4 = 180 kbytes.
318Here is a table which summarises the maximum memory usage for different
319block sizes.  Also recorded is the total compressed size for 14 files of
320the Calgary Text Compression Corpus totalling 3,141,622 bytes.  This
321column gives some feel for how compression varies with block size.
322These figures tend to understate the advantage of larger block sizes for
323larger files, since the Corpus is dominated by smaller files.
325           Compress   Decompress   Decompress   Corpus
326    Flag     usage      usage       -s usage     Size
328     -1      1200k       500k         350k      914704
329     -2      2000k       900k         600k      877703
330     -3      2800k      1300k         850k      860338
331     -4      3600k      1700k        1100k      846899
332     -5      4400k      2100k        1350k      845160
333     -6      5200k      2500k        1600k      838626
334     -7      6100k      2900k        1850k      834096
335     -8      6800k      3300k        2100k      828642
336     -9      7600k      3700k        2350k      828642
339.I bzip2
340compresses files in blocks, usually 900kbytes long.  Each
341block is handled independently.  If a media or transmission error causes
342a multi-block .bz2
343file to become damaged, it may be possible to
344recover data from the undamaged blocks in the file.
346The compressed representation of each block is delimited by a 48-bit
347pattern, which makes it possible to find the block boundaries with
348reasonable certainty.  Each block also carries its own 32-bit CRC, so
349damaged blocks can be distinguished from undamaged ones.
351.I bzip2recover
352is a simple program whose purpose is to search for
353blocks in .bz2 files, and write each block out into its own .bz2 
354file.  You can then use
355.I bzip2 
357to test the
358integrity of the resulting files, and decompress those which are
361.I bzip2recover
362takes a single argument, the name of the damaged file, 
363and writes a number of files "rec00001file.bz2",
364"rec00002file.bz2", etc, containing the  extracted  blocks.
365The  output  filenames  are  designed  so  that the use of
366wildcards in subsequent processing -- for example,  
367"bzip2 -dc  rec*file.bz2 > recovered_data" -- processes the files in
368the correct order.
370.I bzip2recover
371should be of most use dealing with large .bz2
372files,  as  these will contain many blocks.  It is clearly
373futile to use it on damaged single-block  files,  since  a
374damaged  block  cannot  be recovered.  If you wish to minimise 
375any potential data loss through media  or  transmission errors, 
376you might consider compressing with a smaller
377block size.
380The sorting phase of compression gathers together similar strings in the
381file.  Because of this, files containing very long runs of repeated
382symbols, like "aabaabaabaab ..."  (repeated several hundred times) may
383compress more slowly than normal.  Versions 0.9.5 and above fare much
384better than previous versions in this respect.  The ratio between
385worst-case and average-case compression time is in the region of 10:1.
386For previous versions, this figure was more like 100:1.  You can use the
387\-vvvv option to monitor progress in great detail, if you want.
389Decompression speed is unaffected by these phenomena.
391.I bzip2
392usually allocates several megabytes of memory to operate
393in, and then charges all over it in a fairly random fashion.  This means
394that performance, both for compressing and decompressing, is largely
395determined by the speed at which your machine can service cache misses.
396Because of this, small changes to the code to reduce the miss rate have
397been observed to give disproportionately large performance improvements.
398I imagine 
399.I bzip2
400will perform best on machines with very large caches.
403I/O error messages are not as helpful as they could be.
404.I bzip2
405tries hard to detect I/O errors and exit cleanly, but the details of
406what the problem is sometimes seem rather misleading.
408This manual page pertains to version 1.0.6 of
409.I bzip2.  
410Compressed data created by this version is entirely forwards and
411backwards compatible with the previous public releases, versions
4120.1pl2, 0.9.0, 0.9.5, 1.0.0, 1.0.1, 1.0.2 and above, but with the following
413exception: 0.9.0 and above can correctly decompress multiple
414concatenated compressed files.  0.1pl2 cannot do this; it will stop
415after decompressing just the first file in the stream.
417.I bzip2recover
418versions prior to 1.0.2 used 32-bit integers to represent
419bit positions in compressed files, so they could not handle compressed
420files more than 512 megabytes long.  Versions 1.0.2 and above use
42164-bit ints on some platforms which support them (GNU supported
422targets, and Windows).  To establish whether or not bzip2recover was
423built with such a limitation, run it without arguments.  In any event
424you can build yourself an unlimited version if you can recompile it
425with MaybeUInt64 set to be an unsigned 64-bit integer.
430Julian Seward,
434The ideas embodied in
435.I bzip2
436are due to (at least) the following
437people: Michael Burrows and David Wheeler (for the block sorting
438transformation), David Wheeler (again, for the Huffman coder), Peter
439Fenwick (for the structured coding model in the original
440.I bzip,
441and many refinements), and Alistair Moffat, Radford Neal and Ian Witten
442(for the arithmetic coder in the original
443.I bzip).  
444I am much
445indebted for their help, support and advice.  See the manual in the
446source distribution for pointers to sources of documentation.  Christian
447von Roques encouraged me to look for faster sorting algorithms, so as to
448speed up compression.  Bela Lubkin encouraged me to improve the
449worst-case compression performance.  
450Donna Robinson XMLised the documentation.
451The bz* scripts are derived from those of GNU gzip.
452Many people sent patches, helped
453with portability problems, lent machines, gave advice and were generally