PageRenderTime 112ms CodeModel.GetById 88ms app.highlight 11ms RepoModel.GetById 1ms app.codeStats 1ms

/StormLib/stormlib/bzip2/manual.xml

http://ghostcb.googlecode.com/
XML | 2964 lines | 2402 code | 561 blank | 1 comment | 0 complexity | 7a003c59ef2f36322fcb36ab195f3f70 MD5 | raw file

Large files files are truncated, but you can click here to view the full file

   1<?xml version="1.0"?> <!-- -*- sgml -*- -->
   2<!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"
   3  "http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"[
   4
   5<!-- various strings, dates etc. common to all docs -->
   6<!ENTITY % common-ents SYSTEM "entities.xml"> %common-ents;
   7]>
   8
   9<book lang="en" id="userman" xreflabel="bzip2 Manual">
  10
  11 <bookinfo>
  12  <title>bzip2 and libbzip2, version 1.0.5</title>
  13  <subtitle>A program and library for data compression</subtitle>
  14  <copyright>
  15   <year>&bz-lifespan;</year>
  16   <holder>Julian Seward</holder>
  17  </copyright>
  18  <releaseinfo>Version &bz-version; of &bz-date;</releaseinfo>
  19
  20  <authorgroup>
  21   <author>
  22    <firstname>Julian</firstname>
  23    <surname>Seward</surname>
  24    <affiliation>
  25     <orgname>&bz-url;</orgname>
  26    </affiliation>
  27   </author>
  28  </authorgroup>
  29
  30  <legalnotice>
  31
  32  <para>This program, <computeroutput>bzip2</computeroutput>, the
  33  associated library <computeroutput>libbzip2</computeroutput>, and
  34  all documentation, are copyright &copy; &bz-lifespan; Julian Seward.
  35  All rights reserved.</para>
  36
  37  <para>Redistribution and use in source and binary forms, with
  38  or without modification, are permitted provided that the
  39  following conditions are met:</para>
  40
  41  <itemizedlist mark='bullet'>
  42
  43   <listitem><para>Redistributions of source code must retain the
  44   above copyright notice, this list of conditions and the
  45   following disclaimer.</para></listitem>
  46
  47   <listitem><para>The origin of this software must not be
  48   misrepresented; you must not claim that you wrote the original
  49   software.  If you use this software in a product, an
  50   acknowledgment in the product documentation would be
  51   appreciated but is not required.</para></listitem>
  52
  53   <listitem><para>Altered source versions must be plainly marked
  54   as such, and must not be misrepresented as being the original
  55   software.</para></listitem>
  56
  57   <listitem><para>The name of the author may not be used to
  58   endorse or promote products derived from this software without
  59   specific prior written permission.</para></listitem>
  60
  61  </itemizedlist>
  62
  63  <para>THIS SOFTWARE IS PROVIDED BY THE AUTHOR "AS IS" AND ANY
  64  EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO,
  65  THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A
  66  PARTICULAR PURPOSE ARE DISCLAIMED.  IN NO EVENT SHALL THE
  67  AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
  68  EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED
  69  TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
  70  DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
  71  ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT
  72  LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING
  73  IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF
  74  THE POSSIBILITY OF SUCH DAMAGE.</para>
  75
  76 <para>PATENTS: To the best of my knowledge,
  77 <computeroutput>bzip2</computeroutput> and
  78 <computeroutput>libbzip2</computeroutput> do not use any patented
  79 algorithms.  However, I do not have the resources to carry
  80 out a patent search.  Therefore I cannot give any guarantee of
  81 the above statement.
  82 </para>
  83
  84</legalnotice>
  85
  86</bookinfo>
  87
  88
  89
  90<chapter id="intro" xreflabel="Introduction">
  91<title>Introduction</title>
  92
  93<para><computeroutput>bzip2</computeroutput> compresses files
  94using the Burrows-Wheeler block-sorting text compression
  95algorithm, and Huffman coding.  Compression is generally
  96considerably better than that achieved by more conventional
  97LZ77/LZ78-based compressors, and approaches the performance of
  98the PPM family of statistical compressors.</para>
  99
 100<para><computeroutput>bzip2</computeroutput> is built on top of
 101<computeroutput>libbzip2</computeroutput>, a flexible library for
 102handling compressed data in the
 103<computeroutput>bzip2</computeroutput> format.  This manual
 104describes both how to use the program and how to work with the
 105library interface.  Most of the manual is devoted to this
 106library, not the program, which is good news if your interest is
 107only in the program.</para>
 108
 109<itemizedlist mark='bullet'>
 110
 111 <listitem><para><xref linkend="using"/> describes how to use
 112 <computeroutput>bzip2</computeroutput>; this is the only part
 113 you need to read if you just want to know how to operate the
 114 program.</para></listitem>
 115
 116 <listitem><para><xref linkend="libprog"/> describes the
 117 programming interfaces in detail, and</para></listitem>
 118
 119 <listitem><para><xref linkend="misc"/> records some
 120 miscellaneous notes which I thought ought to be recorded
 121 somewhere.</para></listitem>
 122
 123</itemizedlist>
 124
 125</chapter>
 126
 127
 128<chapter id="using" xreflabel="How to use bzip2">
 129<title>How to use bzip2</title>
 130
 131<para>This chapter contains a copy of the
 132<computeroutput>bzip2</computeroutput> man page, and nothing
 133else.</para>
 134
 135<sect1 id="name" xreflabel="NAME">
 136<title>NAME</title>
 137
 138<itemizedlist mark='bullet'>
 139
 140 <listitem><para><computeroutput>bzip2</computeroutput>,
 141  <computeroutput>bunzip2</computeroutput> - a block-sorting file
 142  compressor, v1.0.4</para></listitem>
 143
 144 <listitem><para><computeroutput>bzcat</computeroutput> -
 145   decompresses files to stdout</para></listitem>
 146
 147 <listitem><para><computeroutput>bzip2recover</computeroutput> -
 148   recovers data from damaged bzip2 files</para></listitem>
 149
 150</itemizedlist>
 151
 152</sect1>
 153
 154
 155<sect1 id="synopsis" xreflabel="SYNOPSIS">
 156<title>SYNOPSIS</title>
 157
 158<itemizedlist mark='bullet'>
 159
 160 <listitem><para><computeroutput>bzip2</computeroutput> [
 161  -cdfkqstvzVL123456789 ] [ filenames ...  ]</para></listitem>
 162
 163 <listitem><para><computeroutput>bunzip2</computeroutput> [
 164  -fkvsVL ] [ filenames ...  ]</para></listitem>
 165
 166 <listitem><para><computeroutput>bzcat</computeroutput> [ -s ] [
 167  filenames ...  ]</para></listitem>
 168
 169 <listitem><para><computeroutput>bzip2recover</computeroutput>
 170  filename</para></listitem>
 171
 172</itemizedlist>
 173
 174</sect1>
 175
 176
 177<sect1 id="description" xreflabel="DESCRIPTION">
 178<title>DESCRIPTION</title>
 179
 180<para><computeroutput>bzip2</computeroutput> compresses files
 181using the Burrows-Wheeler block sorting text compression
 182algorithm, and Huffman coding.  Compression is generally
 183considerably better than that achieved by more conventional
 184LZ77/LZ78-based compressors, and approaches the performance of
 185the PPM family of statistical compressors.</para>
 186
 187<para>The command-line options are deliberately very similar to
 188those of GNU <computeroutput>gzip</computeroutput>, but they are
 189not identical.</para>
 190
 191<para><computeroutput>bzip2</computeroutput> expects a list of
 192file names to accompany the command-line flags.  Each file is
 193replaced by a compressed version of itself, with the name
 194<computeroutput>original_name.bz2</computeroutput>.  Each
 195compressed file has the same modification date, permissions, and,
 196when possible, ownership as the corresponding original, so that
 197these properties can be correctly restored at decompression time.
 198File name handling is naive in the sense that there is no
 199mechanism for preserving original file names, permissions,
 200ownerships or dates in filesystems which lack these concepts, or
 201have serious file name length restrictions, such as
 202MS-DOS.</para>
 203
 204<para><computeroutput>bzip2</computeroutput> and
 205<computeroutput>bunzip2</computeroutput> will by default not
 206overwrite existing files.  If you want this to happen, specify
 207the <computeroutput>-f</computeroutput> flag.</para>
 208
 209<para>If no file names are specified,
 210<computeroutput>bzip2</computeroutput> compresses from standard
 211input to standard output.  In this case,
 212<computeroutput>bzip2</computeroutput> will decline to write
 213compressed output to a terminal, as this would be entirely
 214incomprehensible and therefore pointless.</para>
 215
 216<para><computeroutput>bunzip2</computeroutput> (or
 217<computeroutput>bzip2 -d</computeroutput>) decompresses all
 218specified files.  Files which were not created by
 219<computeroutput>bzip2</computeroutput> will be detected and
 220ignored, and a warning issued.
 221<computeroutput>bzip2</computeroutput> attempts to guess the
 222filename for the decompressed file from that of the compressed
 223file as follows:</para>
 224
 225<itemizedlist mark='bullet'>
 226
 227 <listitem><para><computeroutput>filename.bz2 </computeroutput>
 228  becomes
 229  <computeroutput>filename</computeroutput></para></listitem>
 230
 231 <listitem><para><computeroutput>filename.bz </computeroutput>
 232  becomes
 233  <computeroutput>filename</computeroutput></para></listitem>
 234
 235 <listitem><para><computeroutput>filename.tbz2</computeroutput>
 236  becomes
 237  <computeroutput>filename.tar</computeroutput></para></listitem>
 238
 239 <listitem><para><computeroutput>filename.tbz </computeroutput>
 240  becomes
 241  <computeroutput>filename.tar</computeroutput></para></listitem>
 242
 243 <listitem><para><computeroutput>anyothername </computeroutput>
 244  becomes
 245  <computeroutput>anyothername.out</computeroutput></para></listitem>
 246
 247</itemizedlist>
 248
 249<para>If the file does not end in one of the recognised endings,
 250<computeroutput>.bz2</computeroutput>,
 251<computeroutput>.bz</computeroutput>,
 252<computeroutput>.tbz2</computeroutput> or
 253<computeroutput>.tbz</computeroutput>,
 254<computeroutput>bzip2</computeroutput> complains that it cannot
 255guess the name of the original file, and uses the original name
 256with <computeroutput>.out</computeroutput> appended.</para>
 257
 258<para>As with compression, supplying no filenames causes
 259decompression from standard input to standard output.</para>
 260
 261<para><computeroutput>bunzip2</computeroutput> will correctly
 262decompress a file which is the concatenation of two or more
 263compressed files.  The result is the concatenation of the
 264corresponding uncompressed files.  Integrity testing
 265(<computeroutput>-t</computeroutput>) of concatenated compressed
 266files is also supported.</para>
 267
 268<para>You can also compress or decompress files to the standard
 269output by giving the <computeroutput>-c</computeroutput> flag.
 270Multiple files may be compressed and decompressed like this.  The
 271resulting outputs are fed sequentially to stdout.  Compression of
 272multiple files in this manner generates a stream containing
 273multiple compressed file representations.  Such a stream can be
 274decompressed correctly only by
 275<computeroutput>bzip2</computeroutput> version 0.9.0 or later.
 276Earlier versions of <computeroutput>bzip2</computeroutput> will
 277stop after decompressing the first file in the stream.</para>
 278
 279<para><computeroutput>bzcat</computeroutput> (or
 280<computeroutput>bzip2 -dc</computeroutput>) decompresses all
 281specified files to the standard output.</para>
 282
 283<para><computeroutput>bzip2</computeroutput> will read arguments
 284from the environment variables
 285<computeroutput>BZIP2</computeroutput> and
 286<computeroutput>BZIP</computeroutput>, in that order, and will
 287process them before any arguments read from the command line.
 288This gives a convenient way to supply default arguments.</para>
 289
 290<para>Compression is always performed, even if the compressed
 291file is slightly larger than the original.  Files of less than
 292about one hundred bytes tend to get larger, since the compression
 293mechanism has a constant overhead in the region of 50 bytes.
 294Random data (including the output of most file compressors) is
 295coded at about 8.05 bits per byte, giving an expansion of around
 2960.5%.</para>
 297
 298<para>As a self-check for your protection,
 299<computeroutput>bzip2</computeroutput> uses 32-bit CRCs to make
 300sure that the decompressed version of a file is identical to the
 301original.  This guards against corruption of the compressed data,
 302and against undetected bugs in
 303<computeroutput>bzip2</computeroutput> (hopefully very unlikely).
 304The chances of data corruption going undetected is microscopic,
 305about one chance in four billion for each file processed.  Be
 306aware, though, that the check occurs upon decompression, so it
 307can only tell you that something is wrong.  It can't help you
 308recover the original uncompressed data.  You can use
 309<computeroutput>bzip2recover</computeroutput> to try to recover
 310data from damaged files.</para>
 311
 312<para>Return values: 0 for a normal exit, 1 for environmental
 313problems (file not found, invalid flags, I/O errors, etc.), 2
 314to indicate a corrupt compressed file, 3 for an internal
 315consistency error (eg, bug) which caused
 316<computeroutput>bzip2</computeroutput> to panic.</para>
 317
 318</sect1>
 319
 320
 321<sect1 id="options" xreflabel="OPTIONS">
 322<title>OPTIONS</title>
 323
 324<variablelist>
 325
 326 <varlistentry>
 327 <term><computeroutput>-c --stdout</computeroutput></term>
 328 <listitem><para>Compress or decompress to standard
 329  output.</para></listitem>
 330 </varlistentry>
 331
 332 <varlistentry>
 333 <term><computeroutput>-d --decompress</computeroutput></term>
 334 <listitem><para>Force decompression.
 335  <computeroutput>bzip2</computeroutput>,
 336  <computeroutput>bunzip2</computeroutput> and
 337  <computeroutput>bzcat</computeroutput> are really the same
 338  program, and the decision about what actions to take is done on
 339  the basis of which name is used.  This flag overrides that
 340  mechanism, and forces bzip2 to decompress.</para></listitem>
 341 </varlistentry>
 342
 343 <varlistentry>
 344 <term><computeroutput>-z --compress</computeroutput></term>
 345 <listitem><para>The complement to
 346  <computeroutput>-d</computeroutput>: forces compression,
 347  regardless of the invokation name.</para></listitem>
 348 </varlistentry>
 349
 350 <varlistentry>
 351 <term><computeroutput>-t --test</computeroutput></term>
 352 <listitem><para>Check integrity of the specified file(s), but
 353  don't decompress them.  This really performs a trial
 354  decompression and throws away the result.</para></listitem>
 355 </varlistentry>
 356
 357 <varlistentry>
 358 <term><computeroutput>-f --force</computeroutput></term>
 359 <listitem><para>Force overwrite of output files.  Normally,
 360  <computeroutput>bzip2</computeroutput> will not overwrite
 361  existing output files.  Also forces
 362  <computeroutput>bzip2</computeroutput> to break hard links to
 363  files, which it otherwise wouldn't do.</para>
 364  <para><computeroutput>bzip2</computeroutput> normally declines
 365  to decompress files which don't have the correct magic header
 366  bytes. If forced (<computeroutput>-f</computeroutput>),
 367  however, it will pass such files through unmodified. This is
 368  how GNU <computeroutput>gzip</computeroutput> behaves.</para>
 369 </listitem>
 370 </varlistentry>
 371
 372 <varlistentry>
 373 <term><computeroutput>-k --keep</computeroutput></term>
 374 <listitem><para>Keep (don't delete) input files during
 375  compression or decompression.</para></listitem>
 376 </varlistentry>
 377
 378 <varlistentry>
 379 <term><computeroutput>-s --small</computeroutput></term>
 380 <listitem><para>Reduce memory usage, for compression,
 381  decompression and testing.  Files are decompressed and tested
 382  using a modified algorithm which only requires 2.5 bytes per
 383  block byte.  This means any file can be decompressed in 2300k
 384  of memory, albeit at about half the normal speed.</para>
 385  <para>During compression, <computeroutput>-s</computeroutput>
 386  selects a block size of 200k, which limits memory use to around
 387  the same figure, at the expense of your compression ratio.  In
 388  short, if your machine is low on memory (8 megabytes or less),
 389  use <computeroutput>-s</computeroutput> for everything.  See
 390  <xref linkend="memory-management"/> below.</para></listitem>
 391 </varlistentry>
 392
 393 <varlistentry>
 394 <term><computeroutput>-q --quiet</computeroutput></term>
 395 <listitem><para>Suppress non-essential warning messages.
 396  Messages pertaining to I/O errors and other critical events
 397  will not be suppressed.</para></listitem>
 398 </varlistentry>
 399
 400 <varlistentry>
 401 <term><computeroutput>-v --verbose</computeroutput></term>
 402 <listitem><para>Verbose mode -- show the compression ratio for
 403  each file processed.  Further
 404  <computeroutput>-v</computeroutput>'s increase the verbosity
 405  level, spewing out lots of information which is primarily of
 406  interest for diagnostic purposes.</para></listitem>
 407 </varlistentry>
 408
 409 <varlistentry>
 410 <term><computeroutput>-L --license -V --version</computeroutput></term>
 411 <listitem><para>Display the software version, license terms and
 412  conditions.</para></listitem>
 413 </varlistentry>
 414
 415 <varlistentry>
 416 <term><computeroutput>-1</computeroutput> (or
 417 <computeroutput>--fast</computeroutput>) to
 418 <computeroutput>-9</computeroutput> (or
 419 <computeroutput>-best</computeroutput>)</term>
 420 <listitem><para>Set the block size to 100 k, 200 k ...  900 k
 421  when compressing.  Has no effect when decompressing.  See <xref
 422  linkend="memory-management" /> below.  The
 423  <computeroutput>--fast</computeroutput> and
 424  <computeroutput>--best</computeroutput> aliases are primarily
 425  for GNU <computeroutput>gzip</computeroutput> compatibility.
 426  In particular, <computeroutput>--fast</computeroutput> doesn't
 427  make things significantly faster.  And
 428  <computeroutput>--best</computeroutput> merely selects the
 429  default behaviour.</para></listitem>
 430 </varlistentry>
 431
 432 <varlistentry>
 433 <term><computeroutput>--</computeroutput></term>
 434 <listitem><para>Treats all subsequent arguments as file names,
 435  even if they start with a dash.  This is so you can handle
 436  files with names beginning with a dash, for example:
 437  <computeroutput>bzip2 --
 438  -myfilename</computeroutput>.</para></listitem>
 439 </varlistentry>
 440
 441 <varlistentry>
 442 <term><computeroutput>--repetitive-fast</computeroutput></term>
 443 <term><computeroutput>--repetitive-best</computeroutput></term>
 444 <listitem><para>These flags are redundant in versions 0.9.5 and
 445  above.  They provided some coarse control over the behaviour of
 446  the sorting algorithm in earlier versions, which was sometimes
 447  useful.  0.9.5 and above have an improved algorithm which
 448  renders these flags irrelevant.</para></listitem>
 449 </varlistentry>
 450
 451</variablelist>
 452
 453</sect1>
 454
 455
 456<sect1 id="memory-management" xreflabel="MEMORY MANAGEMENT">
 457<title>MEMORY MANAGEMENT</title>
 458
 459<para><computeroutput>bzip2</computeroutput> compresses large
 460files in blocks.  The block size affects both the compression
 461ratio achieved, and the amount of memory needed for compression
 462and decompression.  The flags <computeroutput>-1</computeroutput>
 463through <computeroutput>-9</computeroutput> specify the block
 464size to be 100,000 bytes through 900,000 bytes (the default)
 465respectively.  At decompression time, the block size used for
 466compression is read from the header of the compressed file, and
 467<computeroutput>bunzip2</computeroutput> then allocates itself
 468just enough memory to decompress the file.  Since block sizes are
 469stored in compressed files, it follows that the flags
 470<computeroutput>-1</computeroutput> to
 471<computeroutput>-9</computeroutput> are irrelevant to and so
 472ignored during decompression.</para>
 473
 474<para>Compression and decompression requirements, in bytes, can be
 475estimated as:</para>
 476<programlisting>
 477Compression:   400k + ( 8 x block size )
 478
 479Decompression: 100k + ( 4 x block size ), or
 480               100k + ( 2.5 x block size )
 481</programlisting>
 482
 483<para>Larger block sizes give rapidly diminishing marginal
 484returns.  Most of the compression comes from the first two or
 485three hundred k of block size, a fact worth bearing in mind when
 486using <computeroutput>bzip2</computeroutput> on small machines.
 487It is also important to appreciate that the decompression memory
 488requirement is set at compression time by the choice of block
 489size.</para>
 490
 491<para>For files compressed with the default 900k block size,
 492<computeroutput>bunzip2</computeroutput> will require about 3700
 493kbytes to decompress.  To support decompression of any file on a
 4944 megabyte machine, <computeroutput>bunzip2</computeroutput> has
 495an option to decompress using approximately half this amount of
 496memory, about 2300 kbytes.  Decompression speed is also halved,
 497so you should use this option only where necessary.  The relevant
 498flag is <computeroutput>-s</computeroutput>.</para>
 499
 500<para>In general, try and use the largest block size memory
 501constraints allow, since that maximises the compression achieved.
 502Compression and decompression speed are virtually unaffected by
 503block size.</para>
 504
 505<para>Another significant point applies to files which fit in a
 506single block -- that means most files you'd encounter using a
 507large block size.  The amount of real memory touched is
 508proportional to the size of the file, since the file is smaller
 509than a block.  For example, compressing a file 20,000 bytes long
 510with the flag <computeroutput>-9</computeroutput> will cause the
 511compressor to allocate around 7600k of memory, but only touch
 512400k + 20000 * 8 = 560 kbytes of it.  Similarly, the decompressor
 513will allocate 3700k but only touch 100k + 20000 * 4 = 180
 514kbytes.</para>
 515
 516<para>Here is a table which summarises the maximum memory usage
 517for different block sizes.  Also recorded is the total compressed
 518size for 14 files of the Calgary Text Compression Corpus
 519totalling 3,141,622 bytes.  This column gives some feel for how
 520compression varies with block size.  These figures tend to
 521understate the advantage of larger block sizes for larger files,
 522since the Corpus is dominated by smaller files.</para>
 523
 524<programlisting>
 525        Compress   Decompress   Decompress   Corpus
 526Flag     usage      usage       -s usage     Size
 527
 528 -1      1200k       500k         350k      914704
 529 -2      2000k       900k         600k      877703
 530 -3      2800k      1300k         850k      860338
 531 -4      3600k      1700k        1100k      846899
 532 -5      4400k      2100k        1350k      845160
 533 -6      5200k      2500k        1600k      838626
 534 -7      6100k      2900k        1850k      834096
 535 -8      6800k      3300k        2100k      828642
 536 -9      7600k      3700k        2350k      828642
 537</programlisting>
 538
 539</sect1>
 540
 541
 542<sect1 id="recovering" xreflabel="RECOVERING DATA FROM DAMAGED FILES">
 543<title>RECOVERING DATA FROM DAMAGED FILES</title>
 544
 545<para><computeroutput>bzip2</computeroutput> compresses files in
 546blocks, usually 900kbytes long.  Each block is handled
 547independently.  If a media or transmission error causes a
 548multi-block <computeroutput>.bz2</computeroutput> file to become
 549damaged, it may be possible to recover data from the undamaged
 550blocks in the file.</para>
 551
 552<para>The compressed representation of each block is delimited by
 553a 48-bit pattern, which makes it possible to find the block
 554boundaries with reasonable certainty.  Each block also carries
 555its own 32-bit CRC, so damaged blocks can be distinguished from
 556undamaged ones.</para>
 557
 558<para><computeroutput>bzip2recover</computeroutput> is a simple
 559program whose purpose is to search for blocks in
 560<computeroutput>.bz2</computeroutput> files, and write each block
 561out into its own <computeroutput>.bz2</computeroutput> file.  You
 562can then use <computeroutput>bzip2 -t</computeroutput> to test
 563the integrity of the resulting files, and decompress those which
 564are undamaged.</para>
 565
 566<para><computeroutput>bzip2recover</computeroutput> takes a
 567single argument, the name of the damaged file, and writes a
 568number of files <computeroutput>rec0001file.bz2</computeroutput>,
 569<computeroutput>rec0002file.bz2</computeroutput>, etc, containing
 570the extracted blocks.  The output filenames are designed so that
 571the use of wildcards in subsequent processing -- for example,
 572<computeroutput>bzip2 -dc rec*file.bz2 &#62;
 573recovered_data</computeroutput> -- lists the files in the correct
 574order.</para>
 575
 576<para><computeroutput>bzip2recover</computeroutput> should be of
 577most use dealing with large <computeroutput>.bz2</computeroutput>
 578files, as these will contain many blocks.  It is clearly futile
 579to use it on damaged single-block files, since a damaged block
 580cannot be recovered.  If you wish to minimise any potential data
 581loss through media or transmission errors, you might consider
 582compressing with a smaller block size.</para>
 583
 584</sect1>
 585
 586
 587<sect1 id="performance" xreflabel="PERFORMANCE NOTES">
 588<title>PERFORMANCE NOTES</title>
 589
 590<para>The sorting phase of compression gathers together similar
 591strings in the file.  Because of this, files containing very long
 592runs of repeated symbols, like "aabaabaabaab ..."  (repeated
 593several hundred times) may compress more slowly than normal.
 594Versions 0.9.5 and above fare much better than previous versions
 595in this respect.  The ratio between worst-case and average-case
 596compression time is in the region of 10:1.  For previous
 597versions, this figure was more like 100:1.  You can use the
 598<computeroutput>-vvvv</computeroutput> option to monitor progress
 599in great detail, if you want.</para>
 600
 601<para>Decompression speed is unaffected by these
 602phenomena.</para>
 603
 604<para><computeroutput>bzip2</computeroutput> usually allocates
 605several megabytes of memory to operate in, and then charges all
 606over it in a fairly random fashion.  This means that performance,
 607both for compressing and decompressing, is largely determined by
 608the speed at which your machine can service cache misses.
 609Because of this, small changes to the code to reduce the miss
 610rate have been observed to give disproportionately large
 611performance improvements.  I imagine
 612<computeroutput>bzip2</computeroutput> will perform best on
 613machines with very large caches.</para>
 614
 615</sect1>
 616
 617
 618
 619<sect1 id="caveats" xreflabel="CAVEATS">
 620<title>CAVEATS</title>
 621
 622<para>I/O error messages are not as helpful as they could be.
 623<computeroutput>bzip2</computeroutput> tries hard to detect I/O
 624errors and exit cleanly, but the details of what the problem is
 625sometimes seem rather misleading.</para>
 626
 627<para>This manual page pertains to version &bz-version; of
 628<computeroutput>bzip2</computeroutput>.  Compressed data created by
 629this version is entirely forwards and backwards compatible with the
 630previous public releases, versions 0.1pl2, 0.9.0 and 0.9.5, 1.0.0,
 6311.0.1, 1.0.2 and 1.0.3, but with the following exception: 0.9.0 and
 632above can correctly decompress multiple concatenated compressed files.
 6330.1pl2 cannot do this; it will stop after decompressing just the first
 634file in the stream.</para>
 635
 636<para><computeroutput>bzip2recover</computeroutput> versions
 637prior to 1.0.2 used 32-bit integers to represent bit positions in
 638compressed files, so it could not handle compressed files more
 639than 512 megabytes long.  Versions 1.0.2 and above use 64-bit ints
 640on some platforms which support them (GNU supported targets, and
 641Windows). To establish whether or not
 642<computeroutput>bzip2recover</computeroutput> was built with such
 643a limitation, run it without arguments. In any event you can
 644build yourself an unlimited version if you can recompile it with
 645<computeroutput>MaybeUInt64</computeroutput> set to be an
 646unsigned 64-bit integer.</para>
 647
 648</sect1>
 649
 650
 651
 652<sect1 id="author" xreflabel="AUTHOR">
 653<title>AUTHOR</title>
 654
 655<para>Julian Seward,
 656<computeroutput>&bz-email;</computeroutput></para>
 657
 658<para>The ideas embodied in
 659<computeroutput>bzip2</computeroutput> are due to (at least) the
 660following people: Michael Burrows and David Wheeler (for the
 661block sorting transformation), David Wheeler (again, for the
 662Huffman coder), Peter Fenwick (for the structured coding model in
 663the original <computeroutput>bzip</computeroutput>, and many
 664refinements), and Alistair Moffat, Radford Neal and Ian Witten
 665(for the arithmetic coder in the original
 666<computeroutput>bzip</computeroutput>).  I am much indebted for
 667their help, support and advice.  See the manual in the source
 668distribution for pointers to sources of documentation.  Christian
 669von Roques encouraged me to look for faster sorting algorithms,
 670so as to speed up compression.  Bela Lubkin encouraged me to
 671improve the worst-case compression performance.  
 672Donna Robinson XMLised the documentation.
 673Many people sent
 674patches, helped with portability problems, lent machines, gave
 675advice and were generally helpful.</para>
 676
 677</sect1>
 678
 679</chapter>
 680
 681
 682
 683<chapter id="libprog" xreflabel="Programming with libbzip2">
 684<title>
 685Programming with <computeroutput>libbzip2</computeroutput>
 686</title>
 687
 688<para>This chapter describes the programming interface to
 689<computeroutput>libbzip2</computeroutput>.</para>
 690
 691<para>For general background information, particularly about
 692memory use and performance aspects, you'd be well advised to read
 693<xref linkend="using"/> as well.</para>
 694
 695
 696<sect1 id="top-level" xreflabel="Top-level structure">
 697<title>Top-level structure</title>
 698
 699<para><computeroutput>libbzip2</computeroutput> is a flexible
 700library for compressing and decompressing data in the
 701<computeroutput>bzip2</computeroutput> data format.  Although
 702packaged as a single entity, it helps to regard the library as
 703three separate parts: the low level interface, and the high level
 704interface, and some utility functions.</para>
 705
 706<para>The structure of
 707<computeroutput>libbzip2</computeroutput>'s interfaces is similar
 708to that of Jean-loup Gailly's and Mark Adler's excellent
 709<computeroutput>zlib</computeroutput> library.</para>
 710
 711<para>All externally visible symbols have names beginning
 712<computeroutput>BZ2_</computeroutput>.  This is new in version
 7131.0.  The intention is to minimise pollution of the namespaces of
 714library clients.</para>
 715
 716<para>To use any part of the library, you need to
 717<computeroutput>#include &lt;bzlib.h&gt;</computeroutput>
 718into your sources.</para>
 719
 720
 721
 722<sect2 id="ll-summary" xreflabel="Low-level summary">
 723<title>Low-level summary</title>
 724
 725<para>This interface provides services for compressing and
 726decompressing data in memory.  There's no provision for dealing
 727with files, streams or any other I/O mechanisms, just straight
 728memory-to-memory work.  In fact, this part of the library can be
 729compiled without inclusion of
 730<computeroutput>stdio.h</computeroutput>, which may be helpful
 731for embedded applications.</para>
 732
 733<para>The low-level part of the library has no global variables
 734and is therefore thread-safe.</para>
 735
 736<para>Six routines make up the low level interface:
 737<computeroutput>BZ2_bzCompressInit</computeroutput>,
 738<computeroutput>BZ2_bzCompress</computeroutput>, and
 739<computeroutput>BZ2_bzCompressEnd</computeroutput> for
 740compression, and a corresponding trio
 741<computeroutput>BZ2_bzDecompressInit</computeroutput>,
 742<computeroutput>BZ2_bzDecompress</computeroutput> and
 743<computeroutput>BZ2_bzDecompressEnd</computeroutput> for
 744decompression.  The <computeroutput>*Init</computeroutput>
 745functions allocate memory for compression/decompression and do
 746other initialisations, whilst the
 747<computeroutput>*End</computeroutput> functions close down
 748operations and release memory.</para>
 749
 750<para>The real work is done by
 751<computeroutput>BZ2_bzCompress</computeroutput> and
 752<computeroutput>BZ2_bzDecompress</computeroutput>.  These
 753compress and decompress data from a user-supplied input buffer to
 754a user-supplied output buffer.  These buffers can be any size;
 755arbitrary quantities of data are handled by making repeated calls
 756to these functions.  This is a flexible mechanism allowing a
 757consumer-pull style of activity, or producer-push, or a mixture
 758of both.</para>
 759
 760</sect2>
 761
 762
 763<sect2 id="hl-summary" xreflabel="High-level summary">
 764<title>High-level summary</title>
 765
 766<para>This interface provides some handy wrappers around the
 767low-level interface to facilitate reading and writing
 768<computeroutput>bzip2</computeroutput> format files
 769(<computeroutput>.bz2</computeroutput> files).  The routines
 770provide hooks to facilitate reading files in which the
 771<computeroutput>bzip2</computeroutput> data stream is embedded
 772within some larger-scale file structure, or where there are
 773multiple <computeroutput>bzip2</computeroutput> data streams
 774concatenated end-to-end.</para>
 775
 776<para>For reading files,
 777<computeroutput>BZ2_bzReadOpen</computeroutput>,
 778<computeroutput>BZ2_bzRead</computeroutput>,
 779<computeroutput>BZ2_bzReadClose</computeroutput> and 
 780<computeroutput>BZ2_bzReadGetUnused</computeroutput> are
 781supplied.  For writing files,
 782<computeroutput>BZ2_bzWriteOpen</computeroutput>,
 783<computeroutput>BZ2_bzWrite</computeroutput> and
 784<computeroutput>BZ2_bzWriteFinish</computeroutput> are
 785available.</para>
 786
 787<para>As with the low-level library, no global variables are used
 788so the library is per se thread-safe.  However, if I/O errors
 789occur whilst reading or writing the underlying compressed files,
 790you may have to consult <computeroutput>errno</computeroutput> to
 791determine the cause of the error.  In that case, you'd need a C
 792library which correctly supports
 793<computeroutput>errno</computeroutput> in a multithreaded
 794environment.</para>
 795
 796<para>To make the library a little simpler and more portable,
 797<computeroutput>BZ2_bzReadOpen</computeroutput> and
 798<computeroutput>BZ2_bzWriteOpen</computeroutput> require you to
 799pass them file handles (<computeroutput>FILE*</computeroutput>s)
 800which have previously been opened for reading or writing
 801respectively.  That avoids portability problems associated with
 802file operations and file attributes, whilst not being much of an
 803imposition on the programmer.</para>
 804
 805</sect2>
 806
 807
 808<sect2 id="util-fns-summary" xreflabel="Utility functions summary">
 809<title>Utility functions summary</title>
 810
 811<para>For very simple needs,
 812<computeroutput>BZ2_bzBuffToBuffCompress</computeroutput> and
 813<computeroutput>BZ2_bzBuffToBuffDecompress</computeroutput> are
 814provided.  These compress data in memory from one buffer to
 815another buffer in a single function call.  You should assess
 816whether these functions fulfill your memory-to-memory
 817compression/decompression requirements before investing effort in
 818understanding the more general but more complex low-level
 819interface.</para>
 820
 821<para>Yoshioka Tsuneo
 822(<computeroutput>tsuneo@rr.iij4u.or.jp</computeroutput>) has
 823contributed some functions to give better
 824<computeroutput>zlib</computeroutput> compatibility.  These
 825functions are <computeroutput>BZ2_bzopen</computeroutput>,
 826<computeroutput>BZ2_bzread</computeroutput>,
 827<computeroutput>BZ2_bzwrite</computeroutput>,
 828<computeroutput>BZ2_bzflush</computeroutput>,
 829<computeroutput>BZ2_bzclose</computeroutput>,
 830<computeroutput>BZ2_bzerror</computeroutput> and
 831<computeroutput>BZ2_bzlibVersion</computeroutput>.  You may find
 832these functions more convenient for simple file reading and
 833writing, than those in the high-level interface.  These functions
 834are not (yet) officially part of the library, and are minimally
 835documented here.  If they break, you get to keep all the pieces.
 836I hope to document them properly when time permits.</para>
 837
 838<para>Yoshioka also contributed modifications to allow the
 839library to be built as a Windows DLL.</para>
 840
 841</sect2>
 842
 843</sect1>
 844
 845
 846<sect1 id="err-handling" xreflabel="Error handling">
 847<title>Error handling</title>
 848
 849<para>The library is designed to recover cleanly in all
 850situations, including the worst-case situation of decompressing
 851random data.  I'm not 100% sure that it can always do this, so
 852you might want to add a signal handler to catch segmentation
 853violations during decompression if you are feeling especially
 854paranoid.  I would be interested in hearing more about the
 855robustness of the library to corrupted compressed data.</para>
 856
 857<para>Version 1.0.3 more robust in this respect than any
 858previous version.  Investigations with Valgrind (a tool for detecting
 859problems with memory management) indicate
 860that, at least for the few files I tested, all single-bit errors
 861in the decompressed data are caught properly, with no
 862segmentation faults, no uses of uninitialised data, no out of
 863range reads or writes, and no infinite looping in the decompressor.
 864So it's certainly pretty robust, although
 865I wouldn't claim it to be totally bombproof.</para>
 866
 867<para>The file <computeroutput>bzlib.h</computeroutput> contains
 868all definitions needed to use the library.  In particular, you
 869should definitely not include
 870<computeroutput>bzlib_private.h</computeroutput>.</para>
 871
 872<para>In <computeroutput>bzlib.h</computeroutput>, the various
 873return values are defined.  The following list is not intended as
 874an exhaustive description of the circumstances in which a given
 875value may be returned -- those descriptions are given later.
 876Rather, it is intended to convey the rough meaning of each return
 877value.  The first five actions are normal and not intended to
 878denote an error situation.</para>
 879
 880<variablelist>
 881
 882 <varlistentry>
 883  <term><computeroutput>BZ_OK</computeroutput></term>
 884  <listitem><para>The requested action was completed
 885   successfully.</para></listitem>
 886 </varlistentry>
 887
 888 <varlistentry>
 889  <term><computeroutput>BZ_RUN_OK, BZ_FLUSH_OK,
 890    BZ_FINISH_OK</computeroutput></term>
 891  <listitem><para>In 
 892   <computeroutput>BZ2_bzCompress</computeroutput>, the requested
 893   flush/finish/nothing-special action was completed
 894   successfully.</para></listitem>
 895 </varlistentry>
 896
 897 <varlistentry>
 898  <term><computeroutput>BZ_STREAM_END</computeroutput></term>
 899  <listitem><para>Compression of data was completed, or the
 900   logical stream end was detected during
 901   decompression.</para></listitem>
 902 </varlistentry>
 903
 904</variablelist>
 905
 906<para>The following return values indicate an error of some
 907kind.</para>
 908
 909<variablelist>
 910
 911 <varlistentry>
 912  <term><computeroutput>BZ_CONFIG_ERROR</computeroutput></term>
 913  <listitem><para>Indicates that the library has been improperly
 914   compiled on your platform -- a major configuration error.
 915   Specifically, it means that
 916   <computeroutput>sizeof(char)</computeroutput>,
 917   <computeroutput>sizeof(short)</computeroutput> and
 918   <computeroutput>sizeof(int)</computeroutput> are not 1, 2 and
 919   4 respectively, as they should be.  Note that the library
 920   should still work properly on 64-bit platforms which follow
 921   the LP64 programming model -- that is, where
 922   <computeroutput>sizeof(long)</computeroutput> and
 923   <computeroutput>sizeof(void*)</computeroutput> are 8.  Under
 924   LP64, <computeroutput>sizeof(int)</computeroutput> is still 4,
 925   so <computeroutput>libbzip2</computeroutput>, which doesn't
 926   use the <computeroutput>long</computeroutput> type, is
 927   OK.</para></listitem>
 928 </varlistentry>
 929
 930 <varlistentry>
 931  <term><computeroutput>BZ_SEQUENCE_ERROR</computeroutput></term>
 932  <listitem><para>When using the library, it is important to call
 933   the functions in the correct sequence and with data structures
 934   (buffers etc) in the correct states.
 935   <computeroutput>libbzip2</computeroutput> checks as much as it
 936   can to ensure this is happening, and returns
 937   <computeroutput>BZ_SEQUENCE_ERROR</computeroutput> if not.
 938   Code which complies precisely with the function semantics, as
 939   detailed below, should never receive this value; such an event
 940   denotes buggy code which you should
 941   investigate.</para></listitem>
 942 </varlistentry>
 943
 944 <varlistentry>
 945  <term><computeroutput>BZ_PARAM_ERROR</computeroutput></term>
 946  <listitem><para>Returned when a parameter to a function call is
 947   out of range or otherwise manifestly incorrect.  As with
 948   <computeroutput>BZ_SEQUENCE_ERROR</computeroutput>, this
 949   denotes a bug in the client code.  The distinction between
 950   <computeroutput>BZ_PARAM_ERROR</computeroutput> and
 951   <computeroutput>BZ_SEQUENCE_ERROR</computeroutput> is a bit
 952   hazy, but still worth making.</para></listitem>
 953 </varlistentry>
 954
 955 <varlistentry>
 956  <term><computeroutput>BZ_MEM_ERROR</computeroutput></term>
 957  <listitem><para>Returned when a request to allocate memory
 958   failed.  Note that the quantity of memory needed to decompress
 959   a stream cannot be determined until the stream's header has
 960   been read.  So
 961   <computeroutput>BZ2_bzDecompress</computeroutput> and
 962   <computeroutput>BZ2_bzRead</computeroutput> may return
 963   <computeroutput>BZ_MEM_ERROR</computeroutput> even though some
 964   of the compressed data has been read.  The same is not true
 965   for compression; once
 966   <computeroutput>BZ2_bzCompressInit</computeroutput> or
 967   <computeroutput>BZ2_bzWriteOpen</computeroutput> have
 968   successfully completed,
 969   <computeroutput>BZ_MEM_ERROR</computeroutput> cannot
 970   occur.</para></listitem>
 971 </varlistentry>
 972
 973 <varlistentry>
 974  <term><computeroutput>BZ_DATA_ERROR</computeroutput></term>
 975  <listitem><para>Returned when a data integrity error is
 976   detected during decompression.  Most importantly, this means
 977   when stored and computed CRCs for the data do not match.  This
 978   value is also returned upon detection of any other anomaly in
 979   the compressed data.</para></listitem>
 980 </varlistentry>
 981
 982 <varlistentry>
 983  <term><computeroutput>BZ_DATA_ERROR_MAGIC</computeroutput></term>
 984  <listitem><para>As a special case of
 985   <computeroutput>BZ_DATA_ERROR</computeroutput>, it is
 986   sometimes useful to know when the compressed stream does not
 987   start with the correct magic bytes (<computeroutput>'B' 'Z'
 988   'h'</computeroutput>).</para></listitem>
 989 </varlistentry>
 990
 991 <varlistentry>
 992  <term><computeroutput>BZ_IO_ERROR</computeroutput></term>
 993  <listitem><para>Returned by
 994   <computeroutput>BZ2_bzRead</computeroutput> and
 995   <computeroutput>BZ2_bzWrite</computeroutput> when there is an
 996   error reading or writing in the compressed file, and by
 997   <computeroutput>BZ2_bzReadOpen</computeroutput> and
 998   <computeroutput>BZ2_bzWriteOpen</computeroutput> for attempts
 999   to use a file for which the error indicator (viz,
1000   <computeroutput>ferror(f)</computeroutput>) is set.  On
1001   receipt of <computeroutput>BZ_IO_ERROR</computeroutput>, the
1002   caller should consult <computeroutput>errno</computeroutput>
1003   and/or <computeroutput>perror</computeroutput> to acquire
1004   operating-system specific information about the
1005   problem.</para></listitem>
1006 </varlistentry>
1007
1008 <varlistentry>
1009  <term><computeroutput>BZ_UNEXPECTED_EOF</computeroutput></term>
1010  <listitem><para>Returned by
1011   <computeroutput>BZ2_bzRead</computeroutput> when the
1012   compressed file finishes before the logical end of stream is
1013   detected.</para></listitem>
1014 </varlistentry>
1015
1016 <varlistentry>
1017  <term><computeroutput>BZ_OUTBUFF_FULL</computeroutput></term>
1018  <listitem><para>Returned by
1019   <computeroutput>BZ2_bzBuffToBuffCompress</computeroutput> and
1020   <computeroutput>BZ2_bzBuffToBuffDecompress</computeroutput> to
1021   indicate that the output data will not fit into the output
1022   buffer provided.</para></listitem>
1023 </varlistentry>
1024
1025</variablelist>
1026
1027</sect1>
1028
1029
1030
1031<sect1 id="low-level" xreflabel=">Low-level interface">
1032<title>Low-level interface</title>
1033
1034
1035<sect2 id="bzcompress-init" xreflabel="BZ2_bzCompressInit">
1036<title><computeroutput>BZ2_bzCompressInit</computeroutput></title>
1037
1038<programlisting>
1039typedef struct {
1040  char *next_in;
1041  unsigned int avail_in;
1042  unsigned int total_in_lo32;
1043  unsigned int total_in_hi32;
1044
1045  char *next_out;
1046  unsigned int avail_out;
1047  unsigned int total_out_lo32;
1048  unsigned int total_out_hi32;
1049
1050  void *state;
1051
1052  void *(*bzalloc)(void *,int,int);
1053  void (*bzfree)(void *,void *);
1054  void *opaque;
1055} bz_stream;
1056
1057int BZ2_bzCompressInit ( bz_stream *strm, 
1058                         int blockSize100k, 
1059                         int verbosity,
1060                         int workFactor );
1061</programlisting>
1062
1063<para>Prepares for compression.  The
1064<computeroutput>bz_stream</computeroutput> structure holds all
1065data pertaining to the compression activity.  A
1066<computeroutput>bz_stream</computeroutput> structure should be
1067allocated and initialised prior to the call.  The fields of
1068<computeroutput>bz_stream</computeroutput> comprise the entirety
1069of the user-visible data.  <computeroutput>state</computeroutput>
1070is a pointer to the private data structures required for
1071compression.</para>
1072
1073<para>Custom memory allocators are supported, via fields
1074<computeroutput>bzalloc</computeroutput>,
1075<computeroutput>bzfree</computeroutput>, and
1076<computeroutput>opaque</computeroutput>.  The value
1077<computeroutput>opaque</computeroutput> is passed to as the first
1078argument to all calls to <computeroutput>bzalloc</computeroutput>
1079and <computeroutput>bzfree</computeroutput>, but is otherwise
1080ignored by the library.  The call <computeroutput>bzalloc (
1081opaque, n, m )</computeroutput> is expected to return a pointer
1082<computeroutput>p</computeroutput> to <computeroutput>n *
1083m</computeroutput> bytes of memory, and <computeroutput>bzfree (
1084opaque, p )</computeroutput> should free that memory.</para>
1085
1086<para>If you don't want to use a custom memory allocator, set
1087<computeroutput>bzalloc</computeroutput>,
1088<computeroutput>bzfree</computeroutput> and
1089<computeroutput>opaque</computeroutput> to
1090<computeroutput>NULL</computeroutput>, and the library will then
1091use the standard <computeroutput>malloc</computeroutput> /
1092<computeroutput>free</computeroutput> routines.</para>
1093
1094<para>Before calling
1095<computeroutput>BZ2_bzCompressInit</computeroutput>, fields
1096<computeroutput>bzalloc</computeroutput>,
1097<computeroutput>bzfree</computeroutput> and
1098<computeroutput>opaque</computeroutput> should be filled
1099appropriately, as just described.  Upon return, the internal
1100state will have been allocated and initialised, and
1101<computeroutput>total_in_lo32</computeroutput>,
1102<computeroutput>total_in_hi32</computeroutput>,
1103<computeroutput>total_out_lo32</computeroutput> and
1104<computeroutput>total_out_hi32</computeroutput> will have been
1105set to zero.  These four fields are used by the library to inform
1106the caller of the total amount of data passed into and out of the
1107library, respectively.  You should not try to change them.  As of
1108version 1.0, 64-bit counts are maintained, even on 32-bit
1109platforms, using the <computeroutput>_hi32</computeroutput>
1110fields to store the upper 32 bits of the count.  So, for example,
1111the total amount of data in is <computeroutput>(total_in_hi32
1112&#60;&#60; 32) + total_in_lo32</computeroutput>.</para>
1113
1114<para>Parameter <computeroutput>blockSize100k</computeroutput>
1115specifies the block size to be used for compression.  It should
1116be a value between 1 and 9 inclusive, and the actual block size
1117used is 100000 x this figure.  9 gives the best compression but
1118takes most memory.</para>
1119
1120<para>Parameter <computeroutput>verbosity</computeroutput> should
1121be set to a number between 0 and 4 inclusive.  0 is silent, and
1122greater numbers give increasingly verbose monitoring/debugging
1123output.  If the library has been compiled with
1124<computeroutput>-DBZ_NO_STDIO</computeroutput>, no such output
1125will appear for any verbosity setting.</para>
1126
1127<para>Parameter <computeroutput>workFactor</computeroutput>
1128controls how the compression phase behaves when presented with
1129worst case, highly repetitive, input data.  If compression runs
1130into difficulties caused by repetitive data, the library switches
1131from the standard sorting algorithm to a fallback algorithm.  The
1132fallback is slower than the standard algorithm by perhaps a
1133factor of three, but always behaves reasonably, no matter how bad
1134the input.</para>
1135
1136<para>Lower values of <computeroutput>workFactor</computeroutput>
1137reduce the amount of effort the standard algorithm will expend
1138before resorting to the fallback.  You should set this parameter
1139carefully; too low, and many inputs will be handled by the
1140fallback algorithm and so compress rather slowly, too high, and
1141your average-to-worst case compression times can become very
1142large.  The default value of 30 gives reasonable behaviour over a
1143wide range of circumstances.</para>
1144
1145<para>Allowable values range from 0 to 250 inclusive.  0 is a
1146special case, equivalent to using the default value of 30.</para>
1147
1148<para>Note that the compressed output generated is the same
1149regardless of whether or not the fallback algorithm is
1150used.</para>
1151
1152<para>Be aware also that this parameter may disappear entirely in
1153future versions of the library.  In principle it should be
1154possible to devise a good way to automatically choose which
1155algorithm to use.  Such a mechanism would render the parameter
1156obsolete.</para>
1157
1158<para>Possible return values:</para>
1159
1160<programlisting>
1161BZ_CONFIG_ERROR
1162  if the library has been mis-compiled
1163BZ_PARAM_ERROR
1164  if strm is NULL 
1165  or blockSize < 1 or blockSize > 9
1166  or verbosity < 0 or verbosity > 4
1167  or workFactor < 0 or workFactor > 250
1168BZ_MEM_ERROR 
1169  if not enough memory is available
1170BZ_OK 
1171  otherwise
1172</programlisting>
1173
1174<para>Allowable next actions:</para>
1175
1176<programlisting>
1177BZ2_bzCompress
1178  if BZ_OK is returned
1179  no specific action needed in case of error
1180</programlisting>
1181
1182</sect2>
1183
1184
1185<sect2 id="bzCompress" xreflabel="BZ2_bzCompress">
1186<title><computeroutput>BZ2_bzCompress</computeroutput></title>
1187
1188<programlisting>
1189int BZ2_bzCompress ( bz_stream *strm, int action );
1190</programlisting>
1191
1192<para>Provides more input and/or output buffer space for the
1193library.  The caller maintains input and output buffers, and
1194calls <computeroutput>BZ2_bzCompress</computeroutput> to transfer
1195data between them.</para>
1196
1197<para>Before each call to
1198<computeroutput>BZ2_bzCompress</computeroutput>,
1199<computeroutput>next_in</computeroutput> should point at the data
1200to be compressed, and <computeroutput>avail_in</computeroutput>
1201should indicate how many bytes the library may read.
1202<computeroutput>BZ2_bzCompress</computeroutput> updates
1203<computeroutput>next_in</computeroutput>,
1204<computeroutput>avail_in</computeroutput> and
1205<computeroutput>total_in</computeroutput> to reflect the number
1206of bytes it has read.</para>
1207
1208<para>Similarly, <computeroutput>next_out</computeroutput> should
1209point to a buffer in which the compressed data is to be placed,
1210with <computeroutput>avail_out</computeroutput> indicating how
1211much output space is available.
1212<computeroutput>BZ2_bzCompress</computeroutput> updates
1213<computeroutput>next_out</computeroutput>,
1214<computeroutput>avail_out</computeroutput> and
1215<computeroutput>total_out</computeroutput> to reflect the number
1216of bytes output.</para>
1217
1218<para>You may provide and remove as little or as much data as you
1219like on each call of
1220<computeroutput>BZ2_bzCompress</computeroutput>.  In the limit,
1221it is acceptable to supply and remove data one byte at a time,
1222although this would be terribly inefficient.  You should always
1223ensure that at least one byte of output space is available at
1224each call.</para>
1225
1226<para>A second purpose of
1227<computeroutput>BZ2_bzCompress</computeroutput> is to request a
1228change of mode of the compressed stream.</para>
1229
1230<para>Conceptually, a compressed stream can be in one of four
1231states: IDLE, RUNNING, FLUSHING and FINISHING.  Before
1232initialisation
1233(<computeroutput>BZ2_bzCompressInit</computeroutput>) and after
1234termination (<computeroutput>BZ2_bzCompressEnd</computeroutput>),
1235a stream is regarded as IDLE.</para>
1236
1237<para>Upon initialisation
1238(<computeroutput>BZ2_bzCompressInit</computeroutput>), the stream
1239is placed in the RUNNING state.  Subsequent calls to
1240<computeroutput>BZ2_bzCo…

Large files files are truncated, but you can click here to view the full file