/StormLib/stormlib/bzip2/manual.xml
XML | 2964 lines | 2402 code | 561 blank | 1 comment | 0 complexity | 7a003c59ef2f36322fcb36ab195f3f70 MD5 | raw file
Large files files are truncated, but you can click here to view the full file
1<?xml version="1.0"?> <!-- -*- sgml -*- --> 2<!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN" 3 "http://www.oasis-open.org/docbook/xml/4.2/docbookx.dtd"[ 4 5<!-- various strings, dates etc. common to all docs --> 6<!ENTITY % common-ents SYSTEM "entities.xml"> %common-ents; 7]> 8 9<book lang="en" id="userman" xreflabel="bzip2 Manual"> 10 11 <bookinfo> 12 <title>bzip2 and libbzip2, version 1.0.5</title> 13 <subtitle>A program and library for data compression</subtitle> 14 <copyright> 15 <year>&bz-lifespan;</year> 16 <holder>Julian Seward</holder> 17 </copyright> 18 <releaseinfo>Version &bz-version; of &bz-date;</releaseinfo> 19 20 <authorgroup> 21 <author> 22 <firstname>Julian</firstname> 23 <surname>Seward</surname> 24 <affiliation> 25 <orgname>&bz-url;</orgname> 26 </affiliation> 27 </author> 28 </authorgroup> 29 30 <legalnotice> 31 32 <para>This program, <computeroutput>bzip2</computeroutput>, the 33 associated library <computeroutput>libbzip2</computeroutput>, and 34 all documentation, are copyright © &bz-lifespan; Julian Seward. 35 All rights reserved.</para> 36 37 <para>Redistribution and use in source and binary forms, with 38 or without modification, are permitted provided that the 39 following conditions are met:</para> 40 41 <itemizedlist mark='bullet'> 42 43 <listitem><para>Redistributions of source code must retain the 44 above copyright notice, this list of conditions and the 45 following disclaimer.</para></listitem> 46 47 <listitem><para>The origin of this software must not be 48 misrepresented; you must not claim that you wrote the original 49 software. If you use this software in a product, an 50 acknowledgment in the product documentation would be 51 appreciated but is not required.</para></listitem> 52 53 <listitem><para>Altered source versions must be plainly marked 54 as such, and must not be misrepresented as being the original 55 software.</para></listitem> 56 57 <listitem><para>The name of the author may not be used to 58 endorse or promote products derived from this software without 59 specific prior written permission.</para></listitem> 60 61 </itemizedlist> 62 63 <para>THIS SOFTWARE IS PROVIDED BY THE AUTHOR "AS IS" AND ANY 64 EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, 65 THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A 66 PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE 67 AUTHOR BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, 68 EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED 69 TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, 70 DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND 71 ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT 72 LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING 73 IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF 74 THE POSSIBILITY OF SUCH DAMAGE.</para> 75 76 <para>PATENTS: To the best of my knowledge, 77 <computeroutput>bzip2</computeroutput> and 78 <computeroutput>libbzip2</computeroutput> do not use any patented 79 algorithms. However, I do not have the resources to carry 80 out a patent search. Therefore I cannot give any guarantee of 81 the above statement. 82 </para> 83 84</legalnotice> 85 86</bookinfo> 87 88 89 90<chapter id="intro" xreflabel="Introduction"> 91<title>Introduction</title> 92 93<para><computeroutput>bzip2</computeroutput> compresses files 94using the Burrows-Wheeler block-sorting text compression 95algorithm, and Huffman coding. Compression is generally 96considerably better than that achieved by more conventional 97LZ77/LZ78-based compressors, and approaches the performance of 98the PPM family of statistical compressors.</para> 99 100<para><computeroutput>bzip2</computeroutput> is built on top of 101<computeroutput>libbzip2</computeroutput>, a flexible library for 102handling compressed data in the 103<computeroutput>bzip2</computeroutput> format. This manual 104describes both how to use the program and how to work with the 105library interface. Most of the manual is devoted to this 106library, not the program, which is good news if your interest is 107only in the program.</para> 108 109<itemizedlist mark='bullet'> 110 111 <listitem><para><xref linkend="using"/> describes how to use 112 <computeroutput>bzip2</computeroutput>; this is the only part 113 you need to read if you just want to know how to operate the 114 program.</para></listitem> 115 116 <listitem><para><xref linkend="libprog"/> describes the 117 programming interfaces in detail, and</para></listitem> 118 119 <listitem><para><xref linkend="misc"/> records some 120 miscellaneous notes which I thought ought to be recorded 121 somewhere.</para></listitem> 122 123</itemizedlist> 124 125</chapter> 126 127 128<chapter id="using" xreflabel="How to use bzip2"> 129<title>How to use bzip2</title> 130 131<para>This chapter contains a copy of the 132<computeroutput>bzip2</computeroutput> man page, and nothing 133else.</para> 134 135<sect1 id="name" xreflabel="NAME"> 136<title>NAME</title> 137 138<itemizedlist mark='bullet'> 139 140 <listitem><para><computeroutput>bzip2</computeroutput>, 141 <computeroutput>bunzip2</computeroutput> - a block-sorting file 142 compressor, v1.0.4</para></listitem> 143 144 <listitem><para><computeroutput>bzcat</computeroutput> - 145 decompresses files to stdout</para></listitem> 146 147 <listitem><para><computeroutput>bzip2recover</computeroutput> - 148 recovers data from damaged bzip2 files</para></listitem> 149 150</itemizedlist> 151 152</sect1> 153 154 155<sect1 id="synopsis" xreflabel="SYNOPSIS"> 156<title>SYNOPSIS</title> 157 158<itemizedlist mark='bullet'> 159 160 <listitem><para><computeroutput>bzip2</computeroutput> [ 161 -cdfkqstvzVL123456789 ] [ filenames ... ]</para></listitem> 162 163 <listitem><para><computeroutput>bunzip2</computeroutput> [ 164 -fkvsVL ] [ filenames ... ]</para></listitem> 165 166 <listitem><para><computeroutput>bzcat</computeroutput> [ -s ] [ 167 filenames ... ]</para></listitem> 168 169 <listitem><para><computeroutput>bzip2recover</computeroutput> 170 filename</para></listitem> 171 172</itemizedlist> 173 174</sect1> 175 176 177<sect1 id="description" xreflabel="DESCRIPTION"> 178<title>DESCRIPTION</title> 179 180<para><computeroutput>bzip2</computeroutput> compresses files 181using the Burrows-Wheeler block sorting text compression 182algorithm, and Huffman coding. Compression is generally 183considerably better than that achieved by more conventional 184LZ77/LZ78-based compressors, and approaches the performance of 185the PPM family of statistical compressors.</para> 186 187<para>The command-line options are deliberately very similar to 188those of GNU <computeroutput>gzip</computeroutput>, but they are 189not identical.</para> 190 191<para><computeroutput>bzip2</computeroutput> expects a list of 192file names to accompany the command-line flags. Each file is 193replaced by a compressed version of itself, with the name 194<computeroutput>original_name.bz2</computeroutput>. Each 195compressed file has the same modification date, permissions, and, 196when possible, ownership as the corresponding original, so that 197these properties can be correctly restored at decompression time. 198File name handling is naive in the sense that there is no 199mechanism for preserving original file names, permissions, 200ownerships or dates in filesystems which lack these concepts, or 201have serious file name length restrictions, such as 202MS-DOS.</para> 203 204<para><computeroutput>bzip2</computeroutput> and 205<computeroutput>bunzip2</computeroutput> will by default not 206overwrite existing files. If you want this to happen, specify 207the <computeroutput>-f</computeroutput> flag.</para> 208 209<para>If no file names are specified, 210<computeroutput>bzip2</computeroutput> compresses from standard 211input to standard output. In this case, 212<computeroutput>bzip2</computeroutput> will decline to write 213compressed output to a terminal, as this would be entirely 214incomprehensible and therefore pointless.</para> 215 216<para><computeroutput>bunzip2</computeroutput> (or 217<computeroutput>bzip2 -d</computeroutput>) decompresses all 218specified files. Files which were not created by 219<computeroutput>bzip2</computeroutput> will be detected and 220ignored, and a warning issued. 221<computeroutput>bzip2</computeroutput> attempts to guess the 222filename for the decompressed file from that of the compressed 223file as follows:</para> 224 225<itemizedlist mark='bullet'> 226 227 <listitem><para><computeroutput>filename.bz2 </computeroutput> 228 becomes 229 <computeroutput>filename</computeroutput></para></listitem> 230 231 <listitem><para><computeroutput>filename.bz </computeroutput> 232 becomes 233 <computeroutput>filename</computeroutput></para></listitem> 234 235 <listitem><para><computeroutput>filename.tbz2</computeroutput> 236 becomes 237 <computeroutput>filename.tar</computeroutput></para></listitem> 238 239 <listitem><para><computeroutput>filename.tbz </computeroutput> 240 becomes 241 <computeroutput>filename.tar</computeroutput></para></listitem> 242 243 <listitem><para><computeroutput>anyothername </computeroutput> 244 becomes 245 <computeroutput>anyothername.out</computeroutput></para></listitem> 246 247</itemizedlist> 248 249<para>If the file does not end in one of the recognised endings, 250<computeroutput>.bz2</computeroutput>, 251<computeroutput>.bz</computeroutput>, 252<computeroutput>.tbz2</computeroutput> or 253<computeroutput>.tbz</computeroutput>, 254<computeroutput>bzip2</computeroutput> complains that it cannot 255guess the name of the original file, and uses the original name 256with <computeroutput>.out</computeroutput> appended.</para> 257 258<para>As with compression, supplying no filenames causes 259decompression from standard input to standard output.</para> 260 261<para><computeroutput>bunzip2</computeroutput> will correctly 262decompress a file which is the concatenation of two or more 263compressed files. The result is the concatenation of the 264corresponding uncompressed files. Integrity testing 265(<computeroutput>-t</computeroutput>) of concatenated compressed 266files is also supported.</para> 267 268<para>You can also compress or decompress files to the standard 269output by giving the <computeroutput>-c</computeroutput> flag. 270Multiple files may be compressed and decompressed like this. The 271resulting outputs are fed sequentially to stdout. Compression of 272multiple files in this manner generates a stream containing 273multiple compressed file representations. Such a stream can be 274decompressed correctly only by 275<computeroutput>bzip2</computeroutput> version 0.9.0 or later. 276Earlier versions of <computeroutput>bzip2</computeroutput> will 277stop after decompressing the first file in the stream.</para> 278 279<para><computeroutput>bzcat</computeroutput> (or 280<computeroutput>bzip2 -dc</computeroutput>) decompresses all 281specified files to the standard output.</para> 282 283<para><computeroutput>bzip2</computeroutput> will read arguments 284from the environment variables 285<computeroutput>BZIP2</computeroutput> and 286<computeroutput>BZIP</computeroutput>, in that order, and will 287process them before any arguments read from the command line. 288This gives a convenient way to supply default arguments.</para> 289 290<para>Compression is always performed, even if the compressed 291file is slightly larger than the original. Files of less than 292about one hundred bytes tend to get larger, since the compression 293mechanism has a constant overhead in the region of 50 bytes. 294Random data (including the output of most file compressors) is 295coded at about 8.05 bits per byte, giving an expansion of around 2960.5%.</para> 297 298<para>As a self-check for your protection, 299<computeroutput>bzip2</computeroutput> uses 32-bit CRCs to make 300sure that the decompressed version of a file is identical to the 301original. This guards against corruption of the compressed data, 302and against undetected bugs in 303<computeroutput>bzip2</computeroutput> (hopefully very unlikely). 304The chances of data corruption going undetected is microscopic, 305about one chance in four billion for each file processed. Be 306aware, though, that the check occurs upon decompression, so it 307can only tell you that something is wrong. It can't help you 308recover the original uncompressed data. You can use 309<computeroutput>bzip2recover</computeroutput> to try to recover 310data from damaged files.</para> 311 312<para>Return values: 0 for a normal exit, 1 for environmental 313problems (file not found, invalid flags, I/O errors, etc.), 2 314to indicate a corrupt compressed file, 3 for an internal 315consistency error (eg, bug) which caused 316<computeroutput>bzip2</computeroutput> to panic.</para> 317 318</sect1> 319 320 321<sect1 id="options" xreflabel="OPTIONS"> 322<title>OPTIONS</title> 323 324<variablelist> 325 326 <varlistentry> 327 <term><computeroutput>-c --stdout</computeroutput></term> 328 <listitem><para>Compress or decompress to standard 329 output.</para></listitem> 330 </varlistentry> 331 332 <varlistentry> 333 <term><computeroutput>-d --decompress</computeroutput></term> 334 <listitem><para>Force decompression. 335 <computeroutput>bzip2</computeroutput>, 336 <computeroutput>bunzip2</computeroutput> and 337 <computeroutput>bzcat</computeroutput> are really the same 338 program, and the decision about what actions to take is done on 339 the basis of which name is used. This flag overrides that 340 mechanism, and forces bzip2 to decompress.</para></listitem> 341 </varlistentry> 342 343 <varlistentry> 344 <term><computeroutput>-z --compress</computeroutput></term> 345 <listitem><para>The complement to 346 <computeroutput>-d</computeroutput>: forces compression, 347 regardless of the invokation name.</para></listitem> 348 </varlistentry> 349 350 <varlistentry> 351 <term><computeroutput>-t --test</computeroutput></term> 352 <listitem><para>Check integrity of the specified file(s), but 353 don't decompress them. This really performs a trial 354 decompression and throws away the result.</para></listitem> 355 </varlistentry> 356 357 <varlistentry> 358 <term><computeroutput>-f --force</computeroutput></term> 359 <listitem><para>Force overwrite of output files. Normally, 360 <computeroutput>bzip2</computeroutput> will not overwrite 361 existing output files. Also forces 362 <computeroutput>bzip2</computeroutput> to break hard links to 363 files, which it otherwise wouldn't do.</para> 364 <para><computeroutput>bzip2</computeroutput> normally declines 365 to decompress files which don't have the correct magic header 366 bytes. If forced (<computeroutput>-f</computeroutput>), 367 however, it will pass such files through unmodified. This is 368 how GNU <computeroutput>gzip</computeroutput> behaves.</para> 369 </listitem> 370 </varlistentry> 371 372 <varlistentry> 373 <term><computeroutput>-k --keep</computeroutput></term> 374 <listitem><para>Keep (don't delete) input files during 375 compression or decompression.</para></listitem> 376 </varlistentry> 377 378 <varlistentry> 379 <term><computeroutput>-s --small</computeroutput></term> 380 <listitem><para>Reduce memory usage, for compression, 381 decompression and testing. Files are decompressed and tested 382 using a modified algorithm which only requires 2.5 bytes per 383 block byte. This means any file can be decompressed in 2300k 384 of memory, albeit at about half the normal speed.</para> 385 <para>During compression, <computeroutput>-s</computeroutput> 386 selects a block size of 200k, which limits memory use to around 387 the same figure, at the expense of your compression ratio. In 388 short, if your machine is low on memory (8 megabytes or less), 389 use <computeroutput>-s</computeroutput> for everything. See 390 <xref linkend="memory-management"/> below.</para></listitem> 391 </varlistentry> 392 393 <varlistentry> 394 <term><computeroutput>-q --quiet</computeroutput></term> 395 <listitem><para>Suppress non-essential warning messages. 396 Messages pertaining to I/O errors and other critical events 397 will not be suppressed.</para></listitem> 398 </varlistentry> 399 400 <varlistentry> 401 <term><computeroutput>-v --verbose</computeroutput></term> 402 <listitem><para>Verbose mode -- show the compression ratio for 403 each file processed. Further 404 <computeroutput>-v</computeroutput>'s increase the verbosity 405 level, spewing out lots of information which is primarily of 406 interest for diagnostic purposes.</para></listitem> 407 </varlistentry> 408 409 <varlistentry> 410 <term><computeroutput>-L --license -V --version</computeroutput></term> 411 <listitem><para>Display the software version, license terms and 412 conditions.</para></listitem> 413 </varlistentry> 414 415 <varlistentry> 416 <term><computeroutput>-1</computeroutput> (or 417 <computeroutput>--fast</computeroutput>) to 418 <computeroutput>-9</computeroutput> (or 419 <computeroutput>-best</computeroutput>)</term> 420 <listitem><para>Set the block size to 100 k, 200 k ... 900 k 421 when compressing. Has no effect when decompressing. See <xref 422 linkend="memory-management" /> below. The 423 <computeroutput>--fast</computeroutput> and 424 <computeroutput>--best</computeroutput> aliases are primarily 425 for GNU <computeroutput>gzip</computeroutput> compatibility. 426 In particular, <computeroutput>--fast</computeroutput> doesn't 427 make things significantly faster. And 428 <computeroutput>--best</computeroutput> merely selects the 429 default behaviour.</para></listitem> 430 </varlistentry> 431 432 <varlistentry> 433 <term><computeroutput>--</computeroutput></term> 434 <listitem><para>Treats all subsequent arguments as file names, 435 even if they start with a dash. This is so you can handle 436 files with names beginning with a dash, for example: 437 <computeroutput>bzip2 -- 438 -myfilename</computeroutput>.</para></listitem> 439 </varlistentry> 440 441 <varlistentry> 442 <term><computeroutput>--repetitive-fast</computeroutput></term> 443 <term><computeroutput>--repetitive-best</computeroutput></term> 444 <listitem><para>These flags are redundant in versions 0.9.5 and 445 above. They provided some coarse control over the behaviour of 446 the sorting algorithm in earlier versions, which was sometimes 447 useful. 0.9.5 and above have an improved algorithm which 448 renders these flags irrelevant.</para></listitem> 449 </varlistentry> 450 451</variablelist> 452 453</sect1> 454 455 456<sect1 id="memory-management" xreflabel="MEMORY MANAGEMENT"> 457<title>MEMORY MANAGEMENT</title> 458 459<para><computeroutput>bzip2</computeroutput> compresses large 460files in blocks. The block size affects both the compression 461ratio achieved, and the amount of memory needed for compression 462and decompression. The flags <computeroutput>-1</computeroutput> 463through <computeroutput>-9</computeroutput> specify the block 464size to be 100,000 bytes through 900,000 bytes (the default) 465respectively. At decompression time, the block size used for 466compression is read from the header of the compressed file, and 467<computeroutput>bunzip2</computeroutput> then allocates itself 468just enough memory to decompress the file. Since block sizes are 469stored in compressed files, it follows that the flags 470<computeroutput>-1</computeroutput> to 471<computeroutput>-9</computeroutput> are irrelevant to and so 472ignored during decompression.</para> 473 474<para>Compression and decompression requirements, in bytes, can be 475estimated as:</para> 476<programlisting> 477Compression: 400k + ( 8 x block size ) 478 479Decompression: 100k + ( 4 x block size ), or 480 100k + ( 2.5 x block size ) 481</programlisting> 482 483<para>Larger block sizes give rapidly diminishing marginal 484returns. Most of the compression comes from the first two or 485three hundred k of block size, a fact worth bearing in mind when 486using <computeroutput>bzip2</computeroutput> on small machines. 487It is also important to appreciate that the decompression memory 488requirement is set at compression time by the choice of block 489size.</para> 490 491<para>For files compressed with the default 900k block size, 492<computeroutput>bunzip2</computeroutput> will require about 3700 493kbytes to decompress. To support decompression of any file on a 4944 megabyte machine, <computeroutput>bunzip2</computeroutput> has 495an option to decompress using approximately half this amount of 496memory, about 2300 kbytes. Decompression speed is also halved, 497so you should use this option only where necessary. The relevant 498flag is <computeroutput>-s</computeroutput>.</para> 499 500<para>In general, try and use the largest block size memory 501constraints allow, since that maximises the compression achieved. 502Compression and decompression speed are virtually unaffected by 503block size.</para> 504 505<para>Another significant point applies to files which fit in a 506single block -- that means most files you'd encounter using a 507large block size. The amount of real memory touched is 508proportional to the size of the file, since the file is smaller 509than a block. For example, compressing a file 20,000 bytes long 510with the flag <computeroutput>-9</computeroutput> will cause the 511compressor to allocate around 7600k of memory, but only touch 512400k + 20000 * 8 = 560 kbytes of it. Similarly, the decompressor 513will allocate 3700k but only touch 100k + 20000 * 4 = 180 514kbytes.</para> 515 516<para>Here is a table which summarises the maximum memory usage 517for different block sizes. Also recorded is the total compressed 518size for 14 files of the Calgary Text Compression Corpus 519totalling 3,141,622 bytes. This column gives some feel for how 520compression varies with block size. These figures tend to 521understate the advantage of larger block sizes for larger files, 522since the Corpus is dominated by smaller files.</para> 523 524<programlisting> 525 Compress Decompress Decompress Corpus 526Flag usage usage -s usage Size 527 528 -1 1200k 500k 350k 914704 529 -2 2000k 900k 600k 877703 530 -3 2800k 1300k 850k 860338 531 -4 3600k 1700k 1100k 846899 532 -5 4400k 2100k 1350k 845160 533 -6 5200k 2500k 1600k 838626 534 -7 6100k 2900k 1850k 834096 535 -8 6800k 3300k 2100k 828642 536 -9 7600k 3700k 2350k 828642 537</programlisting> 538 539</sect1> 540 541 542<sect1 id="recovering" xreflabel="RECOVERING DATA FROM DAMAGED FILES"> 543<title>RECOVERING DATA FROM DAMAGED FILES</title> 544 545<para><computeroutput>bzip2</computeroutput> compresses files in 546blocks, usually 900kbytes long. Each block is handled 547independently. If a media or transmission error causes a 548multi-block <computeroutput>.bz2</computeroutput> file to become 549damaged, it may be possible to recover data from the undamaged 550blocks in the file.</para> 551 552<para>The compressed representation of each block is delimited by 553a 48-bit pattern, which makes it possible to find the block 554boundaries with reasonable certainty. Each block also carries 555its own 32-bit CRC, so damaged blocks can be distinguished from 556undamaged ones.</para> 557 558<para><computeroutput>bzip2recover</computeroutput> is a simple 559program whose purpose is to search for blocks in 560<computeroutput>.bz2</computeroutput> files, and write each block 561out into its own <computeroutput>.bz2</computeroutput> file. You 562can then use <computeroutput>bzip2 -t</computeroutput> to test 563the integrity of the resulting files, and decompress those which 564are undamaged.</para> 565 566<para><computeroutput>bzip2recover</computeroutput> takes a 567single argument, the name of the damaged file, and writes a 568number of files <computeroutput>rec0001file.bz2</computeroutput>, 569<computeroutput>rec0002file.bz2</computeroutput>, etc, containing 570the extracted blocks. The output filenames are designed so that 571the use of wildcards in subsequent processing -- for example, 572<computeroutput>bzip2 -dc rec*file.bz2 > 573recovered_data</computeroutput> -- lists the files in the correct 574order.</para> 575 576<para><computeroutput>bzip2recover</computeroutput> should be of 577most use dealing with large <computeroutput>.bz2</computeroutput> 578files, as these will contain many blocks. It is clearly futile 579to use it on damaged single-block files, since a damaged block 580cannot be recovered. If you wish to minimise any potential data 581loss through media or transmission errors, you might consider 582compressing with a smaller block size.</para> 583 584</sect1> 585 586 587<sect1 id="performance" xreflabel="PERFORMANCE NOTES"> 588<title>PERFORMANCE NOTES</title> 589 590<para>The sorting phase of compression gathers together similar 591strings in the file. Because of this, files containing very long 592runs of repeated symbols, like "aabaabaabaab ..." (repeated 593several hundred times) may compress more slowly than normal. 594Versions 0.9.5 and above fare much better than previous versions 595in this respect. The ratio between worst-case and average-case 596compression time is in the region of 10:1. For previous 597versions, this figure was more like 100:1. You can use the 598<computeroutput>-vvvv</computeroutput> option to monitor progress 599in great detail, if you want.</para> 600 601<para>Decompression speed is unaffected by these 602phenomena.</para> 603 604<para><computeroutput>bzip2</computeroutput> usually allocates 605several megabytes of memory to operate in, and then charges all 606over it in a fairly random fashion. This means that performance, 607both for compressing and decompressing, is largely determined by 608the speed at which your machine can service cache misses. 609Because of this, small changes to the code to reduce the miss 610rate have been observed to give disproportionately large 611performance improvements. I imagine 612<computeroutput>bzip2</computeroutput> will perform best on 613machines with very large caches.</para> 614 615</sect1> 616 617 618 619<sect1 id="caveats" xreflabel="CAVEATS"> 620<title>CAVEATS</title> 621 622<para>I/O error messages are not as helpful as they could be. 623<computeroutput>bzip2</computeroutput> tries hard to detect I/O 624errors and exit cleanly, but the details of what the problem is 625sometimes seem rather misleading.</para> 626 627<para>This manual page pertains to version &bz-version; of 628<computeroutput>bzip2</computeroutput>. Compressed data created by 629this version is entirely forwards and backwards compatible with the 630previous public releases, versions 0.1pl2, 0.9.0 and 0.9.5, 1.0.0, 6311.0.1, 1.0.2 and 1.0.3, but with the following exception: 0.9.0 and 632above can correctly decompress multiple concatenated compressed files. 6330.1pl2 cannot do this; it will stop after decompressing just the first 634file in the stream.</para> 635 636<para><computeroutput>bzip2recover</computeroutput> versions 637prior to 1.0.2 used 32-bit integers to represent bit positions in 638compressed files, so it could not handle compressed files more 639than 512 megabytes long. Versions 1.0.2 and above use 64-bit ints 640on some platforms which support them (GNU supported targets, and 641Windows). To establish whether or not 642<computeroutput>bzip2recover</computeroutput> was built with such 643a limitation, run it without arguments. In any event you can 644build yourself an unlimited version if you can recompile it with 645<computeroutput>MaybeUInt64</computeroutput> set to be an 646unsigned 64-bit integer.</para> 647 648</sect1> 649 650 651 652<sect1 id="author" xreflabel="AUTHOR"> 653<title>AUTHOR</title> 654 655<para>Julian Seward, 656<computeroutput>&bz-email;</computeroutput></para> 657 658<para>The ideas embodied in 659<computeroutput>bzip2</computeroutput> are due to (at least) the 660following people: Michael Burrows and David Wheeler (for the 661block sorting transformation), David Wheeler (again, for the 662Huffman coder), Peter Fenwick (for the structured coding model in 663the original <computeroutput>bzip</computeroutput>, and many 664refinements), and Alistair Moffat, Radford Neal and Ian Witten 665(for the arithmetic coder in the original 666<computeroutput>bzip</computeroutput>). I am much indebted for 667their help, support and advice. See the manual in the source 668distribution for pointers to sources of documentation. Christian 669von Roques encouraged me to look for faster sorting algorithms, 670so as to speed up compression. Bela Lubkin encouraged me to 671improve the worst-case compression performance. 672Donna Robinson XMLised the documentation. 673Many people sent 674patches, helped with portability problems, lent machines, gave 675advice and were generally helpful.</para> 676 677</sect1> 678 679</chapter> 680 681 682 683<chapter id="libprog" xreflabel="Programming with libbzip2"> 684<title> 685Programming with <computeroutput>libbzip2</computeroutput> 686</title> 687 688<para>This chapter describes the programming interface to 689<computeroutput>libbzip2</computeroutput>.</para> 690 691<para>For general background information, particularly about 692memory use and performance aspects, you'd be well advised to read 693<xref linkend="using"/> as well.</para> 694 695 696<sect1 id="top-level" xreflabel="Top-level structure"> 697<title>Top-level structure</title> 698 699<para><computeroutput>libbzip2</computeroutput> is a flexible 700library for compressing and decompressing data in the 701<computeroutput>bzip2</computeroutput> data format. Although 702packaged as a single entity, it helps to regard the library as 703three separate parts: the low level interface, and the high level 704interface, and some utility functions.</para> 705 706<para>The structure of 707<computeroutput>libbzip2</computeroutput>'s interfaces is similar 708to that of Jean-loup Gailly's and Mark Adler's excellent 709<computeroutput>zlib</computeroutput> library.</para> 710 711<para>All externally visible symbols have names beginning 712<computeroutput>BZ2_</computeroutput>. This is new in version 7131.0. The intention is to minimise pollution of the namespaces of 714library clients.</para> 715 716<para>To use any part of the library, you need to 717<computeroutput>#include <bzlib.h></computeroutput> 718into your sources.</para> 719 720 721 722<sect2 id="ll-summary" xreflabel="Low-level summary"> 723<title>Low-level summary</title> 724 725<para>This interface provides services for compressing and 726decompressing data in memory. There's no provision for dealing 727with files, streams or any other I/O mechanisms, just straight 728memory-to-memory work. In fact, this part of the library can be 729compiled without inclusion of 730<computeroutput>stdio.h</computeroutput>, which may be helpful 731for embedded applications.</para> 732 733<para>The low-level part of the library has no global variables 734and is therefore thread-safe.</para> 735 736<para>Six routines make up the low level interface: 737<computeroutput>BZ2_bzCompressInit</computeroutput>, 738<computeroutput>BZ2_bzCompress</computeroutput>, and 739<computeroutput>BZ2_bzCompressEnd</computeroutput> for 740compression, and a corresponding trio 741<computeroutput>BZ2_bzDecompressInit</computeroutput>, 742<computeroutput>BZ2_bzDecompress</computeroutput> and 743<computeroutput>BZ2_bzDecompressEnd</computeroutput> for 744decompression. The <computeroutput>*Init</computeroutput> 745functions allocate memory for compression/decompression and do 746other initialisations, whilst the 747<computeroutput>*End</computeroutput> functions close down 748operations and release memory.</para> 749 750<para>The real work is done by 751<computeroutput>BZ2_bzCompress</computeroutput> and 752<computeroutput>BZ2_bzDecompress</computeroutput>. These 753compress and decompress data from a user-supplied input buffer to 754a user-supplied output buffer. These buffers can be any size; 755arbitrary quantities of data are handled by making repeated calls 756to these functions. This is a flexible mechanism allowing a 757consumer-pull style of activity, or producer-push, or a mixture 758of both.</para> 759 760</sect2> 761 762 763<sect2 id="hl-summary" xreflabel="High-level summary"> 764<title>High-level summary</title> 765 766<para>This interface provides some handy wrappers around the 767low-level interface to facilitate reading and writing 768<computeroutput>bzip2</computeroutput> format files 769(<computeroutput>.bz2</computeroutput> files). The routines 770provide hooks to facilitate reading files in which the 771<computeroutput>bzip2</computeroutput> data stream is embedded 772within some larger-scale file structure, or where there are 773multiple <computeroutput>bzip2</computeroutput> data streams 774concatenated end-to-end.</para> 775 776<para>For reading files, 777<computeroutput>BZ2_bzReadOpen</computeroutput>, 778<computeroutput>BZ2_bzRead</computeroutput>, 779<computeroutput>BZ2_bzReadClose</computeroutput> and 780<computeroutput>BZ2_bzReadGetUnused</computeroutput> are 781supplied. For writing files, 782<computeroutput>BZ2_bzWriteOpen</computeroutput>, 783<computeroutput>BZ2_bzWrite</computeroutput> and 784<computeroutput>BZ2_bzWriteFinish</computeroutput> are 785available.</para> 786 787<para>As with the low-level library, no global variables are used 788so the library is per se thread-safe. However, if I/O errors 789occur whilst reading or writing the underlying compressed files, 790you may have to consult <computeroutput>errno</computeroutput> to 791determine the cause of the error. In that case, you'd need a C 792library which correctly supports 793<computeroutput>errno</computeroutput> in a multithreaded 794environment.</para> 795 796<para>To make the library a little simpler and more portable, 797<computeroutput>BZ2_bzReadOpen</computeroutput> and 798<computeroutput>BZ2_bzWriteOpen</computeroutput> require you to 799pass them file handles (<computeroutput>FILE*</computeroutput>s) 800which have previously been opened for reading or writing 801respectively. That avoids portability problems associated with 802file operations and file attributes, whilst not being much of an 803imposition on the programmer.</para> 804 805</sect2> 806 807 808<sect2 id="util-fns-summary" xreflabel="Utility functions summary"> 809<title>Utility functions summary</title> 810 811<para>For very simple needs, 812<computeroutput>BZ2_bzBuffToBuffCompress</computeroutput> and 813<computeroutput>BZ2_bzBuffToBuffDecompress</computeroutput> are 814provided. These compress data in memory from one buffer to 815another buffer in a single function call. You should assess 816whether these functions fulfill your memory-to-memory 817compression/decompression requirements before investing effort in 818understanding the more general but more complex low-level 819interface.</para> 820 821<para>Yoshioka Tsuneo 822(<computeroutput>tsuneo@rr.iij4u.or.jp</computeroutput>) has 823contributed some functions to give better 824<computeroutput>zlib</computeroutput> compatibility. These 825functions are <computeroutput>BZ2_bzopen</computeroutput>, 826<computeroutput>BZ2_bzread</computeroutput>, 827<computeroutput>BZ2_bzwrite</computeroutput>, 828<computeroutput>BZ2_bzflush</computeroutput>, 829<computeroutput>BZ2_bzclose</computeroutput>, 830<computeroutput>BZ2_bzerror</computeroutput> and 831<computeroutput>BZ2_bzlibVersion</computeroutput>. You may find 832these functions more convenient for simple file reading and 833writing, than those in the high-level interface. These functions 834are not (yet) officially part of the library, and are minimally 835documented here. If they break, you get to keep all the pieces. 836I hope to document them properly when time permits.</para> 837 838<para>Yoshioka also contributed modifications to allow the 839library to be built as a Windows DLL.</para> 840 841</sect2> 842 843</sect1> 844 845 846<sect1 id="err-handling" xreflabel="Error handling"> 847<title>Error handling</title> 848 849<para>The library is designed to recover cleanly in all 850situations, including the worst-case situation of decompressing 851random data. I'm not 100% sure that it can always do this, so 852you might want to add a signal handler to catch segmentation 853violations during decompression if you are feeling especially 854paranoid. I would be interested in hearing more about the 855robustness of the library to corrupted compressed data.</para> 856 857<para>Version 1.0.3 more robust in this respect than any 858previous version. Investigations with Valgrind (a tool for detecting 859problems with memory management) indicate 860that, at least for the few files I tested, all single-bit errors 861in the decompressed data are caught properly, with no 862segmentation faults, no uses of uninitialised data, no out of 863range reads or writes, and no infinite looping in the decompressor. 864So it's certainly pretty robust, although 865I wouldn't claim it to be totally bombproof.</para> 866 867<para>The file <computeroutput>bzlib.h</computeroutput> contains 868all definitions needed to use the library. In particular, you 869should definitely not include 870<computeroutput>bzlib_private.h</computeroutput>.</para> 871 872<para>In <computeroutput>bzlib.h</computeroutput>, the various 873return values are defined. The following list is not intended as 874an exhaustive description of the circumstances in which a given 875value may be returned -- those descriptions are given later. 876Rather, it is intended to convey the rough meaning of each return 877value. The first five actions are normal and not intended to 878denote an error situation.</para> 879 880<variablelist> 881 882 <varlistentry> 883 <term><computeroutput>BZ_OK</computeroutput></term> 884 <listitem><para>The requested action was completed 885 successfully.</para></listitem> 886 </varlistentry> 887 888 <varlistentry> 889 <term><computeroutput>BZ_RUN_OK, BZ_FLUSH_OK, 890 BZ_FINISH_OK</computeroutput></term> 891 <listitem><para>In 892 <computeroutput>BZ2_bzCompress</computeroutput>, the requested 893 flush/finish/nothing-special action was completed 894 successfully.</para></listitem> 895 </varlistentry> 896 897 <varlistentry> 898 <term><computeroutput>BZ_STREAM_END</computeroutput></term> 899 <listitem><para>Compression of data was completed, or the 900 logical stream end was detected during 901 decompression.</para></listitem> 902 </varlistentry> 903 904</variablelist> 905 906<para>The following return values indicate an error of some 907kind.</para> 908 909<variablelist> 910 911 <varlistentry> 912 <term><computeroutput>BZ_CONFIG_ERROR</computeroutput></term> 913 <listitem><para>Indicates that the library has been improperly 914 compiled on your platform -- a major configuration error. 915 Specifically, it means that 916 <computeroutput>sizeof(char)</computeroutput>, 917 <computeroutput>sizeof(short)</computeroutput> and 918 <computeroutput>sizeof(int)</computeroutput> are not 1, 2 and 919 4 respectively, as they should be. Note that the library 920 should still work properly on 64-bit platforms which follow 921 the LP64 programming model -- that is, where 922 <computeroutput>sizeof(long)</computeroutput> and 923 <computeroutput>sizeof(void*)</computeroutput> are 8. Under 924 LP64, <computeroutput>sizeof(int)</computeroutput> is still 4, 925 so <computeroutput>libbzip2</computeroutput>, which doesn't 926 use the <computeroutput>long</computeroutput> type, is 927 OK.</para></listitem> 928 </varlistentry> 929 930 <varlistentry> 931 <term><computeroutput>BZ_SEQUENCE_ERROR</computeroutput></term> 932 <listitem><para>When using the library, it is important to call 933 the functions in the correct sequence and with data structures 934 (buffers etc) in the correct states. 935 <computeroutput>libbzip2</computeroutput> checks as much as it 936 can to ensure this is happening, and returns 937 <computeroutput>BZ_SEQUENCE_ERROR</computeroutput> if not. 938 Code which complies precisely with the function semantics, as 939 detailed below, should never receive this value; such an event 940 denotes buggy code which you should 941 investigate.</para></listitem> 942 </varlistentry> 943 944 <varlistentry> 945 <term><computeroutput>BZ_PARAM_ERROR</computeroutput></term> 946 <listitem><para>Returned when a parameter to a function call is 947 out of range or otherwise manifestly incorrect. As with 948 <computeroutput>BZ_SEQUENCE_ERROR</computeroutput>, this 949 denotes a bug in the client code. The distinction between 950 <computeroutput>BZ_PARAM_ERROR</computeroutput> and 951 <computeroutput>BZ_SEQUENCE_ERROR</computeroutput> is a bit 952 hazy, but still worth making.</para></listitem> 953 </varlistentry> 954 955 <varlistentry> 956 <term><computeroutput>BZ_MEM_ERROR</computeroutput></term> 957 <listitem><para>Returned when a request to allocate memory 958 failed. Note that the quantity of memory needed to decompress 959 a stream cannot be determined until the stream's header has 960 been read. So 961 <computeroutput>BZ2_bzDecompress</computeroutput> and 962 <computeroutput>BZ2_bzRead</computeroutput> may return 963 <computeroutput>BZ_MEM_ERROR</computeroutput> even though some 964 of the compressed data has been read. The same is not true 965 for compression; once 966 <computeroutput>BZ2_bzCompressInit</computeroutput> or 967 <computeroutput>BZ2_bzWriteOpen</computeroutput> have 968 successfully completed, 969 <computeroutput>BZ_MEM_ERROR</computeroutput> cannot 970 occur.</para></listitem> 971 </varlistentry> 972 973 <varlistentry> 974 <term><computeroutput>BZ_DATA_ERROR</computeroutput></term> 975 <listitem><para>Returned when a data integrity error is 976 detected during decompression. Most importantly, this means 977 when stored and computed CRCs for the data do not match. This 978 value is also returned upon detection of any other anomaly in 979 the compressed data.</para></listitem> 980 </varlistentry> 981 982 <varlistentry> 983 <term><computeroutput>BZ_DATA_ERROR_MAGIC</computeroutput></term> 984 <listitem><para>As a special case of 985 <computeroutput>BZ_DATA_ERROR</computeroutput>, it is 986 sometimes useful to know when the compressed stream does not 987 start with the correct magic bytes (<computeroutput>'B' 'Z' 988 'h'</computeroutput>).</para></listitem> 989 </varlistentry> 990 991 <varlistentry> 992 <term><computeroutput>BZ_IO_ERROR</computeroutput></term> 993 <listitem><para>Returned by 994 <computeroutput>BZ2_bzRead</computeroutput> and 995 <computeroutput>BZ2_bzWrite</computeroutput> when there is an 996 error reading or writing in the compressed file, and by 997 <computeroutput>BZ2_bzReadOpen</computeroutput> and 998 <computeroutput>BZ2_bzWriteOpen</computeroutput> for attempts 999 to use a file for which the error indicator (viz, 1000 <computeroutput>ferror(f)</computeroutput>) is set. On 1001 receipt of <computeroutput>BZ_IO_ERROR</computeroutput>, the 1002 caller should consult <computeroutput>errno</computeroutput> 1003 and/or <computeroutput>perror</computeroutput> to acquire 1004 operating-system specific information about the 1005 problem.</para></listitem> 1006 </varlistentry> 1007 1008 <varlistentry> 1009 <term><computeroutput>BZ_UNEXPECTED_EOF</computeroutput></term> 1010 <listitem><para>Returned by 1011 <computeroutput>BZ2_bzRead</computeroutput> when the 1012 compressed file finishes before the logical end of stream is 1013 detected.</para></listitem> 1014 </varlistentry> 1015 1016 <varlistentry> 1017 <term><computeroutput>BZ_OUTBUFF_FULL</computeroutput></term> 1018 <listitem><para>Returned by 1019 <computeroutput>BZ2_bzBuffToBuffCompress</computeroutput> and 1020 <computeroutput>BZ2_bzBuffToBuffDecompress</computeroutput> to 1021 indicate that the output data will not fit into the output 1022 buffer provided.</para></listitem> 1023 </varlistentry> 1024 1025</variablelist> 1026 1027</sect1> 1028 1029 1030 1031<sect1 id="low-level" xreflabel=">Low-level interface"> 1032<title>Low-level interface</title> 1033 1034 1035<sect2 id="bzcompress-init" xreflabel="BZ2_bzCompressInit"> 1036<title><computeroutput>BZ2_bzCompressInit</computeroutput></title> 1037 1038<programlisting> 1039typedef struct { 1040 char *next_in; 1041 unsigned int avail_in; 1042 unsigned int total_in_lo32; 1043 unsigned int total_in_hi32; 1044 1045 char *next_out; 1046 unsigned int avail_out; 1047 unsigned int total_out_lo32; 1048 unsigned int total_out_hi32; 1049 1050 void *state; 1051 1052 void *(*bzalloc)(void *,int,int); 1053 void (*bzfree)(void *,void *); 1054 void *opaque; 1055} bz_stream; 1056 1057int BZ2_bzCompressInit ( bz_stream *strm, 1058 int blockSize100k, 1059 int verbosity, 1060 int workFactor ); 1061</programlisting> 1062 1063<para>Prepares for compression. The 1064<computeroutput>bz_stream</computeroutput> structure holds all 1065data pertaining to the compression activity. A 1066<computeroutput>bz_stream</computeroutput> structure should be 1067allocated and initialised prior to the call. The fields of 1068<computeroutput>bz_stream</computeroutput> comprise the entirety 1069of the user-visible data. <computeroutput>state</computeroutput> 1070is a pointer to the private data structures required for 1071compression.</para> 1072 1073<para>Custom memory allocators are supported, via fields 1074<computeroutput>bzalloc</computeroutput>, 1075<computeroutput>bzfree</computeroutput>, and 1076<computeroutput>opaque</computeroutput>. The value 1077<computeroutput>opaque</computeroutput> is passed to as the first 1078argument to all calls to <computeroutput>bzalloc</computeroutput> 1079and <computeroutput>bzfree</computeroutput>, but is otherwise 1080ignored by the library. The call <computeroutput>bzalloc ( 1081opaque, n, m )</computeroutput> is expected to return a pointer 1082<computeroutput>p</computeroutput> to <computeroutput>n * 1083m</computeroutput> bytes of memory, and <computeroutput>bzfree ( 1084opaque, p )</computeroutput> should free that memory.</para> 1085 1086<para>If you don't want to use a custom memory allocator, set 1087<computeroutput>bzalloc</computeroutput>, 1088<computeroutput>bzfree</computeroutput> and 1089<computeroutput>opaque</computeroutput> to 1090<computeroutput>NULL</computeroutput>, and the library will then 1091use the standard <computeroutput>malloc</computeroutput> / 1092<computeroutput>free</computeroutput> routines.</para> 1093 1094<para>Before calling 1095<computeroutput>BZ2_bzCompressInit</computeroutput>, fields 1096<computeroutput>bzalloc</computeroutput>, 1097<computeroutput>bzfree</computeroutput> and 1098<computeroutput>opaque</computeroutput> should be filled 1099appropriately, as just described. Upon return, the internal 1100state will have been allocated and initialised, and 1101<computeroutput>total_in_lo32</computeroutput>, 1102<computeroutput>total_in_hi32</computeroutput>, 1103<computeroutput>total_out_lo32</computeroutput> and 1104<computeroutput>total_out_hi32</computeroutput> will have been 1105set to zero. These four fields are used by the library to inform 1106the caller of the total amount of data passed into and out of the 1107library, respectively. You should not try to change them. As of 1108version 1.0, 64-bit counts are maintained, even on 32-bit 1109platforms, using the <computeroutput>_hi32</computeroutput> 1110fields to store the upper 32 bits of the count. So, for example, 1111the total amount of data in is <computeroutput>(total_in_hi32 1112<< 32) + total_in_lo32</computeroutput>.</para> 1113 1114<para>Parameter <computeroutput>blockSize100k</computeroutput> 1115specifies the block size to be used for compression. It should 1116be a value between 1 and 9 inclusive, and the actual block size 1117used is 100000 x this figure. 9 gives the best compression but 1118takes most memory.</para> 1119 1120<para>Parameter <computeroutput>verbosity</computeroutput> should 1121be set to a number between 0 and 4 inclusive. 0 is silent, and 1122greater numbers give increasingly verbose monitoring/debugging 1123output. If the library has been compiled with 1124<computeroutput>-DBZ_NO_STDIO</computeroutput>, no such output 1125will appear for any verbosity setting.</para> 1126 1127<para>Parameter <computeroutput>workFactor</computeroutput> 1128controls how the compression phase behaves when presented with 1129worst case, highly repetitive, input data. If compression runs 1130into difficulties caused by repetitive data, the library switches 1131from the standard sorting algorithm to a fallback algorithm. The 1132fallback is slower than the standard algorithm by perhaps a 1133factor of three, but always behaves reasonably, no matter how bad 1134the input.</para> 1135 1136<para>Lower values of <computeroutput>workFactor</computeroutput> 1137reduce the amount of effort the standard algorithm will expend 1138before resorting to the fallback. You should set this parameter 1139carefully; too low, and many inputs will be handled by the 1140fallback algorithm and so compress rather slowly, too high, and 1141your average-to-worst case compression times can become very 1142large. The default value of 30 gives reasonable behaviour over a 1143wide range of circumstances.</para> 1144 1145<para>Allowable values range from 0 to 250 inclusive. 0 is a 1146special case, equivalent to using the default value of 30.</para> 1147 1148<para>Note that the compressed output generated is the same 1149regardless of whether or not the fallback algorithm is 1150used.</para> 1151 1152<para>Be aware also that this parameter may disappear entirely in 1153future versions of the library. In principle it should be 1154possible to devise a good way to automatically choose which 1155algorithm to use. Such a mechanism would render the parameter 1156obsolete.</para> 1157 1158<para>Possible return values:</para> 1159 1160<programlisting> 1161BZ_CONFIG_ERROR 1162 if the library has been mis-compiled 1163BZ_PARAM_ERROR 1164 if strm is NULL 1165 or blockSize < 1 or blockSize > 9 1166 or verbosity < 0 or verbosity > 4 1167 or workFactor < 0 or workFactor > 250 1168BZ_MEM_ERROR 1169 if not enough memory is available 1170BZ_OK 1171 otherwise 1172</programlisting> 1173 1174<para>Allowable next actions:</para> 1175 1176<programlisting> 1177BZ2_bzCompress 1178 if BZ_OK is returned 1179 no specific action needed in case of error 1180</programlisting> 1181 1182</sect2> 1183 1184 1185<sect2 id="bzCompress" xreflabel="BZ2_bzCompress"> 1186<title><computeroutput>BZ2_bzCompress</computeroutput></title> 1187 1188<programlisting> 1189int BZ2_bzCompress ( bz_stream *strm, int action ); 1190</programlisting> 1191 1192<para>Provides more input and/or output buffer space for the 1193library. The caller maintains input and output buffers, and 1194calls <computeroutput>BZ2_bzCompress</computeroutput> to transfer 1195data between them.</para> 1196 1197<para>Before each call to 1198<computeroutput>BZ2_bzCompress</computeroutput>, 1199<computeroutput>next_in</computeroutput> should point at the data 1200to be compressed, and <computeroutput>avail_in</computeroutput> 1201should indicate how many bytes the library may read. 1202<computeroutput>BZ2_bzCompress</computeroutput> updates 1203<computeroutput>next_in</computeroutput>, 1204<computeroutput>avail_in</computeroutput> and 1205<computeroutput>total_in</computeroutput> to reflect the number 1206of bytes it has read.</para> 1207 1208<para>Similarly, <computeroutput>next_out</computeroutput> should 1209point to a buffer in which the compressed data is to be placed, 1210with <computeroutput>avail_out</computeroutput> indicating how 1211much output space is available. 1212<computeroutput>BZ2_bzCompress</computeroutput> updates 1213<computeroutput>next_out</computeroutput>, 1214<computeroutput>avail_out</computeroutput> and 1215<computeroutput>total_out</computeroutput> to reflect the number 1216of bytes output.</para> 1217 1218<para>You may provide and remove as little or as much data as you 1219like on each call of 1220<computeroutput>BZ2_bzCompress</computeroutput>. In the limit, 1221it is acceptable to supply and remove data one byte at a time, 1222although this would be terribly inefficient. You should always 1223ensure that at least one byte of output space is available at 1224each call.</para> 1225 1226<para>A second purpose of 1227<computeroutput>BZ2_bzCompress</computeroutput> is to request a 1228change of mode of the compressed stream.</para> 1229 1230<para>Conceptually, a compressed stream can be in one of four 1231states: IDLE, RUNNING, FLUSHING and FINISHING. Before 1232initialisation 1233(<computeroutput>BZ2_bzCompressInit</computeroutput>) and after 1234termination (<computeroutput>BZ2_bzCompressEnd</computeroutput>), 1235a stream is regarded as IDLE.</para> 1236 1237<para>Upon initialisation 1238(<computeroutput>BZ2_bzCompressInit</computeroutput>), the stream 1239is placed in the RUNNING state. Subsequent calls to 1240<computeroutput>BZ2_bzCo…
Large files files are truncated, but you can click here to view the full file