PageRenderTime 48ms CodeModel.GetById 16ms app.highlight 8ms RepoModel.GetById 1ms app.codeStats 1ms

/src/FreeImage/Source/LibJPEG/structure.txt

https://bitbucket.org/cabalistic/ogredeps/
Plain Text | 941 lines | 768 code | 173 blank | 0 comment | 0 complexity | dd2f5307b06a411b1b0fadcd675d612d MD5 | raw file

Large files files are truncated, but you can click here to view the full file

  1IJG JPEG LIBRARY:  SYSTEM ARCHITECTURE
  2
  3Copyright (C) 1991-2012, Thomas G. Lane, Guido Vollbeding.
  4This file is part of the Independent JPEG Group's software.
  5For conditions of distribution and use, see the accompanying README file.
  6
  7
  8This file provides an overview of the architecture of the IJG JPEG software;
  9that is, the functions of the various modules in the system and the interfaces
 10between modules.  For more precise details about any data structure or calling
 11convention, see the include files and comments in the source code.
 12
 13We assume that the reader is already somewhat familiar with the JPEG standard.
 14The README file includes references for learning about JPEG.  The file
 15libjpeg.txt describes the library from the viewpoint of an application
 16programmer using the library; it's best to read that file before this one.
 17Also, the file coderules.txt describes the coding style conventions we use.
 18
 19In this document, JPEG-specific terminology follows the JPEG standard:
 20  A "component" means a color channel, e.g., Red or Luminance.
 21  A "sample" is a single component value (i.e., one number in the image data).
 22  A "coefficient" is a frequency coefficient (a DCT transform output number).
 23  A "block" is an array of samples or coefficients.
 24  An "MCU" (minimum coded unit) is an interleaved set of blocks of size
 25	determined by the sampling factors, or a single block in a
 26	noninterleaved scan.
 27We do not use the terms "pixel" and "sample" interchangeably.  When we say
 28pixel, we mean an element of the full-size image, while a sample is an element
 29of the downsampled image.  Thus the number of samples may vary across
 30components while the number of pixels does not.  (This terminology is not used
 31rigorously throughout the code, but it is used in places where confusion would
 32otherwise result.)
 33
 34
 35*** System features ***
 36
 37The IJG distribution contains two parts:
 38  * A subroutine library for JPEG compression and decompression.
 39  * cjpeg/djpeg, two sample applications that use the library to transform
 40    JFIF JPEG files to and from several other image formats.
 41cjpeg/djpeg are of no great intellectual complexity: they merely add a simple
 42command-line user interface and I/O routines for several uncompressed image
 43formats.  This document concentrates on the library itself.
 44
 45We desire the library to be capable of supporting all JPEG baseline, extended
 46sequential, and progressive DCT processes.  The library does not support the
 47hierarchical or lossless processes defined in the standard.
 48
 49Within these limits, any set of compression parameters allowed by the JPEG
 50spec should be readable for decompression.  (We can be more restrictive about
 51what formats we can generate.)  Although the system design allows for all
 52parameter values, some uncommon settings are not yet implemented and may
 53never be; nonintegral sampling ratios are the prime example.  Furthermore,
 54we treat 8-bit vs. 12-bit data precision as a compile-time switch, not a
 55run-time option, because most machines can store 8-bit pixels much more
 56compactly than 12-bit.
 57
 58By itself, the library handles only interchange JPEG datastreams --- in
 59particular the widely used JFIF file format.  The library can be used by
 60surrounding code to process interchange or abbreviated JPEG datastreams that
 61are embedded in more complex file formats.  (For example, libtiff uses this
 62library to implement JPEG compression within the TIFF file format.)
 63
 64The library includes a substantial amount of code that is not covered by the
 65JPEG standard but is necessary for typical applications of JPEG.  These
 66functions preprocess the image before JPEG compression or postprocess it after
 67decompression.  They include colorspace conversion, downsampling/upsampling,
 68and color quantization.  This code can be omitted if not needed.
 69
 70A wide range of quality vs. speed tradeoffs are possible in JPEG processing,
 71and even more so in decompression postprocessing.  The decompression library
 72provides multiple implementations that cover most of the useful tradeoffs,
 73ranging from very-high-quality down to fast-preview operation.  On the
 74compression side we have generally not provided low-quality choices, since
 75compression is normally less time-critical.  It should be understood that the
 76low-quality modes may not meet the JPEG standard's accuracy requirements;
 77nonetheless, they are useful for viewers.
 78
 79
 80*** Portability issues ***
 81
 82Portability is an essential requirement for the library.  The key portability
 83issues that show up at the level of system architecture are:
 84
 851.  Memory usage.  We want the code to be able to run on PC-class machines
 86with limited memory.  Images should therefore be processed sequentially (in
 87strips), to avoid holding the whole image in memory at once.  Where a
 88full-image buffer is necessary, we should be able to use either virtual memory
 89or temporary files.
 90
 912.  Near/far pointer distinction.  To run efficiently on 80x86 machines, the
 92code should distinguish "small" objects (kept in near data space) from
 93"large" ones (kept in far data space).  This is an annoying restriction, but
 94fortunately it does not impact code quality for less brain-damaged machines,
 95and the source code clutter turns out to be minimal with sufficient use of
 96pointer typedefs.
 97
 983. Data precision.  We assume that "char" is at least 8 bits, "short" and
 99"int" at least 16, "long" at least 32.  The code will work fine with larger
100data sizes, although memory may be used inefficiently in some cases.  However,
101the JPEG compressed datastream must ultimately appear on external storage as a
102sequence of 8-bit bytes if it is to conform to the standard.  This may pose a
103problem on machines where char is wider than 8 bits.  The library represents
104compressed data as an array of values of typedef JOCTET.  If no data type
105exactly 8 bits wide is available, custom data source and data destination
106modules must be written to unpack and pack the chosen JOCTET datatype into
1078-bit external representation.
108
109
110*** System overview ***
111
112The compressor and decompressor are each divided into two main sections:
113the JPEG compressor or decompressor proper, and the preprocessing or
114postprocessing functions.  The interface between these two sections is the
115image data that the official JPEG spec regards as its input or output: this
116data is in the colorspace to be used for compression, and it is downsampled
117to the sampling factors to be used.  The preprocessing and postprocessing
118steps are responsible for converting a normal image representation to or from
119this form.  (Those few applications that want to deal with YCbCr downsampled
120data can skip the preprocessing or postprocessing step.)
121
122Looking more closely, the compressor library contains the following main
123elements:
124
125  Preprocessing:
126    * Color space conversion (e.g., RGB to YCbCr).
127    * Edge expansion and downsampling.  Optionally, this step can do simple
128      smoothing --- this is often helpful for low-quality source data.
129  JPEG proper:
130    * MCU assembly, DCT, quantization.
131    * Entropy coding (sequential or progressive, Huffman or arithmetic).
132
133In addition to these modules we need overall control, marker generation,
134and support code (memory management & error handling).  There is also a
135module responsible for physically writing the output data --- typically
136this is just an interface to fwrite(), but some applications may need to
137do something else with the data.
138
139The decompressor library contains the following main elements:
140
141  JPEG proper:
142    * Entropy decoding (sequential or progressive, Huffman or arithmetic).
143    * Dequantization, inverse DCT, MCU disassembly.
144  Postprocessing:
145    * Upsampling.  Optionally, this step may be able to do more general
146      rescaling of the image.
147    * Color space conversion (e.g., YCbCr to RGB).  This step may also
148      provide gamma adjustment [ currently it does not ].
149    * Optional color quantization (e.g., reduction to 256 colors).
150    * Optional color precision reduction (e.g., 24-bit to 15-bit color).
151      [This feature is not currently implemented.]
152
153We also need overall control, marker parsing, and a data source module.
154The support code (memory management & error handling) can be shared with
155the compression half of the library.
156
157There may be several implementations of each of these elements, particularly
158in the decompressor, where a wide range of speed/quality tradeoffs is very
159useful.  It must be understood that some of the best speedups involve
160merging adjacent steps in the pipeline.  For example, upsampling, color space
161conversion, and color quantization might all be done at once when using a
162low-quality ordered-dither technique.  The system architecture is designed to
163allow such merging where appropriate.
164
165
166Note: it is convenient to regard edge expansion (padding to block boundaries)
167as a preprocessing/postprocessing function, even though the JPEG spec includes
168it in compression/decompression.  We do this because downsampling/upsampling
169can be simplified a little if they work on padded data: it's not necessary to
170have special cases at the right and bottom edges.  Therefore the interface
171buffer is always an integral number of blocks wide and high, and we expect
172compression preprocessing to pad the source data properly.  Padding will occur
173only to the next block (N-sample) boundary.  In an interleaved-scan situation,
174additional dummy blocks may be used to fill out MCUs, but the MCU assembly and
175disassembly logic will create or discard these blocks internally.  (This is
176advantageous for speed reasons, since we avoid DCTing the dummy blocks.
177It also permits a small reduction in file size, because the compressor can
178choose dummy block contents so as to minimize their size in compressed form.
179Finally, it makes the interface buffer specification independent of whether
180the file is actually interleaved or not.)  Applications that wish to deal
181directly with the downsampled data must provide similar buffering and padding
182for odd-sized images.
183
184
185*** Poor man's object-oriented programming ***
186
187It should be clear by now that we have a lot of quasi-independent processing
188steps, many of which have several possible behaviors.  To avoid cluttering the
189code with lots of switch statements, we use a simple form of object-style
190programming to separate out the different possibilities.
191
192For example, two different color quantization algorithms could be implemented
193as two separate modules that present the same external interface; at runtime,
194the calling code will access the proper module indirectly through an "object".
195
196We can get the limited features we need while staying within portable C.
197The basic tool is a function pointer.  An "object" is just a struct
198containing one or more function pointer fields, each of which corresponds to
199a method name in real object-oriented languages.  During initialization we
200fill in the function pointers with references to whichever module we have
201determined we need to use in this run.  Then invocation of the module is done
202by indirecting through a function pointer; on most machines this is no more
203expensive than a switch statement, which would be the only other way of
204making the required run-time choice.  The really significant benefit, of
205course, is keeping the source code clean and well structured.
206
207We can also arrange to have private storage that varies between different
208implementations of the same kind of object.  We do this by making all the
209module-specific object structs be separately allocated entities, which will
210be accessed via pointers in the master compression or decompression struct.
211The "public" fields or methods for a given kind of object are specified by
212a commonly known struct.  But a module's initialization code can allocate
213a larger struct that contains the common struct as its first member, plus
214additional private fields.  With appropriate pointer casting, the module's
215internal functions can access these private fields.  (For a simple example,
216see jdatadst.c, which implements the external interface specified by struct
217jpeg_destination_mgr, but adds extra fields.)
218
219(Of course this would all be a lot easier if we were using C++, but we are
220not yet prepared to assume that everyone has a C++ compiler.)
221
222An important benefit of this scheme is that it is easy to provide multiple
223versions of any method, each tuned to a particular case.  While a lot of
224precalculation might be done to select an optimal implementation of a method,
225the cost per invocation is constant.  For example, the upsampling step might
226have a "generic" method, plus one or more "hardwired" methods for the most
227popular sampling factors; the hardwired methods would be faster because they'd
228use straight-line code instead of for-loops.  The cost to determine which
229method to use is paid only once, at startup, and the selection criteria are
230hidden from the callers of the method.
231
232This plan differs a little bit from usual object-oriented structures, in that
233only one instance of each object class will exist during execution.  The
234reason for having the class structure is that on different runs we may create
235different instances (choose to execute different modules).  You can think of
236the term "method" as denoting the common interface presented by a particular
237set of interchangeable functions, and "object" as denoting a group of related
238methods, or the total shared interface behavior of a group of modules.
239
240
241*** Overall control structure ***
242
243We previously mentioned the need for overall control logic in the compression
244and decompression libraries.  In IJG implementations prior to v5, overall
245control was mostly provided by "pipeline control" modules, which proved to be
246large, unwieldy, and hard to understand.  To improve the situation, the
247control logic has been subdivided into multiple modules.  The control modules
248consist of:
249
2501. Master control for module selection and initialization.  This has two
251responsibilities:
252
253   1A.  Startup initialization at the beginning of image processing.
254        The individual processing modules to be used in this run are selected
255        and given initialization calls.
256
257   1B.  Per-pass control.  This determines how many passes will be performed
258        and calls each active processing module to configure itself
259        appropriately at the beginning of each pass.  End-of-pass processing,
260	where necessary, is also invoked from the master control module.
261
262   Method selection is partially distributed, in that a particular processing
263   module may contain several possible implementations of a particular method,
264   which it will select among when given its initialization call.  The master
265   control code need only be concerned with decisions that affect more than
266   one module.
267 
2682. Data buffering control.  A separate control module exists for each
269   inter-processing-step data buffer.  This module is responsible for
270   invoking the processing steps that write or read that data buffer.
271
272Each buffer controller sees the world as follows:
273
274input data => processing step A => buffer => processing step B => output data
275                      |              |               |
276              ------------------ controller ------------------
277
278The controller knows the dataflow requirements of steps A and B: how much data
279they want to accept in one chunk and how much they output in one chunk.  Its
280function is to manage its buffer and call A and B at the proper times.
281
282A data buffer control module may itself be viewed as a processing step by a
283higher-level control module; thus the control modules form a binary tree with
284elementary processing steps at the leaves of the tree.
285
286The control modules are objects.  A considerable amount of flexibility can
287be had by replacing implementations of a control module.  For example:
288* Merging of adjacent steps in the pipeline is done by replacing a control
289  module and its pair of processing-step modules with a single processing-
290  step module.  (Hence the possible merges are determined by the tree of
291  control modules.)
292* In some processing modes, a given interstep buffer need only be a "strip"
293  buffer large enough to accommodate the desired data chunk sizes.  In other
294  modes, a full-image buffer is needed and several passes are required.
295  The control module determines which kind of buffer is used and manipulates
296  virtual array buffers as needed.  One or both processing steps may be
297  unaware of the multi-pass behavior.
298
299In theory, we might be able to make all of the data buffer controllers
300interchangeable and provide just one set of implementations for all.  In
301practice, each one contains considerable special-case processing for its
302particular job.  The buffer controller concept should be regarded as an
303overall system structuring principle, not as a complete description of the
304task performed by any one controller.
305
306
307*** Compression object structure ***
308
309Here is a sketch of the logical structure of the JPEG compression library:
310
311                                                 |-- Colorspace conversion
312                  |-- Preprocessing controller --|
313                  |                              |-- Downsampling
314Main controller --|
315                  |                            |-- Forward DCT, quantize
316                  |-- Coefficient controller --|
317                                               |-- Entropy encoding
318
319This sketch also describes the flow of control (subroutine calls) during
320typical image data processing.  Each of the components shown in the diagram is
321an "object" which may have several different implementations available.  One
322or more source code files contain the actual implementation(s) of each object.
323
324The objects shown above are:
325
326* Main controller: buffer controller for the subsampled-data buffer, which
327  holds the preprocessed input data.  This controller invokes preprocessing to
328  fill the subsampled-data buffer, and JPEG compression to empty it.  There is
329  usually no need for a full-image buffer here; a strip buffer is adequate.
330
331* Preprocessing controller: buffer controller for the downsampling input data
332  buffer, which lies between colorspace conversion and downsampling.  Note
333  that a unified conversion/downsampling module would probably replace this
334  controller entirely.
335
336* Colorspace conversion: converts application image data into the desired
337  JPEG color space; also changes the data from pixel-interleaved layout to
338  separate component planes.  Processes one pixel row at a time.
339
340* Downsampling: performs reduction of chroma components as required.
341  Optionally may perform pixel-level smoothing as well.  Processes a "row
342  group" at a time, where a row group is defined as Vmax pixel rows of each
343  component before downsampling, and Vk sample rows afterwards (remember Vk
344  differs across components).  Some downsampling or smoothing algorithms may
345  require context rows above and below the current row group; the
346  preprocessing controller is responsible for supplying these rows via proper
347  buffering.  The downsampler is responsible for edge expansion at the right
348  edge (i.e., extending each sample row to a multiple of N samples); but the
349  preprocessing controller is responsible for vertical edge expansion (i.e.,
350  duplicating the bottom sample row as needed to make a multiple of N rows).
351
352* Coefficient controller: buffer controller for the DCT-coefficient data.
353  This controller handles MCU assembly, including insertion of dummy DCT
354  blocks when needed at the right or bottom edge.  When performing
355  Huffman-code optimization or emitting a multiscan JPEG file, this
356  controller is responsible for buffering the full image.  The equivalent of
357  one fully interleaved MCU row of subsampled data is processed per call,
358  even when the JPEG file is noninterleaved.
359
360* Forward DCT and quantization: Perform DCT, quantize, and emit coefficients.
361  Works on one or more DCT blocks at a time.  (Note: the coefficients are now
362  emitted in normal array order, which the entropy encoder is expected to
363  convert to zigzag order as necessary.  Prior versions of the IJG code did
364  the conversion to zigzag order within the quantization step.)
365
366* Entropy encoding: Perform Huffman or arithmetic entropy coding and emit the
367  coded data to the data destination module.  Works on one MCU per call.
368  For progressive JPEG, the same DCT blocks are fed to the entropy coder
369  during each pass, and the coder must emit the appropriate subset of
370  coefficients.
371
372In addition to the above objects, the compression library includes these
373objects:
374
375* Master control: determines the number of passes required, controls overall
376  and per-pass initialization of the other modules.
377
378* Marker writing: generates JPEG markers (except for RSTn, which is emitted
379  by the entropy encoder when needed).
380
381* Data destination manager: writes the output JPEG datastream to its final
382  destination (e.g., a file).  The destination manager supplied with the
383  library knows how to write to a stdio stream or to a memory buffer;
384  for other behaviors, the surrounding application may provide its own
385  destination manager.
386
387* Memory manager: allocates and releases memory, controls virtual arrays
388  (with backing store management, where required).
389
390* Error handler: performs formatting and output of error and trace messages;
391  determines handling of nonfatal errors.  The surrounding application may
392  override some or all of this object's methods to change error handling.
393
394* Progress monitor: supports output of "percent-done" progress reports.
395  This object represents an optional callback to the surrounding application:
396  if wanted, it must be supplied by the application.
397
398The error handler, destination manager, and progress monitor objects are
399defined as separate objects in order to simplify application-specific
400customization of the JPEG library.  A surrounding application may override
401individual methods or supply its own all-new implementation of one of these
402objects.  The object interfaces for these objects are therefore treated as
403part of the application interface of the library, whereas the other objects
404are internal to the library.
405
406The error handler and memory manager are shared by JPEG compression and
407decompression; the progress monitor, if used, may be shared as well.
408
409
410*** Decompression object structure ***
411
412Here is a sketch of the logical structure of the JPEG decompression library:
413
414                                               |-- Entropy decoding
415                  |-- Coefficient controller --|
416                  |                            |-- Dequantize, Inverse DCT
417Main controller --|
418                  |                               |-- Upsampling
419                  |-- Postprocessing controller --|   |-- Colorspace conversion
420                                                  |-- Color quantization
421                                                  |-- Color precision reduction
422
423As before, this diagram also represents typical control flow.  The objects
424shown are:
425
426* Main controller: buffer controller for the subsampled-data buffer, which
427  holds the output of JPEG decompression proper.  This controller's primary
428  task is to feed the postprocessing procedure.  Some upsampling algorithms
429  may require context rows above and below the current row group; when this
430  is true, the main controller is responsible for managing its buffer so as
431  to make context rows available.  In the current design, the main buffer is
432  always a strip buffer; a full-image buffer is never required.
433
434* Coefficient controller: buffer controller for the DCT-coefficient data.
435  This controller handles MCU disassembly, including deletion of any dummy
436  DCT blocks at the right or bottom edge.  When reading a multiscan JPEG
437  file, this controller is responsible for buffering the full image.
438  (Buffering DCT coefficients, rather than samples, is necessary to support
439  progressive JPEG.)  The equivalent of one fully interleaved MCU row of
440  subsampled data is processed per call, even when the source JPEG file is
441  noninterleaved.
442
443* Entropy decoding: Read coded data from the data source module and perform
444  Huffman or arithmetic entropy decoding.  Works on one MCU per call.
445  For progressive JPEG decoding, the coefficient controller supplies the prior
446  coefficients of each MCU (initially all zeroes), which the entropy decoder
447  modifies in each scan.
448
449* Dequantization and inverse DCT: like it says.  Note that the coefficients
450  buffered by the coefficient controller have NOT been dequantized; we
451  merge dequantization and inverse DCT into a single step for speed reasons.
452  When scaled-down output is asked for, simplified DCT algorithms may be used
453  that need fewer coefficients and emit fewer samples per DCT block, not the
454  full 8x8.  Works on one DCT block at a time.
455
456* Postprocessing controller: buffer controller for the color quantization
457  input buffer, when quantization is in use.  (Without quantization, this
458  controller just calls the upsampler.)  For two-pass quantization, this
459  controller is responsible for buffering the full-image data.
460
461* Upsampling: restores chroma components to full size.  (May support more
462  general output rescaling, too.  Note that if undersized DCT outputs have
463  been emitted by the DCT module, this module must adjust so that properly
464  sized outputs are created.)  Works on one row group at a time.  This module
465  also calls the color conversion module, so its top level is effectively a
466  buffer controller for the upsampling->color conversion buffer.  However, in
467  all but the highest-quality operating modes, upsampling and color
468  conversion are likely to be merged into a single step.
469
470* Colorspace conversion: convert from JPEG color space to output color space,
471  and change data layout from separate component planes to pixel-interleaved.
472  Works on one pixel row at a time.
473
474* Color quantization: reduce the data to colormapped form, using either an
475  externally specified colormap or an internally generated one.  This module
476  is not used for full-color output.  Works on one pixel row at a time; may
477  require two passes to generate a color map.  Note that the output will
478  always be a single component representing colormap indexes.  In the current
479  design, the output values are JSAMPLEs, so an 8-bit compilation cannot
480  quantize to more than 256 colors.  This is unlikely to be a problem in
481  practice.
482
483* Color reduction: this module handles color precision reduction, e.g.,
484  generating 15-bit color (5 bits/primary) from JPEG's 24-bit output.
485  Not quite clear yet how this should be handled... should we merge it with
486  colorspace conversion???
487
488Note that some high-speed operating modes might condense the entire
489postprocessing sequence to a single module (upsample, color convert, and
490quantize in one step).
491
492In addition to the above objects, the decompression library includes these
493objects:
494
495* Master control: determines the number of passes required, controls overall
496  and per-pass initialization of the other modules.  This is subdivided into
497  input and output control: jdinput.c controls only input-side processing,
498  while jdmaster.c handles overall initialization and output-side control.
499
500* Marker reading: decodes JPEG markers (except for RSTn).
501
502* Data source manager: supplies the input JPEG datastream.  The source
503  manager supplied with the library knows how to read from a stdio stream
504  or from a memory buffer;  for other behaviors, the surrounding application
505  may provide its own source manager.
506
507* Memory manager: same as for compression library.
508
509* Error handler: same as for compression library.
510
511* Progress monitor: same as for compression library.
512
513As with compression, the data source manager, error handler, and progress
514monitor are candidates for replacement by a surrounding application.
515
516
517*** Decompression input and output separation ***
518
519To support efficient incremental display of progressive JPEG files, the
520decompressor is divided into two sections that can run independently:
521
5221. Data input includes marker parsing, entropy decoding, and input into the
523   coefficient controller's DCT coefficient buffer.  Note that this
524   processing is relatively cheap and fast.
525
5262. Data output reads from the DCT coefficient buffer and performs the IDCT
527   and all postprocessing steps.
528
529For a progressive JPEG file, the data input processing is allowed to get
530arbitrarily far ahead of the data output processing.  (This occurs only
531if the application calls jpeg_consume_input(); otherwise input and output
532run in lockstep, since the input section is called only when the output
533section needs more data.)  In this way the application can avoid making
534extra display passes when data is arriving faster than the display pass
535can run.  Furthermore, it is possible to abort an output pass without
536losing anything, since the coefficient buffer is read-only as far as the
537output section is concerned.  See libjpeg.txt for more detail.
538
539A full-image coefficient array is only created if the JPEG file has multiple
540scans (or if the application specifies buffered-image mode anyway).  When
541reading a single-scan file, the coefficient controller normally creates only
542a one-MCU buffer, so input and output processing must run in lockstep in this
543case.  jpeg_consume_input() is effectively a no-op in this situation.
544
545The main impact of dividing the decompressor in this fashion is that we must
546be very careful with shared variables in the cinfo data structure.  Each
547variable that can change during the course of decompression must be
548classified as belonging to data input or data output, and each section must
549look only at its own variables.  For example, the data output section may not
550depend on any of the variables that describe the current scan in the JPEG
551file, because these may change as the data input section advances into a new
552scan.
553
554The progress monitor is (somewhat arbitrarily) defined to treat input of the
555file as one pass when buffered-image mode is not used, and to ignore data
556input work completely when buffered-image mode is used.  Note that the
557library has no reliable way to predict the number of passes when dealing
558with a progressive JPEG file, nor can it predict the number of output passes
559in buffered-image mode.  So the work estimate is inherently bogus anyway.
560
561No comparable division is currently made in the compression library, because
562there isn't any real need for it.
563
564
565*** Data formats ***
566
567Arrays of pixel sample values use the following data structure:
568
569    typedef something JSAMPLE;		a pixel component value, 0..MAXJSAMPLE
570    typedef JSAMPLE *JSAMPROW;		ptr to a row of samples
571    typedef JSAMPROW *JSAMPARRAY;	ptr to a list of rows
572    typedef JSAMPARRAY *JSAMPIMAGE;	ptr to a list of color-component arrays
573
574The basic element type JSAMPLE will typically be one of unsigned char,
575(signed) char, or short.  Short will be used if samples wider than 8 bits are
576to be supported (this is a compile-time option).  Otherwise, unsigned char is
577used if possible.  If the compiler only supports signed chars, then it is
578necessary to mask off the value when reading.  Thus, all reads of JSAMPLE
579values must be coded as "GETJSAMPLE(value)", where the macro will be defined
580as "((value) & 0xFF)" on signed-char machines and "((int) (value))" elsewhere.
581
582With these conventions, JSAMPLE values can be assumed to be >= 0.  This helps
583simplify correct rounding during downsampling, etc.  The JPEG standard's
584specification that sample values run from -128..127 is accommodated by
585subtracting 128 from the sample value in the DCT step.  Similarly, during
586decompression the output of the IDCT step will be immediately shifted back to
5870..255.  (NB: different values are required when 12-bit samples are in use.
588The code is written in terms of MAXJSAMPLE and CENTERJSAMPLE, which will be
589defined as 255 and 128 respectively in an 8-bit implementation, and as 4095
590and 2048 in a 12-bit implementation.)
591
592We use a pointer per row, rather than a two-dimensional JSAMPLE array.  This
593choice costs only a small amount of memory and has several benefits:
594* Code using the data structure doesn't need to know the allocated width of
595  the rows.  This simplifies edge expansion/compression, since we can work
596  in an array that's wider than the logical picture width.
597* Indexing doesn't require multiplication; this is a performance win on many
598  machines.
599* Arrays with more than 64K total elements can be supported even on machines
600  where malloc() cannot allocate chunks larger than 64K.
601* The rows forming a component array may be allocated at different times
602  without extra copying.  This trick allows some speedups in smoothing steps
603  that need access to the previous and next rows.
604
605Note that each color component is stored in a separate array; we don't use the
606traditional layout in which the components of a pixel are stored together.
607This simplifies coding of modules that work on each component independently,
608because they don't need to know how many components there are.  Furthermore,
609we can read or write each component to a temporary file independently, which
610is helpful when dealing with noninterleaved JPEG files.
611
612In general, a specific sample value is accessed by code such as
613	GETJSAMPLE(image[colorcomponent][row][col])
614where col is measured from the image left edge, but row is measured from the
615first sample row currently in memory.  Either of the first two indexings can
616be precomputed by copying the relevant pointer.
617
618
619Since most image-processing applications prefer to work on images in which
620the components of a pixel are stored together, the data passed to or from the
621surrounding application uses the traditional convention: a single pixel is
622represented by N consecutive JSAMPLE values, and an image row is an array of
623(# of color components)*(image width) JSAMPLEs.  One or more rows of data can
624be represented by a pointer of type JSAMPARRAY in this scheme.  This scheme is
625converted to component-wise storage inside the JPEG library.  (Applications
626that want to skip JPEG preprocessing or postprocessing will have to contend
627with component-wise storage.)
628
629
630Arrays of DCT-coefficient values use the following data structure:
631
632    typedef short JCOEF;		a 16-bit signed integer
633    typedef JCOEF JBLOCK[DCTSIZE2];	an 8x8 block of coefficients
634    typedef JBLOCK *JBLOCKROW;		ptr to one horizontal row of 8x8 blocks
635    typedef JBLOCKROW *JBLOCKARRAY;	ptr to a list of such rows
636    typedef JBLOCKARRAY *JBLOCKIMAGE;	ptr to a list of color component arrays
637
638The underlying type is at least a 16-bit signed integer; while "short" is big
639enough on all machines of interest, on some machines it is preferable to use
640"int" for speed reasons, despite the storage cost.  Coefficients are grouped
641into 8x8 blocks (but we always use #defines DCTSIZE and DCTSIZE2 rather than
642"8" and "64").
643
644The contents of a coefficient block may be in either "natural" or zigzagged
645order, and may be true values or divided by the quantization coefficients,
646depending on where the block is in the processing pipeline.  In the current
647library, coefficient blocks are kept in natural order everywhere; the entropy
648codecs zigzag or dezigzag the data as it is written or read.  The blocks
649contain quantized coefficients everywhere outside the DCT/IDCT subsystems.
650(This latter decision may need to be revisited to support variable
651quantization a la JPEG Part 3.)
652
653Notice that the allocation unit is now a row of 8x8 coefficient blocks,
654corresponding to N rows of samples.  Otherwise the structure is much the same
655as for samples, and for the same reasons.
656
657On machines where malloc() can't handle a request bigger than 64Kb, this data
658structure limits us to rows of less than 512 JBLOCKs, or a picture width of
6594000+ pixels.  This seems an acceptable restriction.
660
661
662On 80x86 machines, the bottom-level pointer types (JSAMPROW and JBLOCKROW)
663must be declared as "far" pointers, but the upper levels can be "near"
664(implying that the pointer lists are allocated in the DS segment).
665We use a #define symbol FAR, which expands to the "far" keyword when
666compiling on 80x86 machines and to nothing elsewhere.
667
668
669*** Suspendable processing ***
670
671In some applications it is desirable to use the JPEG library as an
672incremental, memory-to-memory filter.  In this situation the data source or
673destination may be a limited-size buffer, and we can't rely on being able to
674empty or refill the buffer at arbitrary times.  Instead the application would
675like to have control return from the library at buffer overflow/underrun, and
676then resume compression or decompression at a later time.
677
678This scenario is supported for simple cases.  (For anything more complex, we
679recommend that the application "bite the bullet" and develop real multitasking
680capability.)  The libjpeg.txt file goes into more detail about the usage and
681limitations of this capability; here we address the implications for library
682structure.
683
684The essence of the problem is that the entropy codec (coder or decoder) must
685be prepared to stop at arbitrary times.  In turn, the controllers that call
686the entropy codec must be able to stop before having produced or consumed all
687the data that they normally would handle in one call.  That part is reasonably
688straightforward: we make the controller call interfaces include "progress
689counters" which indicate the number of data chunks successfully processed, and
690we require callers to test the counter rather than just assume all of the data
691was processed.
692
693Rather than trying to restart at an arbitrary point, the current Huffman
694codecs are designed to restart at the beginning of the current MCU after a
695suspension due to buffer overflow/underrun.  At the start of each call, the
696codec's internal state is loaded from permanent storage (in the JPEG object
697structures) into local variables.  On successful completion of the MCU, the
698permanent state is updated.  (This copying is not very expensive, and may even
699lead to *improved* performance if the local variables can be registerized.)
700If a suspension occurs, the codec simply returns without updating the state,
701thus effectively reverting to the start of the MCU.  Note that this implies
702leaving some data unprocessed in the source/destination buffer (ie, the
703compressed partial MCU).  The data source/destination module interfaces are
704specified so as to make this possible.  This also implies that the data buffer
705must be large enough to hold a worst-case compressed MCU; a couple thousand
706bytes should be enough.
707
708In a successive-approximation AC refinement scan, the progressive Huffman
709decoder has to be able to undo assignments of newly nonzero coefficients if it
710suspends before the MCU is complete, since decoding requires distinguishing
711previously-zero and previously-nonzero coefficients.  This is a bit tedious
712but probably won't have much effect on performance.  Other variants of Huffman
713decoding need not worry about this, since they will just store the same values
714again if forced to repeat the MCU.
715
716This approach would probably not work for an arithmetic codec, since its
717modifiable state is quite large and couldn't be copied cheaply.  Instead it
718would have to suspend and resume exactly at the point of the buffer end.
719
720The JPEG marker reader is designed to cope with suspension at an arbitrary
721point.  It does so by backing up to the start of the marker parameter segment,
722so the data buffer must be big enough to hold the largest marker of interest.
723Again, a couple KB should be adequate.  (A special "skip" convention is used
724to bypass COM and APPn markers, so these can be larger than the buffer size
725without causing problems; otherwise a 64K buffer would be needed in the worst
726case.)
727
728The JPEG marker writer currently does *not* cope with suspension.
729We feel that this is not necessary; it is much easier simply to require
730the application to ensure there is enough buffer space before starting.  (An
731empty 2K buffer is more than sufficient for the header markers; and ensuring
732there are a dozen or two bytes available before calling jpeg_finish_compress()
733will suffice for the trailer.)  This would not work for writing multi-scan
734JPEG files, but we simply do not intend to support that capability with
735suspension.
736
737
738*** Memory manager services ***
739
740The JPEG library's memory manager controls allocation and deallocation of
741memory, and it manages large "virtual" data arrays on machines where the
742operating system does not provide virtual memory.  Note that the same
743memory manager serves both compression and decompression operations.
744
745In all cases, allocated objects are tied to a particular compression or
746decompression master record, and they will be released when that master
747record is destroyed.
748
749The memory manager does not provide explicit deallocation of objects.
750Instead, objects are created in "pools" of free storage, and a whole pool
751can be freed at once.  This approach helps prevent storage-leak bugs, and
752it speeds up operations whenever malloc/free are slow (as they often are).
753The pools can be regarded as lifetime identifiers for objects.  Two
754pools/lifetimes are defined:
755  * JPOOL_PERMANENT	lasts until master record is destroyed
756  * JPOOL_IMAGE		lasts until done with image (JPEG datastream)
757Permanent lifetime is used for parameters and tables that should be carried
758across from one datastream to another; this includes all application-visible
759parameters.  Image lifetime is used for everything else.  (A third lifetime,
760JPOOL_PASS = one processing pass, was originally planned.  However it was
761dropped as not being worthwhile.  The actual usage patterns are such that the
762peak memory usage would be about the same anyway; and having per-pass storage
763substantially complicates the virtual memory allocation rules --- see below.)
764
765The memory manager deals with three kinds of object:
7661. "Small" objects.  Typically these require no more than 10K-20K total.
7672. "Large" objects.  These may require tens to hundreds of K depending on
768   image size.  Semantically they behave the same as small objects, but we
769   distinguish them for two reasons:
770     * On MS-DOS machines, large objects are referenced by FAR pointers,
771       small objects by NEAR pointers.
772     * Pool allocation heuristics may differ for large and small objects.
773   Note that individual "large" objects cannot exceed the size allowed by
774   type size_t, which may be 64K or less on some machines.
7753. "Virtual" objects.  These are large 2-D arrays of JSAMPLEs or JBLOCKs
776   (typically large enough for the entire image being processed).  The
777   memory manager provides stripwise access to these arrays.  On machines
778   without virtual memory, the rest of the array may be swapped out to a
779   temporary file.
780
781(Note: JSAMPARRAY and JBLOCKARRAY data structures are a combination of large
782objects for the data proper and small objects for the row pointers.  For
783convenience and speed, the memory manager provides single routines to create
784these structures.  Similarly, virtual arrays include a small control block
785and a JSAMPARRAY or JBLOCKARRAY working buffer, all created with one call.)
786
787In the present implementation, virtual arrays are only permitted to have image
788lifespan.  (Permanent lifespan would not be reasonable, and pass lifespan is
789not very useful since a virtual array's raison d'etre is to store data for
790multiple passes through the image.)  We also expect that only "small" objects
791will be given permanent lifespan, though this restriction is not required by
792the memory manager.
793
794In a non-virtual-memory machine, some performance benefit can be gained by
795making the in-memory buffers for virtual arrays be as large as possible.
796(For small images, the buffers might fit entirely in memory, so blind
797swapping would be very wasteful.)  The memory manager will adjust the height
798of the buffers to fit within a prespecified maximum memory usage.  In order
799to do this in a reasonably optimal fashion, the manager needs to allocate all
800of the virtual arrays at once.  Therefore, there isn't a one-step allocation
801routine for virtual arrays; instead, there is a "request" routine that simply
802allocates the control block, and a "realize" routine (called just once) that
803determines space allocation and creates all of the actual buffers.  The
804realize routine must allow for space occupied by non-virtual large objects.
805(We don't bother to factor in the space needed for small objects, on the
806grounds that it isn't worth the trouble.)
807
808To support all this, we establish the following protocol for doing business
809with the memory manager:
810  1. Modules must request virtual arrays (which may have only image lifespan)
811     during the initial setup phase, i.e., in their jinit_xxx routines.
812  2. All "large" objects (including JSAMPARRAYs and JBLOCKARRAYs) must also be
813     allocated during initial setup.
814  3. realize_virt_arrays will be called at the completion of initial setup.
815     The above conventions ensure that sufficient information is available
816     for it to choose a good size for virtual array buffers.
817Small objects of any lifespan may be allocated at any time.  We expect that
818the total space used for small objects will be small enough to be negligible
819in the realize_virt_arrays computation.
820
821In a virtual-memory machine, we simply pretend that the available space is
822infinite, thus causing realize_virt_arrays to decide that it can allocate all
823the virtual arrays as full-size in-memory buffers.  The overhead of the
824virtual-array access protocol is very small when no swapping occurs.
825
826A virtual array can be specified to be "pre-zeroed"; when this flag is set,
827never-yet-written sections of the array are set to zero before being made
828available to the caller.  If this flag is not set, never-written sections
829of the array contain garbage.  (This feature exists primarily because the
830equivalent logic would otherwise be needed in jdcoefct.c for progressive
831JPEG mode; we may as well make it available for possible other uses.)
832
833The first write pass on a virtual array is required to occur in top-to-bottom
834order; read passes, as well as any write passes after the first one, may
835access the array in any order.  This restriction exists partly to simplify
836the virtual array control logic, and partly because some file systems may not
837support seeking beyond the current end-of-file in a temporary file.  The main
838implication of this restriction is that rearrangement of rows (such as
839converting top-to-bottom data order to bottom-to-top) must be handled while
840reading data out of the virtual array, not while putting it in.
841
842
843*** Memory manager internal structure ***
844
845To isolate system dependencies as much as possible, we have broken the
846memory manager into two parts.  There is a reasonably system-independent
847"front end" (jmemmgr.c) and a "back end" that contains only the code
848likely to change across systems.  All of the memory management methods
849outlined above are implemented by the front end.  The back end provides
850the following routines for use by the front end (none of these routines
851are known to the rest of the JPEG code):
852
853jpeg_mem_init, jpeg_mem_term	system-dependent initialization/shutdown
854
855jpeg_get_small, jpeg_free_small	interface to malloc and free library routines
856				(or their equivalents)
857
858jpeg_get_large, jpeg_free_large	interface to FAR malloc/free in MSDOS machines;
859				else usually the same as
860				jpeg_get_small/jpeg_free_small
861
862jpeg_mem_available		estimate available memory
863
864jpeg_open_backing_store		create a backing-store object
865
866read_backing_store,		manipulate a backing-store object
867write_backing_store,
868close_backing_store
869
870On some systems there will be more than one type of backing-store object
871(specifically, in MS-DOS a backing store file might be an area of extended
872memory as well as a disk file).  jpeg_open_backing_store is responsible for
873choosing how to implement a given object.  The read/write/close routines
874are method pointers in the structure that describes a given object; this
875lets them be different for different object types.
876
877It may be necessary to ensure that backing store objects are explicitly
878released upon abnormal program termination.  For example, MS-DOS won't free
879extended memory by itself.  To support this, we will expect the main program
880or surrounding application to arrange to call self_destruct (typically via
881jpeg_destroy) upon abnormal termination.  This may require a SIGINT signal
882handler or equivalent.  We don't want to have the back end module install its
883own signal handler, because that would pre-empt the surrounding application's
884ability to control signal handling.
885
886The IJG distribution includes several memory manager back end implementations.
887Usually the same back end should be suitable for all applications on a given
888system, but it is possible for an application to supply its own back end at
889need.
890
891
892*** Implications of DNL marker ***
893
894Some JPEG files may use a DNL marker to postpone definition of the image
895height (this would be useful for a fax-like scanner's output, for instance).
896In these files the SOF marker claims the image height is 0, and you only
897find out the true image height at the end of the first scan.
898
899We could read these files as follows:
9001. Upon seeing zero image height, replace it by 65535 (the maximum allowed).
9012. When the DNL is found, update the image height in the global image
902   descriptor.
903This implies that control modules must avoid making copies of the image
904height, and must re-test for termination after each MCU row.  This would
905be easy enough to do.
906
907In cases where image-size data structures are allocated, this approach will
908result in very inefficient use of virtual memory or much-larger-than-necessary
909temporary files.  This seems acceptable for something that probably won't be a
910mainstream usage.  People might have to forgo use of memory-hogging options
911(such as two-pass color quantization or noninterleaved JPEG files) if they
912want efficient conversion of such files.  (One could improve efficiency by
913demanding a user-supplied upper bound for the height, less than 65536;…

Large files files are truncated, but you can click here to view the full file