/Doc/library/shelve.rst

http://unladen-swallow.googlecode.com/ · ReStructuredText · 194 lines · 137 code · 57 blank · 0 comment · 0 complexity · e9a435b851533038b36be0e0245cbb81 MD5 · raw file

  1. :mod:`shelve` --- Python object persistence
  2. ===========================================
  3. .. module:: shelve
  4. :synopsis: Python object persistence.
  5. .. index:: module: pickle
  6. A "shelf" is a persistent, dictionary-like object. The difference with "dbm"
  7. databases is that the values (not the keys!) in a shelf can be essentially
  8. arbitrary Python objects --- anything that the :mod:`pickle` module can handle.
  9. This includes most class instances, recursive data types, and objects containing
  10. lots of shared sub-objects. The keys are ordinary strings.
  11. .. function:: open(filename[, flag='c'[, protocol=None[, writeback=False]]])
  12. Open a persistent dictionary. The filename specified is the base filename for
  13. the underlying database. As a side-effect, an extension may be added to the
  14. filename and more than one file may be created. By default, the underlying
  15. database file is opened for reading and writing. The optional *flag* parameter
  16. has the same interpretation as the *flag* parameter of :func:`anydbm.open`.
  17. By default, version 0 pickles are used to serialize values. The version of the
  18. pickle protocol can be specified with the *protocol* parameter.
  19. .. versionchanged:: 2.3
  20. The *protocol* parameter was added.
  21. Because of Python semantics, a shelf cannot know when a mutable
  22. persistent-dictionary entry is modified. By default modified objects are
  23. written only when assigned to the shelf (see :ref:`shelve-example`). If
  24. the optional *writeback* parameter is set to *True*, all entries accessed
  25. are cached in memory, and written back at close time; this can make it
  26. handier to mutate mutable entries in the persistent dictionary, but, if
  27. many entries are accessed, it can consume vast amounts of memory for the
  28. cache, and it can make the close operation very slow since all accessed
  29. entries are written back (there is no way to determine which accessed
  30. entries are mutable, nor which ones were actually mutated).
  31. Shelf objects support all methods supported by dictionaries. This eases the
  32. transition from dictionary based scripts to those requiring persistent storage.
  33. One additional method is supported:
  34. .. method:: Shelf.sync()
  35. Write back all entries in the cache if the shelf was opened with *writeback* set
  36. to *True*. Also empty the cache and synchronize the persistent dictionary on
  37. disk, if feasible. This is called automatically when the shelf is closed with
  38. :meth:`close`.
  39. .. seealso::
  40. `Persistent dictionary recipe <http://code.activestate.com/recipes/576642/>`_
  41. with widely supported storage formats and having the speed of native
  42. dictionaries.
  43. Restrictions
  44. ------------
  45. .. index::
  46. module: dbm
  47. module: gdbm
  48. module: bsddb
  49. * The choice of which database package will be used (such as :mod:`dbm`,
  50. :mod:`gdbm` or :mod:`bsddb`) depends on which interface is available. Therefore
  51. it is not safe to open the database directly using :mod:`dbm`. The database is
  52. also (unfortunately) subject to the limitations of :mod:`dbm`, if it is used ---
  53. this means that (the pickled representation of) the objects stored in the
  54. database should be fairly small, and in rare cases key collisions may cause the
  55. database to refuse updates.
  56. * Depending on the implementation, closing a persistent dictionary may or may
  57. not be necessary to flush changes to disk. The :meth:`__del__` method of the
  58. :class:`Shelf` class calls the :meth:`close` method, so the programmer generally
  59. need not do this explicitly.
  60. * The :mod:`shelve` module does not support *concurrent* read/write access to
  61. shelved objects. (Multiple simultaneous read accesses are safe.) When a
  62. program has a shelf open for writing, no other program should have it open for
  63. reading or writing. Unix file locking can be used to solve this, but this
  64. differs across Unix versions and requires knowledge about the database
  65. implementation used.
  66. .. class:: Shelf(dict[, protocol=None[, writeback=False]])
  67. A subclass of :class:`UserDict.DictMixin` which stores pickled values in the
  68. *dict* object.
  69. By default, version 0 pickles are used to serialize values. The version of the
  70. pickle protocol can be specified with the *protocol* parameter. See the
  71. :mod:`pickle` documentation for a discussion of the pickle protocols.
  72. .. versionchanged:: 2.3
  73. The *protocol* parameter was added.
  74. If the *writeback* parameter is ``True``, the object will hold a cache of all
  75. entries accessed and write them back to the *dict* at sync and close times.
  76. This allows natural operations on mutable entries, but can consume much more
  77. memory and make sync and close take a long time.
  78. .. class:: BsdDbShelf(dict[, protocol=None[, writeback=False]])
  79. A subclass of :class:`Shelf` which exposes :meth:`first`, :meth:`next`,
  80. :meth:`previous`, :meth:`last` and :meth:`set_location` which are available in
  81. the :mod:`bsddb` module but not in other database modules. The *dict* object
  82. passed to the constructor must support those methods. This is generally
  83. accomplished by calling one of :func:`bsddb.hashopen`, :func:`bsddb.btopen` or
  84. :func:`bsddb.rnopen`. The optional *protocol* and *writeback* parameters have
  85. the same interpretation as for the :class:`Shelf` class.
  86. .. class:: DbfilenameShelf(filename[, flag='c'[, protocol=None[, writeback=False]]])
  87. A subclass of :class:`Shelf` which accepts a *filename* instead of a dict-like
  88. object. The underlying file will be opened using :func:`anydbm.open`. By
  89. default, the file will be created and opened for both read and write. The
  90. optional *flag* parameter has the same interpretation as for the :func:`open`
  91. function. The optional *protocol* and *writeback* parameters have the same
  92. interpretation as for the :class:`Shelf` class.
  93. .. _shelve-example:
  94. Example
  95. -------
  96. To summarize the interface (``key`` is a string, ``data`` is an arbitrary
  97. object)::
  98. import shelve
  99. d = shelve.open(filename) # open -- file may get suffix added by low-level
  100. # library
  101. d[key] = data # store data at key (overwrites old data if
  102. # using an existing key)
  103. data = d[key] # retrieve a COPY of data at key (raise KeyError if no
  104. # such key)
  105. del d[key] # delete data stored at key (raises KeyError
  106. # if no such key)
  107. flag = d.has_key(key) # true if the key exists
  108. klist = d.keys() # a list of all existing keys (slow!)
  109. # as d was opened WITHOUT writeback=True, beware:
  110. d['xx'] = range(4) # this works as expected, but...
  111. d['xx'].append(5) # *this doesn't!* -- d['xx'] is STILL range(4)!
  112. # having opened d without writeback=True, you need to code carefully:
  113. temp = d['xx'] # extracts the copy
  114. temp.append(5) # mutates the copy
  115. d['xx'] = temp # stores the copy right back, to persist it
  116. # or, d=shelve.open(filename,writeback=True) would let you just code
  117. # d['xx'].append(5) and have it work as expected, BUT it would also
  118. # consume more memory and make the d.close() operation slower.
  119. d.close() # close it
  120. .. seealso::
  121. Module :mod:`anydbm`
  122. Generic interface to ``dbm``\ -style databases.
  123. Module :mod:`bsddb`
  124. BSD ``db`` database interface.
  125. Module :mod:`dbhash`
  126. Thin layer around the :mod:`bsddb` which provides an :func:`open` function like
  127. the other database modules.
  128. Module :mod:`dbm`
  129. Standard Unix database interface.
  130. Module :mod:`dumbdbm`
  131. Portable implementation of the ``dbm`` interface.
  132. Module :mod:`gdbm`
  133. GNU database interface, based on the ``dbm`` interface.
  134. Module :mod:`pickle`
  135. Object serialization used by :mod:`shelve`.
  136. Module :mod:`cPickle`
  137. High-performance version of :mod:`pickle`.