/Doc/library/weakref.rst

  1
  2:mod:`weakref` --- Weak references
  3==================================
  4
  5.. module:: weakref
  6   :synopsis: Support for weak references and weak dictionaries.
  7.. moduleauthor:: Fred L. Drake, Jr. <fdrake@acm.org>
  8.. moduleauthor:: Neil Schemenauer <nas@arctrix.com>
  9.. moduleauthor:: Martin von Lรถwis <martin@loewis.home.cs.tu-berlin.de>
 10.. sectionauthor:: Fred L. Drake, Jr. <fdrake@acm.org>
 11
 12
 13.. versionadded:: 2.1
 14
 15The :mod:`weakref` module allows the Python programmer to create :dfn:`weak
 16references` to objects.
 17
 18.. When making changes to the examples in this file, be sure to update
 19   Lib/test/test_weakref.py::libreftest too!
 20
 21In the following, the term :dfn:`referent` means the object which is referred to
 22by a weak reference.
 23
 24A weak reference to an object is not enough to keep the object alive: when the
 25only remaining references to a referent are weak references,
 26:term:`garbage collection` is free to destroy the referent and reuse its memory
 27for something else.  A primary use for weak references is to implement caches or
 28mappings holding large objects, where it's desired that a large object not be
 29kept alive solely because it appears in a cache or mapping.
 30
 31For example, if you have a number of large binary image objects, you may wish to
 32associate a name with each.  If you used a Python dictionary to map names to
 33images, or images to names, the image objects would remain alive just because
 34they appeared as values or keys in the dictionaries.  The
 35:class:`WeakKeyDictionary` and :class:`WeakValueDictionary` classes supplied by
 36the :mod:`weakref` module are an alternative, using weak references to construct
 37mappings that don't keep objects alive solely because they appear in the mapping
 38objects.  If, for example, an image object is a value in a
 39:class:`WeakValueDictionary`, then when the last remaining references to that
 40image object are the weak references held by weak mappings, garbage collection
 41can reclaim the object, and its corresponding entries in weak mappings are
 42simply deleted.
 43
 44:class:`WeakKeyDictionary` and :class:`WeakValueDictionary` use weak references
 45in their implementation, setting up callback functions on the weak references
 46that notify the weak dictionaries when a key or value has been reclaimed by
 47garbage collection.  Most programs should find that using one of these weak
 48dictionary types is all they need -- it's not usually necessary to create your
 49own weak references directly.  The low-level machinery used by the weak
 50dictionary implementations is exposed by the :mod:`weakref` module for the
 51benefit of advanced uses.
 52
 53.. note::
 54
 55   Weak references to an object are cleared before the object's :meth:`__del__`
 56   is called, to ensure that the weak reference callback (if any) finds the
 57   object still alive.
 58
 59Not all objects can be weakly referenced; those objects which can include class
 60instances, functions written in Python (but not in C), methods (both bound and
 61unbound), sets, frozensets, file objects, :term:`generator`\s, type objects,
 62:class:`DBcursor` objects from the :mod:`bsddb` module, sockets, arrays, deques,
 63and regular expression pattern objects.
 64
 65.. versionchanged:: 2.4
 66   Added support for files, sockets, arrays, and patterns.
 67
 68Several builtin types such as :class:`list` and :class:`dict` do not directly
 69support weak references but can add support through subclassing::
 70
 71   class Dict(dict):
 72       pass
 73
 74   obj = Dict(red=1, green=2, blue=3)   # this object is weak referenceable
 75
 76Extension types can easily be made to support weak references; see
 77:ref:`weakref-support`.
 78
 79
 80.. class:: ref(object[, callback])
 81
 82   Return a weak reference to *object*.  The original object can be retrieved by
 83   calling the reference object if the referent is still alive; if the referent is
 84   no longer alive, calling the reference object will cause :const:`None` to be
 85   returned.  If *callback* is provided and not :const:`None`, and the returned
 86   weakref object is still alive, the callback will be called when the object is
 87   about to be finalized; the weak reference object will be passed as the only
 88   parameter to the callback; the referent will no longer be available.
 89
 90   It is allowable for many weak references to be constructed for the same object.
 91   Callbacks registered for each weak reference will be called from the most
 92   recently registered callback to the oldest registered callback.
 93
 94   Exceptions raised by the callback will be noted on the standard error output,
 95   but cannot be propagated; they are handled in exactly the same way as exceptions
 96   raised from an object's :meth:`__del__` method.
 97
 98   Weak references are :term:`hashable` if the *object* is hashable.  They will maintain
 99   their hash value even after the *object* was deleted.  If :func:`hash` is called
100   the first time only after the *object* was deleted, the call will raise
101   :exc:`TypeError`.
102
103   Weak references support tests for equality, but not ordering.  If the referents
104   are still alive, two references have the same equality relationship as their
105   referents (regardless of the *callback*).  If either referent has been deleted,
106   the references are equal only if the reference objects are the same object.
107
108   .. versionchanged:: 2.4
109      This is now a subclassable type rather than a factory function; it derives from
110      :class:`object`.
111
112
113.. function:: proxy(object[, callback])
114
115   Return a proxy to *object* which uses a weak reference.  This supports use of
116   the proxy in most contexts instead of requiring the explicit dereferencing used
117   with weak reference objects.  The returned object will have a type of either
118   ``ProxyType`` or ``CallableProxyType``, depending on whether *object* is
119   callable.  Proxy objects are not :term:`hashable` regardless of the referent; this
120   avoids a number of problems related to their fundamentally mutable nature, and
121   prevent their use as dictionary keys.  *callback* is the same as the parameter
122   of the same name to the :func:`ref` function.
123
124
125.. function:: getweakrefcount(object)
126
127   Return the number of weak references and proxies which refer to *object*.
128
129
130.. function:: getweakrefs(object)
131
132   Return a list of all weak reference and proxy objects which refer to *object*.
133
134
135.. class:: WeakKeyDictionary([dict])
136
137   Mapping class that references keys weakly.  Entries in the dictionary will be
138   discarded when there is no longer a strong reference to the key.  This can be
139   used to associate additional data with an object owned by other parts of an
140   application without adding attributes to those objects.  This can be especially
141   useful with objects that override attribute accesses.
142
143   .. note::
144
145      Caution: Because a :class:`WeakKeyDictionary` is built on top of a Python
146      dictionary, it must not change size when iterating over it.  This can be
147      difficult to ensure for a :class:`WeakKeyDictionary` because actions
148      performed by the program during iteration may cause items in the
149      dictionary to vanish "by magic" (as a side effect of garbage collection).
150
151:class:`WeakKeyDictionary` objects have the following additional methods.  These
152expose the internal references directly.  The references are not guaranteed to
153be "live" at the time they are used, so the result of calling the references
154needs to be checked before being used.  This can be used to avoid creating
155references that will cause the garbage collector to keep the keys around longer
156than needed.
157
158
159.. method:: WeakKeyDictionary.iterkeyrefs()
160
161   Return an :term:`iterator` that yields the weak references to the keys.
162
163   .. versionadded:: 2.5
164
165
166.. method:: WeakKeyDictionary.keyrefs()
167
168   Return a list of weak references to the keys.
169
170   .. versionadded:: 2.5
171
172
173.. class:: WeakValueDictionary([dict])
174
175   Mapping class that references values weakly.  Entries in the dictionary will be
176   discarded when no strong reference to the value exists any more.
177
178   .. note::
179
180      Caution:  Because a :class:`WeakValueDictionary` is built on top of a Python
181      dictionary, it must not change size when iterating over it.  This can be
182      difficult to ensure for a :class:`WeakValueDictionary` because actions performed
183      by the program during iteration may cause items in the dictionary to vanish "by
184      magic" (as a side effect of garbage collection).
185
186:class:`WeakValueDictionary` objects have the following additional methods.
187These method have the same issues as the :meth:`iterkeyrefs` and :meth:`keyrefs`
188methods of :class:`WeakKeyDictionary` objects.
189
190
191.. method:: WeakValueDictionary.itervaluerefs()
192
193   Return an :term:`iterator` that yields the weak references to the values.
194
195   .. versionadded:: 2.5
196
197
198.. method:: WeakValueDictionary.valuerefs()
199
200   Return a list of weak references to the values.
201
202   .. versionadded:: 2.5
203
204
205.. data:: ReferenceType
206
207   The type object for weak references objects.
208
209
210.. data:: ProxyType
211
212   The type object for proxies of objects which are not callable.
213
214
215.. data:: CallableProxyType
216
217   The type object for proxies of callable objects.
218
219
220.. data:: ProxyTypes
221
222   Sequence containing all the type objects for proxies.  This can make it simpler
223   to test if an object is a proxy without being dependent on naming both proxy
224   types.
225
226
227.. exception:: ReferenceError
228
229   Exception raised when a proxy object is used but the underlying object has been
230   collected.  This is the same as the standard :exc:`ReferenceError` exception.
231
232
233.. seealso::
234
235   :pep:`0205` - Weak References
236      The proposal and rationale for this feature, including links to earlier
237      implementations and information about similar features in other languages.
238
239
240.. _weakref-objects:
241
242Weak Reference Objects
243----------------------
244
245Weak reference objects have no attributes or methods, but do allow the referent
246to be obtained, if it still exists, by calling it:
247
248   >>> import weakref
249   >>> class Object:
250   ...     pass
251   ...
252   >>> o = Object()
253   >>> r = weakref.ref(o)
254   >>> o2 = r()
255   >>> o is o2
256   True
257
258If the referent no longer exists, calling the reference object returns
259:const:`None`:
260
261   >>> del o, o2
262   >>> print r()
263   None
264
265Testing that a weak reference object is still live should be done using the
266expression ``ref() is not None``.  Normally, application code that needs to use
267a reference object should follow this pattern::
268
269   # r is a weak reference object
270   o = r()
271   if o is None:
272       # referent has been garbage collected
273       print "Object has been deallocated; can't frobnicate."
274   else:
275       print "Object is still live!"
276       o.do_something_useful()
277
278Using a separate test for "liveness" creates race conditions in threaded
279applications; another thread can cause a weak reference to become invalidated
280before the weak reference is called; the idiom shown above is safe in threaded
281applications as well as single-threaded applications.
282
283Specialized versions of :class:`ref` objects can be created through subclassing.
284This is used in the implementation of the :class:`WeakValueDictionary` to reduce
285the memory overhead for each entry in the mapping.  This may be most useful to
286associate additional information with a reference, but could also be used to
287insert additional processing on calls to retrieve the referent.
288
289This example shows how a subclass of :class:`ref` can be used to store
290additional information about an object and affect the value that's returned when
291the referent is accessed::
292
293   import weakref
294
295   class ExtendedRef(weakref.ref):
296       def __init__(self, ob, callback=None, **annotations):
297           super(ExtendedRef, self).__init__(ob, callback)
298           self.__counter = 0
299           for k, v in annotations.iteritems():
300               setattr(self, k, v)
301
302       def __call__(self):
303           """Return a pair containing the referent and the number of
304           times the reference has been called.
305           """
306           ob = super(ExtendedRef, self).__call__()
307           if ob is not None:
308               self.__counter += 1
309               ob = (ob, self.__counter)
310           return ob
311
312
313.. _weakref-example:
314
315Example
316-------
317
318This simple example shows how an application can use objects IDs to retrieve
319objects that it has seen before.  The IDs of the objects can then be used in
320other data structures without forcing the objects to remain alive, but the
321objects can still be retrieved by ID if they do.
322
323.. Example contributed by Tim Peters.
324
325::
326
327   import weakref
328
329   _id2obj_dict = weakref.WeakValueDictionary()
330
331   def remember(obj):
332       oid = id(obj)
333       _id2obj_dict[oid] = obj
334       return oid
335
336   def id2obj(oid):
337       return _id2obj_dict[oid]
338