/DOCS.rst

  1Polymorphic Models for Django
  2=============================
  3
  4.. contents:: Table of Contents
  5    :depth: 1
  6
  7
  8Quickstart
  9===========
 10
 11Install
 12-------
 13
 14After uncompressing (if necessary), in the directory "...django_polymorphic",
 15execute  (on Unix-like systems)::
 16
 17    sudo python setup.py install
 18
 19Make Your Models Polymorphic
 20----------------------------
 21
 22Use ``PolymorphicModel`` instead of Django's ``models.Model``, like so::
 23
 24    from polymorphic import PolymorphicModel
 25
 26    class Project(PolymorphicModel):
 27            topic = models.CharField(max_length=30)
 28
 29    class ArtProject(Project):
 30            artist = models.CharField(max_length=30)
 31
 32    class ResearchProject(Project):
 33            supervisor = models.CharField(max_length=30)
 34
 35All models inheriting from your polymorphic models will be polymorphic as well.
 36
 37Create some objects
 38-------------------
 39
 40>>> Project.objects.create(topic="Department Party")
 41>>> ArtProject.objects.create(topic="Painting with Tim", artist="T. Turner")
 42>>> ResearchProject.objects.create(topic="Swallow Aerodynamics", supervisor="Dr. Winter")
 43
 44Get polymorphic query results
 45-----------------------------
 46
 47>>> Project.objects.all()
 48[ <Project:         id 1, topic "Department Party">,
 49  <ArtProject:      id 2, topic "Painting with Tim", artist "T. Turner">,
 50  <ResearchProject: id 3, topic "Swallow Aerodynamics", supervisor "Dr. Winter"> ]
 51
 52use ``instance_of`` or ``not_instance_of`` for narrowing the result to specific subtypes:
 53
 54>>> Project.objects.instance_of(ArtProject)
 55[ <ArtProject:      id 2, topic "Painting with Tim", artist "T. Turner"> ]
 56
 57>>> Project.objects.instance_of(ArtProject) | Project.objects.instance_of(ResearchProject)
 58[ <ArtProject:      id 2, topic "Painting with Tim", artist "T. Turner">,
 59  <ResearchProject: id 3, topic "Swallow Aerodynamics", supervisor "Dr. Winter"> ]
 60
 61Polymorphic filtering: Get all projects where Mr. Turner is involved as an artist
 62or supervisor (note the three underscores):
 63
 64>>> Project.objects.filter(  Q(ArtProject___artist = 'T. Turner') | Q(ResearchProject___supervisor = 'T. Turner')  )
 65[ <ArtProject:      id 2, topic "Painting with Tim", artist "T. Turner">,
 66  <ResearchProject: id 4, topic "Color Use in Late Cubism", supervisor "T. Turner"> ]
 67
 68This is basically all you need to know, as django_polymorphic mostly
 69works fully automatic and just delivers the expected ("pythonic") results.
 70
 71Note: In all example output, above and below, for a nicer and more informative
 72output the ``ShowFieldType`` mixin has been used (documented below).
 73
 74
 75List of Features
 76================
 77
 78*   Fully automatic - generally makes sure that the same objects are
 79    returned from the database that were stored there, regardless how
 80    they are retrieved
 81*   Only on models that request polymorphic behaviour (and the
 82    models inheriting from them)
 83*   Full support for ForeignKeys, ManyToManyFields and OneToToneFields
 84*   Filtering for classes, equivalent to python's isinstance():
 85    ``instance_of(...)`` and ``not_instance_of(...)``
 86*   Polymorphic filtering/ordering etc., allowing the use of fields of
 87    derived models ("ArtProject___artist")
 88*   Support for user-defined custom managers
 89*   Automatic inheritance of custom managers
 90*   Support for user-defined custom queryset classes
 91*   Non-polymorphic queries if needed, with no other change in
 92    features/behaviour
 93*   Combining querysets of different types/models ("qs3 = qs1 | qs2")
 94*   Nice/informative display of polymorphic queryset results
 95
 96
 97More about Installation / Testing
 98=================================
 99
100Requirements
101------------
102
103Django 1.1 (or later) and Python 2.4 or later. This code has been tested
104on Django 1.1 / 1.2 / 1.3 and Python 2.4.6 / 2.5.4 / 2.6.4 on Linux.
105
106Included Test Suite
107-------------------
108
109The repository (or tar file) contains a complete Django project
110that may be used for tests or experiments, without any installation needed.
111
112To run the included test suite, in the directory "...django_polymorphic" execute::
113
114    ./manage test polymorphic
115
116The management command ``pcmd.py`` in the app ``pexp`` can be used
117for quick tests or experiments - modify this file (pexp/management/commands/pcmd.py)
118to your liking, then run::
119
120    ./manage syncdb      # db is created in /var/tmp/... (settings.py)
121    ./manage pcmd
122
123Installation
124------------
125
126In the directory "...django_polymorphic", execute ``sudo python setup.py install``.
127
128Alternatively you can simply copy the ``polymorphic`` subdirectory
129(under "django_polymorphic") into your Django project dir
130(e.g. if you want to distribute your project with more 'batteries included').
131
132If you want to run the test cases in `polymorphic/tests.py`, you need to add
133``polymorphic`` to your INSTALLED_APPS setting.
134
135Django's ContentType framework (``django.contrib.contenttypes``)
136needs to be listed in INSTALLED_APPS (usually it already is).
137
138
139More Polymorphic Functionality
140==============================
141
142In the examples below, these models are being used::
143
144    from polymorphic import PolymorphicModel
145
146    class ModelA(PolymorphicModel):
147        field1 = models.CharField(max_length=10)
148
149    class ModelB(ModelA):
150        field2 = models.CharField(max_length=10)
151
152    class ModelC(ModelB):
153        field3 = models.CharField(max_length=10)
154
155
156Filtering for classes (equivalent to python's isinstance() ):
157-------------------------------------------------------------
158
159>>> ModelA.objects.instance_of(ModelB)
160.
161[ <ModelB: id 2, field1 (CharField), field2 (CharField)>,
162  <ModelC: id 3, field1 (CharField), field2 (CharField), field3 (CharField)> ]
163
164In general, including or excluding parts of the inheritance tree::
165
166    ModelA.objects.instance_of(ModelB [, ModelC ...])
167    ModelA.objects.not_instance_of(ModelB [, ModelC ...])
168
169You can also use this feature in Q-objects (with the same result as above):
170
171>>> ModelA.objects.filter( Q(instance_of=ModelB) )
172
173
174Polymorphic filtering (for fields in derived classes)
175-----------------------------------------------------
176
177For example, cherrypicking objects from multiple derived classes
178anywhere in the inheritance tree, using Q objects (with the
179syntax: ``exact model name + three _ + field name``):
180
181>>> ModelA.objects.filter(  Q(ModelB___field2 = 'B2') | Q(ModelC___field3 = 'C3')  )
182.
183[ <ModelB: id 2, field1 (CharField), field2 (CharField)>,
184  <ModelC: id 3, field1 (CharField), field2 (CharField), field3 (CharField)> ]
185
186
187Combining Querysets
188-------------------
189
190Querysets could now be regarded as object containers that allow the
191aggregation of different object types, very similar to python
192lists - as long as the objects are accessed through the manager of
193a common base class:
194
195>>> Base.objects.instance_of(ModelX) | Base.objects.instance_of(ModelY)
196.
197[ <ModelX: id 1, field_x (CharField)>,
198  <ModelY: id 2, field_y (CharField)> ]
199
200
201ManyToManyField, ForeignKey, OneToOneField
202------------------------------------------
203
204Relationship fields referring to polymorphic models work as
205expected: like polymorphic querysets they now always return the
206referred objects with the same type/class these were created and
207saved as.
208
209E.g., if in your model you define::
210
211    field1 = OneToOneField(ModelA)
212
213then field1 may now also refer to objects of type ``ModelB`` or ``ModelC``.
214
215A ManyToManyField example::
216
217    # The model holding the relation may be any kind of model, polymorphic or not
218    class RelatingModel(models.Model):
219        many2many = models.ManyToManyField('ModelA')  # ManyToMany relation to a polymorphic model
220
221    >>> o=RelatingModel.objects.create()
222    >>> o.many2many.add(ModelA.objects.get(id=1))
223    >>> o.many2many.add(ModelB.objects.get(id=2))
224    >>> o.many2many.add(ModelC.objects.get(id=3))
225
226    >>> o.many2many.all()
227    [ <ModelA: id 1, field1 (CharField)>,
228      <ModelB: id 2, field1 (CharField), field2 (CharField)>,
229      <ModelC: id 3, field1 (CharField), field2 (CharField), field3 (CharField)> ]
230
231
232Using Third Party Models (without modifying them)
233-------------------------------------------------
234
235Third party models can be used as polymorphic models without
236restrictions by subclassing them. E.g. using a third party
237model as the root of a polymorphic inheritance tree::
238
239    from thirdparty import ThirdPartyModel
240
241    class MyThirdPartyBaseModel(PolymorhpicModel, ThirdPartyModel):
242        pass    # or add fields
243
244Or instead integrating the third party model anywhere into an
245existing polymorphic inheritance tree::
246
247    class MyBaseModel(SomePolymorphicModel):
248        my_field = models.CharField(max_length=10)
249
250    class MyModelWithThirdParty(MyBaseModel, ThirdPartyModel):
251        pass    # or add fields
252
253
254Non-Polymorphic Queries
255-----------------------
256
257If you insert ``.non_polymorphic()`` anywhere into the query chain, then
258django_polymorphic will simply leave out the final step of retrieving the
259real objects, and the manager/queryset will return objects of the type of
260the base class you used for the query, like vanilla Django would
261(``ModelA`` in this example). 
262
263>>> qs=ModelA.objects.non_polymorphic().all()
264>>> qs
265[ <ModelA: id 1, field1 (CharField)>,
266  <ModelA: id 2, field1 (CharField)>,
267  <ModelA: id 3, field1 (CharField)> ]
268
269There are no other changes in the behaviour of the queryset. For example,
270enhancements for ``filter()`` or ``instance_of()`` etc. still work as expected.
271If you do the final step yourself, you get the usual polymorphic result:
272
273>>> ModelA.objects.get_real_instances(qs)
274[ <ModelA: id 1, field1 (CharField)>,
275  <ModelB: id 2, field1 (CharField), field2 (CharField)>,
276  <ModelC: id 3, field1 (CharField), field2 (CharField), field3 (CharField)> ]
277
278
279About Queryset Methods
280----------------------
281
282*   ``annotate()`` and ``aggregate()`` work just as usual, with the
283    addition that the ``ModelX___field`` syntax can be used for the
284    keyword arguments (but not for the non-keyword arguments).
285
286*   ``order_by()`` now similarly supports the ``ModelX___field`` syntax
287    for specifying ordering through a field in a submodel.
288
289*   ``distinct()`` works as expected. It only regards the fields of
290    the base class, but this should never make a difference.
291
292*   ``select_related()`` works just as usual, but it can not (yet) be used
293    to select relations in derived models
294    (like ``ModelA.objects.select_related('ModelC___fieldxy')`` )
295
296*   ``extra()`` works as expected (it returns polymorphic results) but
297    currently has one restriction: The resulting objects are required to have
298    a unique primary key within the result set - otherwise an error is thrown
299    (this case could be made to work, however it may be mostly unneeded)..
300    The keyword-argument "polymorphic" is no longer supported.
301    You can get back the old non-polymorphic behaviour (before V1.0)
302    by using ``ModelA.objects.non_polymorphic().extra(...)``.
303
304*   ``get_real_instances()`` allows you to turn a
305    queryset or list  of base model objects efficiently into the real objects.
306    For example, you could do ``base_objects_queryset=ModelA.extra(...).non_polymorphic()``
307    and then call ``real_objects=base_objects_queryset.get_real_instances()``.Or alternatively
308    .``real_objects=ModelA.objects..get_real_instances(base_objects_queryset_or_object_list)``
309
310*   ``values()`` & ``values_list()`` currently do not return polymorphic
311    results. This may change in the future however. If you want to use these
312    methods now, it's best if you use ``Model.base_objects.values...`` as
313    this is guaranteed to not change. 
314
315*   ``defer()`` and ``only()`` are not yet supported (support will be added
316    in the future). 
317
318
319Using enhanced Q-objects in any Places
320--------------------------------------
321
322The queryset enhancements (e.g. ``instance_of``) only work as arguments
323to the member functions of a polymorphic queryset.  Occationally it may
324be useful to be able to use Q objects with these enhancements in other places.
325As Django doesn't understand these enhanced Q objects, you need to
326transform them manually into normal Q objects before you can feed them
327to a Django queryset or function::
328
329    normal_q_object = ModelA.translate_polymorphic_Q_object( Q(instance_of=Model2B) )
330
331This function cannot be used at model creation time however (in models.py),
332as it may need to access the ContentTypes database table.
333
334
335Nicely Displaying Polymorphic Querysets
336---------------------------------------
337
338In order to get the output as seen in all examples here, you need to use the
339ShowFieldType class mixin::
340
341    from polymorphic import PolymorphicModel, ShowFieldType
342
343    class ModelA(ShowFieldType, PolymorphicModel):
344        field1 = models.CharField(max_length=10)
345
346You may also use ShowFieldContent or ShowFieldTypeAndContent to display
347additional information when printing querysets (or converting them to text).
348
349When showing field contents, they will be truncated to 20 characters. You can
350modify this behaviour by setting a class variable in your model like this::
351
352    class ModelA(ShowFieldType, PolymorphicModel):
353        polymorphic_showfield_max_field_width = 20
354        ...
355
356Similarly, pre-V1.0 output formatting can be re-estated by using
357``polymorphic_showfield_old_format = True``.
358
359Custom Managers, Querysets & Manager Inheritance
360================================================
361    
362Using a Custom Manager
363----------------------
364
365A nice feature of Django is the possibility to define one's own custom object managers.
366This is fully supported with django_polymorphic: For creating a custom polymorphic
367manager class, just derive your manager from ``PolymorphicManager`` instead of
368``models.Manager``. As with vanilla Django, in your model class, you should
369explicitly add the default manager first, and then your custom manager::
370
371    from polymorphic import PolymorphicModel, PolymorphicManager
372
373   class TimeOrderedManager(PolymorphicManager):
374        def get_query_set(self):
375            qs = super(TimeOrderedManager,self).get_query_set()
376            return qs.order_by('-start_date')        # order the queryset
377
378        def most_recent(self):
379            qs = self.get_query_set()                # get my ordered queryset
380            return qs[:10]                           # limit => get ten most recent entries
381
382    class Project(PolymorphicModel):
383        objects = PolymorphicManager()               # add the default polymorphic manager first
384        objects_ordered = TimeOrderedManager()       # then add your own manager
385        start_date = DateTimeField()                 # project start is this date/time
386
387The first manager defined ('objects' in the example) is used by
388Django as automatic manager for several purposes, including accessing
389related objects. It must not filter objects and it's safest to use
390the plain ``PolymorphicManager`` here.
391
392Manager Inheritance
393-------------------
394
395Polymorphic models inherit/propagate all managers from their
396base models, as long as these are polymorphic. This means that all
397managers defined in polymorphic base models continue to work as
398expected in models inheriting from this base model::
399
400   from polymorphic import PolymorphicModel, PolymorphicManager
401
402   class TimeOrderedManager(PolymorphicManager):
403        def get_query_set(self):
404            qs = super(TimeOrderedManager,self).get_query_set()
405            return qs.order_by('-start_date')        # order the queryset
406
407        def most_recent(self):
408            qs = self.get_query_set()                # get my ordered queryset
409            return qs[:10]                           # limit => get ten most recent entries
410
411    class Project(PolymorphicModel):
412        objects = PolymorphicManager()               # add the default polymorphic manager first
413        objects_ordered = TimeOrderedManager()       # then add your own manager
414        start_date = DateTimeField()                 # project start is this date/time
415
416    class ArtProject(Project):                       # inherit from Project, inheriting its fields and managers
417        artist = models.CharField(max_length=30)
418
419ArtProject inherited the managers ``objects`` and ``objects_ordered`` from Project.
420
421``ArtProject.objects_ordered.all()`` will return all art projects ordered
422regarding their start time and ``ArtProject.objects_ordered.most_recent()``
423will return the ten most recent art projects.
424.
425
426Using a Custom Queryset Class
427-----------------------------
428
429The ``PolymorphicManager`` class accepts one initialization argument,
430which is the queryset class the manager should use. Just as with vanilla Django,
431you may define your own custom queryset classes. Just use PolymorphicQuerySet
432instead of Django's QuerySet as the base class::
433
434        from polymorphic import PolymorphicModel, PolymorphicManager, PolymorphicQuerySet
435
436        class MyQuerySet(PolymorphicQuerySet):
437            def my_queryset_method(...):
438                ...
439    
440        class MyModel(PolymorphicModel):
441            my_objects=PolymorphicManager(MyQuerySet)
442            ...
443
444
445Performance Considerations
446==========================
447
448The current implementation is rather simple and does not use any
449custom SQL or Django DB layer internals - it is purely based on the
450standard Django ORM.
451
452Specifically, the query::
453
454    result_objects = list( ModelA.objects.filter(...) )
455
456performs one SQL query to retrieve ``ModelA`` objects and one additional
457query for each unique derived class occurring in result_objects.
458The best case for retrieving 100 objects is 1 SQL query if all are
459class ``ModelA``. If 50 objects are ``ModelA`` and 50 are ``ModelB``, then
460two queries are executed. The pathological worst case is 101 db queries if
461result_objects contains 100 different object types (with all of them
462subclasses of ``ModelA``).
463
464Usually, when Django users create their own polymorphic ad-hoc solution
465without a tool like django_polymorphic, this usually results in a variation of ::
466
467    result_objects = [ o.get_real_instance() for o in BaseModel.objects.filter(...) ]
468
469which has very bad performance, as it introduces one additional
470SQL query for every object in the result which is not of class ``BaseModel``.
471
472Compared to these solutions, django_polymorphic has the advantage
473that it only needs one sql request per *object type*, and not *per object*.
474
475.. _performance:
476
477Performance Problems with PostgreSQL, MySQL and SQLite3
478-------------------------------------------------------
479
480Current relational DBM systems seem to have general problems with
481the SQL queries produced by object relational mappers like the Django
482ORM, if these use multi-table inheritance like Django's ORM does.
483The "inner joins" in these queries can perform very badly.
484This is independent of django_polymorphic and affects all uses of
485multi table Model inheritance.
486
487Concrete benchmark results are forthcoming (please see discussion forum).
488
489Please also see this `post (and comments) from Jacob Kaplan-Moss`_.
490
491.. _post (and comments) from Jacob Kaplan-Moss: http://www.jacobian.org/writing/concrete-inheritance/
492
493
494.. _restrictions:
495
496Restrictions & Caveats
497======================
498
499*   Database Performance regarding concrete Model inheritance in general.
500    Please see "Performance Problems" above.
501
502*   Queryset methods ``values()``, ``values_list()``, ``select_related()``,
503    ``defer()`` and ``only()`` are not yet fully supported (see above).
504    ``extra()`` has one restriction: the resulting objects are required to have
505    a unique primary key within the result set.
506
507*   Django Admin Integration: There currently is no specific admin integration,
508    but it would most likely make sense to have one.
509
510*   Diamond shaped inheritance: There seems to be a general problem 
511    with diamond shaped multiple model inheritance with Django models
512    (tested with V1.1 - V1.3).
513    An example is here: http://code.djangoproject.com/ticket/10808.
514    This problem is aggravated when trying to enhance models.Model
515    by subclassing it instead of modifying Django core (as we do here
516    with PolymorphicModel).
517
518*   The enhanced filter-definitions/Q-objects only work as arguments
519    for the methods of the polymorphic querysets. Please see above
520    for ``translate_polymorphic_Q_object``.
521
522*   A reference (``ContentType``) to the real/leaf model is stored
523    in the base model (the base model directly inheriting from
524    PolymorphicModel). You need to be aware of this when using the
525    ``dumpdata`` management command or any other low-level
526    database operations. E.g. if you rename models or apps or copy
527    objects from one database to another, then Django's ContentType
528    table needs to be corrected/copied too. This is of course generally
529    the case for any models using Django's ContentType.
530
531*   Django 1.1 only - the names of polymorphic models must be unique
532    in the whole project, even if they are in two different apps.
533    This results from a restriction in the Django 1.1 "related_name"
534    option (fixed in Django 1.2).
535
536*   Django 1.1 only - when ContentType is used in models, Django's
537    seralisation or fixtures cannot be used (all polymorphic models
538    use ContentType). This issue seems to be resolved for Django 1.2
539    (changeset 11863: Fixed #7052, Added support for natural keys in serialization).
540
541    + http://code.djangoproject.com/ticket/7052
542    + http://stackoverflow.com/questions/853796/problems-with-contenttypes-when-loading-a-fixture-in-django
543
544
545Project Status
546==============   
547 
548Django_polymorphic works well for a considerable number of users now,
549and no major problems have shown up for many months.
550The API can be considered stable beginning with the V1.0 release.
551
552
553Links
554=====
555
556- http://code.djangoproject.com/wiki/ModelInheritance
557- http://lazypython.blogspot.com/2009/02/second-look-at-inheritance-and.html
558- http://www.djangosnippets.org/snippets/1031/
559- http://www.djangosnippets.org/snippets/1034/
560- http://groups.google.com/group/django-developers/browse_frm/thread/7d40ad373ebfa912/a20fabc661b7035d?lnk=gst&q=model+inheritance+CORBA#a20fabc661b7035d
561- http://groups.google.com/group/django-developers/browse_thread/thread/9bc2aaec0796f4e0/0b92971ffc0aa6f8?lnk=gst&q=inheritance#0b92971ffc0aa6f8
562- http://groups.google.com/group/django-developers/browse_thread/thread/3947c594100c4adb/d8c0af3dacad412d?lnk=gst&q=inheritance#d8c0af3dacad412d
563- http://groups.google.com/group/django-users/browse_thread/thread/52f72cffebb705e/b76c9d8c89a5574f
564- http://peterbraden.co.uk/article/django-inheritance
565- http://www.hopelessgeek.com/2009/11/25/a-hack-for-multi-table-inheritance-in-django
566- http://stackoverflow.com/questions/929029/how-do-i-access-the-child-classes-of-an-object-in-django-without-knowing-the-name/929982#929982
567- http://stackoverflow.com/questions/1581024/django-inheritance-how-to-have-one-method-for-all-subclasses
568- http://groups.google.com/group/django-users/browse_thread/thread/cbdaf2273781ccab/e676a537d735d9ef?lnk=gst&q=polymorphic#e676a537d735d9ef
569- http://groups.google.com/group/django-users/browse_thread/thread/52f72cffebb705e/bc18c18b2e83881e?lnk=gst&q=model+inheritance#bc18c18b2e83881e
570- http://code.djangoproject.com/ticket/10808
571- http://code.djangoproject.com/ticket/7270
572