/DOCS.rst
ReStructuredText | 572 lines | 403 code | 169 blank | 0 comment | 0 complexity | 3e1d7af599d39e636d51338e12ec579c MD5 | raw file
1Polymorphic Models for Django 2============================= 3 4.. contents:: Table of Contents 5 :depth: 1 6 7 8Quickstart 9=========== 10 11Install 12------- 13 14After uncompressing (if necessary), in the directory "...django_polymorphic", 15execute (on Unix-like systems):: 16 17 sudo python setup.py install 18 19Make Your Models Polymorphic 20---------------------------- 21 22Use ``PolymorphicModel`` instead of Django's ``models.Model``, like so:: 23 24 from polymorphic import PolymorphicModel 25 26 class Project(PolymorphicModel): 27 topic = models.CharField(max_length=30) 28 29 class ArtProject(Project): 30 artist = models.CharField(max_length=30) 31 32 class ResearchProject(Project): 33 supervisor = models.CharField(max_length=30) 34 35All models inheriting from your polymorphic models will be polymorphic as well. 36 37Create some objects 38------------------- 39 40>>> Project.objects.create(topic="Department Party") 41>>> ArtProject.objects.create(topic="Painting with Tim", artist="T. Turner") 42>>> ResearchProject.objects.create(topic="Swallow Aerodynamics", supervisor="Dr. Winter") 43 44Get polymorphic query results 45----------------------------- 46 47>>> Project.objects.all() 48[ <Project: id 1, topic "Department Party">, 49 <ArtProject: id 2, topic "Painting with Tim", artist "T. Turner">, 50 <ResearchProject: id 3, topic "Swallow Aerodynamics", supervisor "Dr. Winter"> ] 51 52use ``instance_of`` or ``not_instance_of`` for narrowing the result to specific subtypes: 53 54>>> Project.objects.instance_of(ArtProject) 55[ <ArtProject: id 2, topic "Painting with Tim", artist "T. Turner"> ] 56 57>>> Project.objects.instance_of(ArtProject) | Project.objects.instance_of(ResearchProject) 58[ <ArtProject: id 2, topic "Painting with Tim", artist "T. Turner">, 59 <ResearchProject: id 3, topic "Swallow Aerodynamics", supervisor "Dr. Winter"> ] 60 61Polymorphic filtering: Get all projects where Mr. Turner is involved as an artist 62or supervisor (note the three underscores): 63 64>>> Project.objects.filter( Q(ArtProject___artist = 'T. Turner') | Q(ResearchProject___supervisor = 'T. Turner') ) 65[ <ArtProject: id 2, topic "Painting with Tim", artist "T. Turner">, 66 <ResearchProject: id 4, topic "Color Use in Late Cubism", supervisor "T. Turner"> ] 67 68This is basically all you need to know, as django_polymorphic mostly 69works fully automatic and just delivers the expected ("pythonic") results. 70 71Note: In all example output, above and below, for a nicer and more informative 72output the ``ShowFieldType`` mixin has been used (documented below). 73 74 75List of Features 76================ 77 78* Fully automatic - generally makes sure that the same objects are 79 returned from the database that were stored there, regardless how 80 they are retrieved 81* Only on models that request polymorphic behaviour (and the 82 models inheriting from them) 83* Full support for ForeignKeys, ManyToManyFields and OneToToneFields 84* Filtering for classes, equivalent to python's isinstance(): 85 ``instance_of(...)`` and ``not_instance_of(...)`` 86* Polymorphic filtering/ordering etc., allowing the use of fields of 87 derived models ("ArtProject___artist") 88* Support for user-defined custom managers 89* Automatic inheritance of custom managers 90* Support for user-defined custom queryset classes 91* Non-polymorphic queries if needed, with no other change in 92 features/behaviour 93* Combining querysets of different types/models ("qs3 = qs1 | qs2") 94* Nice/informative display of polymorphic queryset results 95 96 97More about Installation / Testing 98================================= 99 100Requirements 101------------ 102 103Django 1.1 (or later) and Python 2.4 or later. This code has been tested 104on Django 1.1 / 1.2 / 1.3 and Python 2.4.6 / 2.5.4 / 2.6.4 on Linux. 105 106Included Test Suite 107------------------- 108 109The repository (or tar file) contains a complete Django project 110that may be used for tests or experiments, without any installation needed. 111 112To run the included test suite, in the directory "...django_polymorphic" execute:: 113 114 ./manage test polymorphic 115 116The management command ``pcmd.py`` in the app ``pexp`` can be used 117for quick tests or experiments - modify this file (pexp/management/commands/pcmd.py) 118to your liking, then run:: 119 120 ./manage syncdb # db is created in /var/tmp/... (settings.py) 121 ./manage pcmd 122 123Installation 124------------ 125 126In the directory "...django_polymorphic", execute ``sudo python setup.py install``. 127 128Alternatively you can simply copy the ``polymorphic`` subdirectory 129(under "django_polymorphic") into your Django project dir 130(e.g. if you want to distribute your project with more 'batteries included'). 131 132If you want to run the test cases in `polymorphic/tests.py`, you need to add 133``polymorphic`` to your INSTALLED_APPS setting. 134 135Django's ContentType framework (``django.contrib.contenttypes``) 136needs to be listed in INSTALLED_APPS (usually it already is). 137 138 139More Polymorphic Functionality 140============================== 141 142In the examples below, these models are being used:: 143 144 from polymorphic import PolymorphicModel 145 146 class ModelA(PolymorphicModel): 147 field1 = models.CharField(max_length=10) 148 149 class ModelB(ModelA): 150 field2 = models.CharField(max_length=10) 151 152 class ModelC(ModelB): 153 field3 = models.CharField(max_length=10) 154 155 156Filtering for classes (equivalent to python's isinstance() ): 157------------------------------------------------------------- 158 159>>> ModelA.objects.instance_of(ModelB) 160. 161[ <ModelB: id 2, field1 (CharField), field2 (CharField)>, 162 <ModelC: id 3, field1 (CharField), field2 (CharField), field3 (CharField)> ] 163 164In general, including or excluding parts of the inheritance tree:: 165 166 ModelA.objects.instance_of(ModelB [, ModelC ...]) 167 ModelA.objects.not_instance_of(ModelB [, ModelC ...]) 168 169You can also use this feature in Q-objects (with the same result as above): 170 171>>> ModelA.objects.filter( Q(instance_of=ModelB) ) 172 173 174Polymorphic filtering (for fields in derived classes) 175----------------------------------------------------- 176 177For example, cherrypicking objects from multiple derived classes 178anywhere in the inheritance tree, using Q objects (with the 179syntax: ``exact model name + three _ + field name``): 180 181>>> ModelA.objects.filter( Q(ModelB___field2 = 'B2') | Q(ModelC___field3 = 'C3') ) 182. 183[ <ModelB: id 2, field1 (CharField), field2 (CharField)>, 184 <ModelC: id 3, field1 (CharField), field2 (CharField), field3 (CharField)> ] 185 186 187Combining Querysets 188------------------- 189 190Querysets could now be regarded as object containers that allow the 191aggregation of different object types, very similar to python 192lists - as long as the objects are accessed through the manager of 193a common base class: 194 195>>> Base.objects.instance_of(ModelX) | Base.objects.instance_of(ModelY) 196. 197[ <ModelX: id 1, field_x (CharField)>, 198 <ModelY: id 2, field_y (CharField)> ] 199 200 201ManyToManyField, ForeignKey, OneToOneField 202------------------------------------------ 203 204Relationship fields referring to polymorphic models work as 205expected: like polymorphic querysets they now always return the 206referred objects with the same type/class these were created and 207saved as. 208 209E.g., if in your model you define:: 210 211 field1 = OneToOneField(ModelA) 212 213then field1 may now also refer to objects of type ``ModelB`` or ``ModelC``. 214 215A ManyToManyField example:: 216 217 # The model holding the relation may be any kind of model, polymorphic or not 218 class RelatingModel(models.Model): 219 many2many = models.ManyToManyField('ModelA') # ManyToMany relation to a polymorphic model 220 221 >>> o=RelatingModel.objects.create() 222 >>> o.many2many.add(ModelA.objects.get(id=1)) 223 >>> o.many2many.add(ModelB.objects.get(id=2)) 224 >>> o.many2many.add(ModelC.objects.get(id=3)) 225 226 >>> o.many2many.all() 227 [ <ModelA: id 1, field1 (CharField)>, 228 <ModelB: id 2, field1 (CharField), field2 (CharField)>, 229 <ModelC: id 3, field1 (CharField), field2 (CharField), field3 (CharField)> ] 230 231 232Using Third Party Models (without modifying them) 233------------------------------------------------- 234 235Third party models can be used as polymorphic models without 236restrictions by subclassing them. E.g. using a third party 237model as the root of a polymorphic inheritance tree:: 238 239 from thirdparty import ThirdPartyModel 240 241 class MyThirdPartyBaseModel(PolymorhpicModel, ThirdPartyModel): 242 pass # or add fields 243 244Or instead integrating the third party model anywhere into an 245existing polymorphic inheritance tree:: 246 247 class MyBaseModel(SomePolymorphicModel): 248 my_field = models.CharField(max_length=10) 249 250 class MyModelWithThirdParty(MyBaseModel, ThirdPartyModel): 251 pass # or add fields 252 253 254Non-Polymorphic Queries 255----------------------- 256 257If you insert ``.non_polymorphic()`` anywhere into the query chain, then 258django_polymorphic will simply leave out the final step of retrieving the 259real objects, and the manager/queryset will return objects of the type of 260the base class you used for the query, like vanilla Django would 261(``ModelA`` in this example). 262 263>>> qs=ModelA.objects.non_polymorphic().all() 264>>> qs 265[ <ModelA: id 1, field1 (CharField)>, 266 <ModelA: id 2, field1 (CharField)>, 267 <ModelA: id 3, field1 (CharField)> ] 268 269There are no other changes in the behaviour of the queryset. For example, 270enhancements for ``filter()`` or ``instance_of()`` etc. still work as expected. 271If you do the final step yourself, you get the usual polymorphic result: 272 273>>> ModelA.objects.get_real_instances(qs) 274[ <ModelA: id 1, field1 (CharField)>, 275 <ModelB: id 2, field1 (CharField), field2 (CharField)>, 276 <ModelC: id 3, field1 (CharField), field2 (CharField), field3 (CharField)> ] 277 278 279About Queryset Methods 280---------------------- 281 282* ``annotate()`` and ``aggregate()`` work just as usual, with the 283 addition that the ``ModelX___field`` syntax can be used for the 284 keyword arguments (but not for the non-keyword arguments). 285 286* ``order_by()`` now similarly supports the ``ModelX___field`` syntax 287 for specifying ordering through a field in a submodel. 288 289* ``distinct()`` works as expected. It only regards the fields of 290 the base class, but this should never make a difference. 291 292* ``select_related()`` works just as usual, but it can not (yet) be used 293 to select relations in derived models 294 (like ``ModelA.objects.select_related('ModelC___fieldxy')`` ) 295 296* ``extra()`` works as expected (it returns polymorphic results) but 297 currently has one restriction: The resulting objects are required to have 298 a unique primary key within the result set - otherwise an error is thrown 299 (this case could be made to work, however it may be mostly unneeded).. 300 The keyword-argument "polymorphic" is no longer supported. 301 You can get back the old non-polymorphic behaviour (before V1.0) 302 by using ``ModelA.objects.non_polymorphic().extra(...)``. 303 304* ``get_real_instances()`` allows you to turn a 305 queryset or list of base model objects efficiently into the real objects. 306 For example, you could do ``base_objects_queryset=ModelA.extra(...).non_polymorphic()`` 307 and then call ``real_objects=base_objects_queryset.get_real_instances()``.Or alternatively 308 .``real_objects=ModelA.objects..get_real_instances(base_objects_queryset_or_object_list)`` 309 310* ``values()`` & ``values_list()`` currently do not return polymorphic 311 results. This may change in the future however. If you want to use these 312 methods now, it's best if you use ``Model.base_objects.values...`` as 313 this is guaranteed to not change. 314 315* ``defer()`` and ``only()`` are not yet supported (support will be added 316 in the future). 317 318 319Using enhanced Q-objects in any Places 320-------------------------------------- 321 322The queryset enhancements (e.g. ``instance_of``) only work as arguments 323to the member functions of a polymorphic queryset. Occationally it may 324be useful to be able to use Q objects with these enhancements in other places. 325As Django doesn't understand these enhanced Q objects, you need to 326transform them manually into normal Q objects before you can feed them 327to a Django queryset or function:: 328 329 normal_q_object = ModelA.translate_polymorphic_Q_object( Q(instance_of=Model2B) ) 330 331This function cannot be used at model creation time however (in models.py), 332as it may need to access the ContentTypes database table. 333 334 335Nicely Displaying Polymorphic Querysets 336--------------------------------------- 337 338In order to get the output as seen in all examples here, you need to use the 339ShowFieldType class mixin:: 340 341 from polymorphic import PolymorphicModel, ShowFieldType 342 343 class ModelA(ShowFieldType, PolymorphicModel): 344 field1 = models.CharField(max_length=10) 345 346You may also use ShowFieldContent or ShowFieldTypeAndContent to display 347additional information when printing querysets (or converting them to text). 348 349When showing field contents, they will be truncated to 20 characters. You can 350modify this behaviour by setting a class variable in your model like this:: 351 352 class ModelA(ShowFieldType, PolymorphicModel): 353 polymorphic_showfield_max_field_width = 20 354 ... 355 356Similarly, pre-V1.0 output formatting can be re-estated by using 357``polymorphic_showfield_old_format = True``. 358 359Custom Managers, Querysets & Manager Inheritance 360================================================ 361 362Using a Custom Manager 363---------------------- 364 365A nice feature of Django is the possibility to define one's own custom object managers. 366This is fully supported with django_polymorphic: For creating a custom polymorphic 367manager class, just derive your manager from ``PolymorphicManager`` instead of 368``models.Manager``. As with vanilla Django, in your model class, you should 369explicitly add the default manager first, and then your custom manager:: 370 371 from polymorphic import PolymorphicModel, PolymorphicManager 372 373 class TimeOrderedManager(PolymorphicManager): 374 def get_query_set(self): 375 qs = super(TimeOrderedManager,self).get_query_set() 376 return qs.order_by('-start_date') # order the queryset 377 378 def most_recent(self): 379 qs = self.get_query_set() # get my ordered queryset 380 return qs[:10] # limit => get ten most recent entries 381 382 class Project(PolymorphicModel): 383 objects = PolymorphicManager() # add the default polymorphic manager first 384 objects_ordered = TimeOrderedManager() # then add your own manager 385 start_date = DateTimeField() # project start is this date/time 386 387The first manager defined ('objects' in the example) is used by 388Django as automatic manager for several purposes, including accessing 389related objects. It must not filter objects and it's safest to use 390the plain ``PolymorphicManager`` here. 391 392Manager Inheritance 393------------------- 394 395Polymorphic models inherit/propagate all managers from their 396base models, as long as these are polymorphic. This means that all 397managers defined in polymorphic base models continue to work as 398expected in models inheriting from this base model:: 399 400 from polymorphic import PolymorphicModel, PolymorphicManager 401 402 class TimeOrderedManager(PolymorphicManager): 403 def get_query_set(self): 404 qs = super(TimeOrderedManager,self).get_query_set() 405 return qs.order_by('-start_date') # order the queryset 406 407 def most_recent(self): 408 qs = self.get_query_set() # get my ordered queryset 409 return qs[:10] # limit => get ten most recent entries 410 411 class Project(PolymorphicModel): 412 objects = PolymorphicManager() # add the default polymorphic manager first 413 objects_ordered = TimeOrderedManager() # then add your own manager 414 start_date = DateTimeField() # project start is this date/time 415 416 class ArtProject(Project): # inherit from Project, inheriting its fields and managers 417 artist = models.CharField(max_length=30) 418 419ArtProject inherited the managers ``objects`` and ``objects_ordered`` from Project. 420 421``ArtProject.objects_ordered.all()`` will return all art projects ordered 422regarding their start time and ``ArtProject.objects_ordered.most_recent()`` 423will return the ten most recent art projects. 424. 425 426Using a Custom Queryset Class 427----------------------------- 428 429The ``PolymorphicManager`` class accepts one initialization argument, 430which is the queryset class the manager should use. Just as with vanilla Django, 431you may define your own custom queryset classes. Just use PolymorphicQuerySet 432instead of Django's QuerySet as the base class:: 433 434 from polymorphic import PolymorphicModel, PolymorphicManager, PolymorphicQuerySet 435 436 class MyQuerySet(PolymorphicQuerySet): 437 def my_queryset_method(...): 438 ... 439 440 class MyModel(PolymorphicModel): 441 my_objects=PolymorphicManager(MyQuerySet) 442 ... 443 444 445Performance Considerations 446========================== 447 448The current implementation is rather simple and does not use any 449custom SQL or Django DB layer internals - it is purely based on the 450standard Django ORM. 451 452Specifically, the query:: 453 454 result_objects = list( ModelA.objects.filter(...) ) 455 456performs one SQL query to retrieve ``ModelA`` objects and one additional 457query for each unique derived class occurring in result_objects. 458The best case for retrieving 100 objects is 1 SQL query if all are 459class ``ModelA``. If 50 objects are ``ModelA`` and 50 are ``ModelB``, then 460two queries are executed. The pathological worst case is 101 db queries if 461result_objects contains 100 different object types (with all of them 462subclasses of ``ModelA``). 463 464Usually, when Django users create their own polymorphic ad-hoc solution 465without a tool like django_polymorphic, this usually results in a variation of :: 466 467 result_objects = [ o.get_real_instance() for o in BaseModel.objects.filter(...) ] 468 469which has very bad performance, as it introduces one additional 470SQL query for every object in the result which is not of class ``BaseModel``. 471 472Compared to these solutions, django_polymorphic has the advantage 473that it only needs one sql request per *object type*, and not *per object*. 474 475.. _performance: 476 477Performance Problems with PostgreSQL, MySQL and SQLite3 478------------------------------------------------------- 479 480Current relational DBM systems seem to have general problems with 481the SQL queries produced by object relational mappers like the Django 482ORM, if these use multi-table inheritance like Django's ORM does. 483The "inner joins" in these queries can perform very badly. 484This is independent of django_polymorphic and affects all uses of 485multi table Model inheritance. 486 487Concrete benchmark results are forthcoming (please see discussion forum). 488 489Please also see this `post (and comments) from Jacob Kaplan-Moss`_. 490 491.. _post (and comments) from Jacob Kaplan-Moss: http://www.jacobian.org/writing/concrete-inheritance/ 492 493 494.. _restrictions: 495 496Restrictions & Caveats 497====================== 498 499* Database Performance regarding concrete Model inheritance in general. 500 Please see "Performance Problems" above. 501 502* Queryset methods ``values()``, ``values_list()``, ``select_related()``, 503 ``defer()`` and ``only()`` are not yet fully supported (see above). 504 ``extra()`` has one restriction: the resulting objects are required to have 505 a unique primary key within the result set. 506 507* Django Admin Integration: There currently is no specific admin integration, 508 but it would most likely make sense to have one. 509 510* Diamond shaped inheritance: There seems to be a general problem 511 with diamond shaped multiple model inheritance with Django models 512 (tested with V1.1 - V1.3). 513 An example is here: http://code.djangoproject.com/ticket/10808. 514 This problem is aggravated when trying to enhance models.Model 515 by subclassing it instead of modifying Django core (as we do here 516 with PolymorphicModel). 517 518* The enhanced filter-definitions/Q-objects only work as arguments 519 for the methods of the polymorphic querysets. Please see above 520 for ``translate_polymorphic_Q_object``. 521 522* A reference (``ContentType``) to the real/leaf model is stored 523 in the base model (the base model directly inheriting from 524 PolymorphicModel). You need to be aware of this when using the 525 ``dumpdata`` management command or any other low-level 526 database operations. E.g. if you rename models or apps or copy 527 objects from one database to another, then Django's ContentType 528 table needs to be corrected/copied too. This is of course generally 529 the case for any models using Django's ContentType. 530 531* Django 1.1 only - the names of polymorphic models must be unique 532 in the whole project, even if they are in two different apps. 533 This results from a restriction in the Django 1.1 "related_name" 534 option (fixed in Django 1.2). 535 536* Django 1.1 only - when ContentType is used in models, Django's 537 seralisation or fixtures cannot be used (all polymorphic models 538 use ContentType). This issue seems to be resolved for Django 1.2 539 (changeset 11863: Fixed #7052, Added support for natural keys in serialization). 540 541 + http://code.djangoproject.com/ticket/7052 542 + http://stackoverflow.com/questions/853796/problems-with-contenttypes-when-loading-a-fixture-in-django 543 544 545Project Status 546============== 547 548Django_polymorphic works well for a considerable number of users now, 549and no major problems have shown up for many months. 550The API can be considered stable beginning with the V1.0 release. 551 552 553Links 554===== 555 556- http://code.djangoproject.com/wiki/ModelInheritance 557- http://lazypython.blogspot.com/2009/02/second-look-at-inheritance-and.html 558- http://www.djangosnippets.org/snippets/1031/ 559- http://www.djangosnippets.org/snippets/1034/ 560- http://groups.google.com/group/django-developers/browse_frm/thread/7d40ad373ebfa912/a20fabc661b7035d?lnk=gst&q=model+inheritance+CORBA#a20fabc661b7035d 561- http://groups.google.com/group/django-developers/browse_thread/thread/9bc2aaec0796f4e0/0b92971ffc0aa6f8?lnk=gst&q=inheritance#0b92971ffc0aa6f8 562- http://groups.google.com/group/django-developers/browse_thread/thread/3947c594100c4adb/d8c0af3dacad412d?lnk=gst&q=inheritance#d8c0af3dacad412d 563- http://groups.google.com/group/django-users/browse_thread/thread/52f72cffebb705e/b76c9d8c89a5574f 564- http://peterbraden.co.uk/article/django-inheritance 565- http://www.hopelessgeek.com/2009/11/25/a-hack-for-multi-table-inheritance-in-django 566- http://stackoverflow.com/questions/929029/how-do-i-access-the-child-classes-of-an-object-in-django-without-knowing-the-name/929982#929982 567- http://stackoverflow.com/questions/1581024/django-inheritance-how-to-have-one-method-for-all-subclasses 568- http://groups.google.com/group/django-users/browse_thread/thread/cbdaf2273781ccab/e676a537d735d9ef?lnk=gst&q=polymorphic#e676a537d735d9ef 569- http://groups.google.com/group/django-users/browse_thread/thread/52f72cffebb705e/bc18c18b2e83881e?lnk=gst&q=model+inheritance#bc18c18b2e83881e 570- http://code.djangoproject.com/ticket/10808 571- http://code.djangoproject.com/ticket/7270 572