/DOCS.rst

https://bitbucket.org/bconstantin/django_polymorphic/ · ReStructuredText · 572 lines · 403 code · 169 blank · 0 comment · 0 complexity · 3e1d7af599d39e636d51338e12ec579c MD5 · raw file

  1. Polymorphic Models for Django
  2. =============================
  3. .. contents:: Table of Contents
  4. :depth: 1
  5. Quickstart
  6. ===========
  7. Install
  8. -------
  9. After uncompressing (if necessary), in the directory "...django_polymorphic",
  10. execute (on Unix-like systems)::
  11. sudo python setup.py install
  12. Make Your Models Polymorphic
  13. ----------------------------
  14. Use ``PolymorphicModel`` instead of Django's ``models.Model``, like so::
  15. from polymorphic import PolymorphicModel
  16. class Project(PolymorphicModel):
  17. topic = models.CharField(max_length=30)
  18. class ArtProject(Project):
  19. artist = models.CharField(max_length=30)
  20. class ResearchProject(Project):
  21. supervisor = models.CharField(max_length=30)
  22. All models inheriting from your polymorphic models will be polymorphic as well.
  23. Create some objects
  24. -------------------
  25. >>> Project.objects.create(topic="Department Party")
  26. >>> ArtProject.objects.create(topic="Painting with Tim", artist="T. Turner")
  27. >>> ResearchProject.objects.create(topic="Swallow Aerodynamics", supervisor="Dr. Winter")
  28. Get polymorphic query results
  29. -----------------------------
  30. >>> Project.objects.all()
  31. [ <Project: id 1, topic "Department Party">,
  32. <ArtProject: id 2, topic "Painting with Tim", artist "T. Turner">,
  33. <ResearchProject: id 3, topic "Swallow Aerodynamics", supervisor "Dr. Winter"> ]
  34. use ``instance_of`` or ``not_instance_of`` for narrowing the result to specific subtypes:
  35. >>> Project.objects.instance_of(ArtProject)
  36. [ <ArtProject: id 2, topic "Painting with Tim", artist "T. Turner"> ]
  37. >>> Project.objects.instance_of(ArtProject) | Project.objects.instance_of(ResearchProject)
  38. [ <ArtProject: id 2, topic "Painting with Tim", artist "T. Turner">,
  39. <ResearchProject: id 3, topic "Swallow Aerodynamics", supervisor "Dr. Winter"> ]
  40. Polymorphic filtering: Get all projects where Mr. Turner is involved as an artist
  41. or supervisor (note the three underscores):
  42. >>> Project.objects.filter( Q(ArtProject___artist = 'T. Turner') | Q(ResearchProject___supervisor = 'T. Turner') )
  43. [ <ArtProject: id 2, topic "Painting with Tim", artist "T. Turner">,
  44. <ResearchProject: id 4, topic "Color Use in Late Cubism", supervisor "T. Turner"> ]
  45. This is basically all you need to know, as django_polymorphic mostly
  46. works fully automatic and just delivers the expected ("pythonic") results.
  47. Note: In all example output, above and below, for a nicer and more informative
  48. output the ``ShowFieldType`` mixin has been used (documented below).
  49. List of Features
  50. ================
  51. * Fully automatic - generally makes sure that the same objects are
  52. returned from the database that were stored there, regardless how
  53. they are retrieved
  54. * Only on models that request polymorphic behaviour (and the
  55. models inheriting from them)
  56. * Full support for ForeignKeys, ManyToManyFields and OneToToneFields
  57. * Filtering for classes, equivalent to python's isinstance():
  58. ``instance_of(...)`` and ``not_instance_of(...)``
  59. * Polymorphic filtering/ordering etc., allowing the use of fields of
  60. derived models ("ArtProject___artist")
  61. * Support for user-defined custom managers
  62. * Automatic inheritance of custom managers
  63. * Support for user-defined custom queryset classes
  64. * Non-polymorphic queries if needed, with no other change in
  65. features/behaviour
  66. * Combining querysets of different types/models ("qs3 = qs1 | qs2")
  67. * Nice/informative display of polymorphic queryset results
  68. More about Installation / Testing
  69. =================================
  70. Requirements
  71. ------------
  72. Django 1.1 (or later) and Python 2.4 or later. This code has been tested
  73. on Django 1.1 / 1.2 / 1.3 and Python 2.4.6 / 2.5.4 / 2.6.4 on Linux.
  74. Included Test Suite
  75. -------------------
  76. The repository (or tar file) contains a complete Django project
  77. that may be used for tests or experiments, without any installation needed.
  78. To run the included test suite, in the directory "...django_polymorphic" execute::
  79. ./manage test polymorphic
  80. The management command ``pcmd.py`` in the app ``pexp`` can be used
  81. for quick tests or experiments - modify this file (pexp/management/commands/pcmd.py)
  82. to your liking, then run::
  83. ./manage syncdb # db is created in /var/tmp/... (settings.py)
  84. ./manage pcmd
  85. Installation
  86. ------------
  87. In the directory "...django_polymorphic", execute ``sudo python setup.py install``.
  88. Alternatively you can simply copy the ``polymorphic`` subdirectory
  89. (under "django_polymorphic") into your Django project dir
  90. (e.g. if you want to distribute your project with more 'batteries included').
  91. If you want to run the test cases in `polymorphic/tests.py`, you need to add
  92. ``polymorphic`` to your INSTALLED_APPS setting.
  93. Django's ContentType framework (``django.contrib.contenttypes``)
  94. needs to be listed in INSTALLED_APPS (usually it already is).
  95. More Polymorphic Functionality
  96. ==============================
  97. In the examples below, these models are being used::
  98. from polymorphic import PolymorphicModel
  99. class ModelA(PolymorphicModel):
  100. field1 = models.CharField(max_length=10)
  101. class ModelB(ModelA):
  102. field2 = models.CharField(max_length=10)
  103. class ModelC(ModelB):
  104. field3 = models.CharField(max_length=10)
  105. Filtering for classes (equivalent to python's isinstance() ):
  106. -------------------------------------------------------------
  107. >>> ModelA.objects.instance_of(ModelB)
  108. .
  109. [ <ModelB: id 2, field1 (CharField), field2 (CharField)>,
  110. <ModelC: id 3, field1 (CharField), field2 (CharField), field3 (CharField)> ]
  111. In general, including or excluding parts of the inheritance tree::
  112. ModelA.objects.instance_of(ModelB [, ModelC ...])
  113. ModelA.objects.not_instance_of(ModelB [, ModelC ...])
  114. You can also use this feature in Q-objects (with the same result as above):
  115. >>> ModelA.objects.filter( Q(instance_of=ModelB) )
  116. Polymorphic filtering (for fields in derived classes)
  117. -----------------------------------------------------
  118. For example, cherrypicking objects from multiple derived classes
  119. anywhere in the inheritance tree, using Q objects (with the
  120. syntax: ``exact model name + three _ + field name``):
  121. >>> ModelA.objects.filter( Q(ModelB___field2 = 'B2') | Q(ModelC___field3 = 'C3') )
  122. .
  123. [ <ModelB: id 2, field1 (CharField), field2 (CharField)>,
  124. <ModelC: id 3, field1 (CharField), field2 (CharField), field3 (CharField)> ]
  125. Combining Querysets
  126. -------------------
  127. Querysets could now be regarded as object containers that allow the
  128. aggregation of different object types, very similar to python
  129. lists - as long as the objects are accessed through the manager of
  130. a common base class:
  131. >>> Base.objects.instance_of(ModelX) | Base.objects.instance_of(ModelY)
  132. .
  133. [ <ModelX: id 1, field_x (CharField)>,
  134. <ModelY: id 2, field_y (CharField)> ]
  135. ManyToManyField, ForeignKey, OneToOneField
  136. ------------------------------------------
  137. Relationship fields referring to polymorphic models work as
  138. expected: like polymorphic querysets they now always return the
  139. referred objects with the same type/class these were created and
  140. saved as.
  141. E.g., if in your model you define::
  142. field1 = OneToOneField(ModelA)
  143. then field1 may now also refer to objects of type ``ModelB`` or ``ModelC``.
  144. A ManyToManyField example::
  145. # The model holding the relation may be any kind of model, polymorphic or not
  146. class RelatingModel(models.Model):
  147. many2many = models.ManyToManyField('ModelA') # ManyToMany relation to a polymorphic model
  148. >>> o=RelatingModel.objects.create()
  149. >>> o.many2many.add(ModelA.objects.get(id=1))
  150. >>> o.many2many.add(ModelB.objects.get(id=2))
  151. >>> o.many2many.add(ModelC.objects.get(id=3))
  152. >>> o.many2many.all()
  153. [ <ModelA: id 1, field1 (CharField)>,
  154. <ModelB: id 2, field1 (CharField), field2 (CharField)>,
  155. <ModelC: id 3, field1 (CharField), field2 (CharField), field3 (CharField)> ]
  156. Using Third Party Models (without modifying them)
  157. -------------------------------------------------
  158. Third party models can be used as polymorphic models without
  159. restrictions by subclassing them. E.g. using a third party
  160. model as the root of a polymorphic inheritance tree::
  161. from thirdparty import ThirdPartyModel
  162. class MyThirdPartyBaseModel(PolymorhpicModel, ThirdPartyModel):
  163. pass # or add fields
  164. Or instead integrating the third party model anywhere into an
  165. existing polymorphic inheritance tree::
  166. class MyBaseModel(SomePolymorphicModel):
  167. my_field = models.CharField(max_length=10)
  168. class MyModelWithThirdParty(MyBaseModel, ThirdPartyModel):
  169. pass # or add fields
  170. Non-Polymorphic Queries
  171. -----------------------
  172. If you insert ``.non_polymorphic()`` anywhere into the query chain, then
  173. django_polymorphic will simply leave out the final step of retrieving the
  174. real objects, and the manager/queryset will return objects of the type of
  175. the base class you used for the query, like vanilla Django would
  176. (``ModelA`` in this example).
  177. >>> qs=ModelA.objects.non_polymorphic().all()
  178. >>> qs
  179. [ <ModelA: id 1, field1 (CharField)>,
  180. <ModelA: id 2, field1 (CharField)>,
  181. <ModelA: id 3, field1 (CharField)> ]
  182. There are no other changes in the behaviour of the queryset. For example,
  183. enhancements for ``filter()`` or ``instance_of()`` etc. still work as expected.
  184. If you do the final step yourself, you get the usual polymorphic result:
  185. >>> ModelA.objects.get_real_instances(qs)
  186. [ <ModelA: id 1, field1 (CharField)>,
  187. <ModelB: id 2, field1 (CharField), field2 (CharField)>,
  188. <ModelC: id 3, field1 (CharField), field2 (CharField), field3 (CharField)> ]
  189. About Queryset Methods
  190. ----------------------
  191. * ``annotate()`` and ``aggregate()`` work just as usual, with the
  192. addition that the ``ModelX___field`` syntax can be used for the
  193. keyword arguments (but not for the non-keyword arguments).
  194. * ``order_by()`` now similarly supports the ``ModelX___field`` syntax
  195. for specifying ordering through a field in a submodel.
  196. * ``distinct()`` works as expected. It only regards the fields of
  197. the base class, but this should never make a difference.
  198. * ``select_related()`` works just as usual, but it can not (yet) be used
  199. to select relations in derived models
  200. (like ``ModelA.objects.select_related('ModelC___fieldxy')`` )
  201. * ``extra()`` works as expected (it returns polymorphic results) but
  202. currently has one restriction: The resulting objects are required to have
  203. a unique primary key within the result set - otherwise an error is thrown
  204. (this case could be made to work, however it may be mostly unneeded)..
  205. The keyword-argument "polymorphic" is no longer supported.
  206. You can get back the old non-polymorphic behaviour (before V1.0)
  207. by using ``ModelA.objects.non_polymorphic().extra(...)``.
  208. * ``get_real_instances()`` allows you to turn a
  209. queryset or list of base model objects efficiently into the real objects.
  210. For example, you could do ``base_objects_queryset=ModelA.extra(...).non_polymorphic()``
  211. and then call ``real_objects=base_objects_queryset.get_real_instances()``.Or alternatively
  212. .``real_objects=ModelA.objects..get_real_instances(base_objects_queryset_or_object_list)``
  213. * ``values()`` & ``values_list()`` currently do not return polymorphic
  214. results. This may change in the future however. If you want to use these
  215. methods now, it's best if you use ``Model.base_objects.values...`` as
  216. this is guaranteed to not change.
  217. * ``defer()`` and ``only()`` are not yet supported (support will be added
  218. in the future).
  219. Using enhanced Q-objects in any Places
  220. --------------------------------------
  221. The queryset enhancements (e.g. ``instance_of``) only work as arguments
  222. to the member functions of a polymorphic queryset. Occationally it may
  223. be useful to be able to use Q objects with these enhancements in other places.
  224. As Django doesn't understand these enhanced Q objects, you need to
  225. transform them manually into normal Q objects before you can feed them
  226. to a Django queryset or function::
  227. normal_q_object = ModelA.translate_polymorphic_Q_object( Q(instance_of=Model2B) )
  228. This function cannot be used at model creation time however (in models.py),
  229. as it may need to access the ContentTypes database table.
  230. Nicely Displaying Polymorphic Querysets
  231. ---------------------------------------
  232. In order to get the output as seen in all examples here, you need to use the
  233. ShowFieldType class mixin::
  234. from polymorphic import PolymorphicModel, ShowFieldType
  235. class ModelA(ShowFieldType, PolymorphicModel):
  236. field1 = models.CharField(max_length=10)
  237. You may also use ShowFieldContent or ShowFieldTypeAndContent to display
  238. additional information when printing querysets (or converting them to text).
  239. When showing field contents, they will be truncated to 20 characters. You can
  240. modify this behaviour by setting a class variable in your model like this::
  241. class ModelA(ShowFieldType, PolymorphicModel):
  242. polymorphic_showfield_max_field_width = 20
  243. ...
  244. Similarly, pre-V1.0 output formatting can be re-estated by using
  245. ``polymorphic_showfield_old_format = True``.
  246. Custom Managers, Querysets & Manager Inheritance
  247. ================================================
  248. Using a Custom Manager
  249. ----------------------
  250. A nice feature of Django is the possibility to define one's own custom object managers.
  251. This is fully supported with django_polymorphic: For creating a custom polymorphic
  252. manager class, just derive your manager from ``PolymorphicManager`` instead of
  253. ``models.Manager``. As with vanilla Django, in your model class, you should
  254. explicitly add the default manager first, and then your custom manager::
  255. from polymorphic import PolymorphicModel, PolymorphicManager
  256. class TimeOrderedManager(PolymorphicManager):
  257. def get_query_set(self):
  258. qs = super(TimeOrderedManager,self).get_query_set()
  259. return qs.order_by('-start_date') # order the queryset
  260. def most_recent(self):
  261. qs = self.get_query_set() # get my ordered queryset
  262. return qs[:10] # limit => get ten most recent entries
  263. class Project(PolymorphicModel):
  264. objects = PolymorphicManager() # add the default polymorphic manager first
  265. objects_ordered = TimeOrderedManager() # then add your own manager
  266. start_date = DateTimeField() # project start is this date/time
  267. The first manager defined ('objects' in the example) is used by
  268. Django as automatic manager for several purposes, including accessing
  269. related objects. It must not filter objects and it's safest to use
  270. the plain ``PolymorphicManager`` here.
  271. Manager Inheritance
  272. -------------------
  273. Polymorphic models inherit/propagate all managers from their
  274. base models, as long as these are polymorphic. This means that all
  275. managers defined in polymorphic base models continue to work as
  276. expected in models inheriting from this base model::
  277. from polymorphic import PolymorphicModel, PolymorphicManager
  278. class TimeOrderedManager(PolymorphicManager):
  279. def get_query_set(self):
  280. qs = super(TimeOrderedManager,self).get_query_set()
  281. return qs.order_by('-start_date') # order the queryset
  282. def most_recent(self):
  283. qs = self.get_query_set() # get my ordered queryset
  284. return qs[:10] # limit => get ten most recent entries
  285. class Project(PolymorphicModel):
  286. objects = PolymorphicManager() # add the default polymorphic manager first
  287. objects_ordered = TimeOrderedManager() # then add your own manager
  288. start_date = DateTimeField() # project start is this date/time
  289. class ArtProject(Project): # inherit from Project, inheriting its fields and managers
  290. artist = models.CharField(max_length=30)
  291. ArtProject inherited the managers ``objects`` and ``objects_ordered`` from Project.
  292. ``ArtProject.objects_ordered.all()`` will return all art projects ordered
  293. regarding their start time and ``ArtProject.objects_ordered.most_recent()``
  294. will return the ten most recent art projects.
  295. .
  296. Using a Custom Queryset Class
  297. -----------------------------
  298. The ``PolymorphicManager`` class accepts one initialization argument,
  299. which is the queryset class the manager should use. Just as with vanilla Django,
  300. you may define your own custom queryset classes. Just use PolymorphicQuerySet
  301. instead of Django's QuerySet as the base class::
  302. from polymorphic import PolymorphicModel, PolymorphicManager, PolymorphicQuerySet
  303. class MyQuerySet(PolymorphicQuerySet):
  304. def my_queryset_method(...):
  305. ...
  306. class MyModel(PolymorphicModel):
  307. my_objects=PolymorphicManager(MyQuerySet)
  308. ...
  309. Performance Considerations
  310. ==========================
  311. The current implementation is rather simple and does not use any
  312. custom SQL or Django DB layer internals - it is purely based on the
  313. standard Django ORM.
  314. Specifically, the query::
  315. result_objects = list( ModelA.objects.filter(...) )
  316. performs one SQL query to retrieve ``ModelA`` objects and one additional
  317. query for each unique derived class occurring in result_objects.
  318. The best case for retrieving 100 objects is 1 SQL query if all are
  319. class ``ModelA``. If 50 objects are ``ModelA`` and 50 are ``ModelB``, then
  320. two queries are executed. The pathological worst case is 101 db queries if
  321. result_objects contains 100 different object types (with all of them
  322. subclasses of ``ModelA``).
  323. Usually, when Django users create their own polymorphic ad-hoc solution
  324. without a tool like django_polymorphic, this usually results in a variation of ::
  325. result_objects = [ o.get_real_instance() for o in BaseModel.objects.filter(...) ]
  326. which has very bad performance, as it introduces one additional
  327. SQL query for every object in the result which is not of class ``BaseModel``.
  328. Compared to these solutions, django_polymorphic has the advantage
  329. that it only needs one sql request per *object type*, and not *per object*.
  330. .. _performance:
  331. Performance Problems with PostgreSQL, MySQL and SQLite3
  332. -------------------------------------------------------
  333. Current relational DBM systems seem to have general problems with
  334. the SQL queries produced by object relational mappers like the Django
  335. ORM, if these use multi-table inheritance like Django's ORM does.
  336. The "inner joins" in these queries can perform very badly.
  337. This is independent of django_polymorphic and affects all uses of
  338. multi table Model inheritance.
  339. Concrete benchmark results are forthcoming (please see discussion forum).
  340. Please also see this `post (and comments) from Jacob Kaplan-Moss`_.
  341. .. _post (and comments) from Jacob Kaplan-Moss: http://www.jacobian.org/writing/concrete-inheritance/
  342. .. _restrictions:
  343. Restrictions & Caveats
  344. ======================
  345. * Database Performance regarding concrete Model inheritance in general.
  346. Please see "Performance Problems" above.
  347. * Queryset methods ``values()``, ``values_list()``, ``select_related()``,
  348. ``defer()`` and ``only()`` are not yet fully supported (see above).
  349. ``extra()`` has one restriction: the resulting objects are required to have
  350. a unique primary key within the result set.
  351. * Django Admin Integration: There currently is no specific admin integration,
  352. but it would most likely make sense to have one.
  353. * Diamond shaped inheritance: There seems to be a general problem
  354. with diamond shaped multiple model inheritance with Django models
  355. (tested with V1.1 - V1.3).
  356. An example is here: http://code.djangoproject.com/ticket/10808.
  357. This problem is aggravated when trying to enhance models.Model
  358. by subclassing it instead of modifying Django core (as we do here
  359. with PolymorphicModel).
  360. * The enhanced filter-definitions/Q-objects only work as arguments
  361. for the methods of the polymorphic querysets. Please see above
  362. for ``translate_polymorphic_Q_object``.
  363. * A reference (``ContentType``) to the real/leaf model is stored
  364. in the base model (the base model directly inheriting from
  365. PolymorphicModel). You need to be aware of this when using the
  366. ``dumpdata`` management command or any other low-level
  367. database operations. E.g. if you rename models or apps or copy
  368. objects from one database to another, then Django's ContentType
  369. table needs to be corrected/copied too. This is of course generally
  370. the case for any models using Django's ContentType.
  371. * Django 1.1 only - the names of polymorphic models must be unique
  372. in the whole project, even if they are in two different apps.
  373. This results from a restriction in the Django 1.1 "related_name"
  374. option (fixed in Django 1.2).
  375. * Django 1.1 only - when ContentType is used in models, Django's
  376. seralisation or fixtures cannot be used (all polymorphic models
  377. use ContentType). This issue seems to be resolved for Django 1.2
  378. (changeset 11863: Fixed #7052, Added support for natural keys in serialization).
  379. + http://code.djangoproject.com/ticket/7052
  380. + http://stackoverflow.com/questions/853796/problems-with-contenttypes-when-loading-a-fixture-in-django
  381. Project Status
  382. ==============
  383. Django_polymorphic works well for a considerable number of users now,
  384. and no major problems have shown up for many months.
  385. The API can be considered stable beginning with the V1.0 release.
  386. Links
  387. =====
  388. - http://code.djangoproject.com/wiki/ModelInheritance
  389. - http://lazypython.blogspot.com/2009/02/second-look-at-inheritance-and.html
  390. - http://www.djangosnippets.org/snippets/1031/
  391. - http://www.djangosnippets.org/snippets/1034/
  392. - http://groups.google.com/group/django-developers/browse_frm/thread/7d40ad373ebfa912/a20fabc661b7035d?lnk=gst&q=model+inheritance+CORBA#a20fabc661b7035d
  393. - http://groups.google.com/group/django-developers/browse_thread/thread/9bc2aaec0796f4e0/0b92971ffc0aa6f8?lnk=gst&q=inheritance#0b92971ffc0aa6f8
  394. - http://groups.google.com/group/django-developers/browse_thread/thread/3947c594100c4adb/d8c0af3dacad412d?lnk=gst&q=inheritance#d8c0af3dacad412d
  395. - http://groups.google.com/group/django-users/browse_thread/thread/52f72cffebb705e/b76c9d8c89a5574f
  396. - http://peterbraden.co.uk/article/django-inheritance
  397. - http://www.hopelessgeek.com/2009/11/25/a-hack-for-multi-table-inheritance-in-django
  398. - http://stackoverflow.com/questions/929029/how-do-i-access-the-child-classes-of-an-object-in-django-without-knowing-the-name/929982#929982
  399. - http://stackoverflow.com/questions/1581024/django-inheritance-how-to-have-one-method-for-all-subclasses
  400. - http://groups.google.com/group/django-users/browse_thread/thread/cbdaf2273781ccab/e676a537d735d9ef?lnk=gst&q=polymorphic#e676a537d735d9ef
  401. - http://groups.google.com/group/django-users/browse_thread/thread/52f72cffebb705e/bc18c18b2e83881e?lnk=gst&q=model+inheritance#bc18c18b2e83881e
  402. - http://code.djangoproject.com/ticket/10808
  403. - http://code.djangoproject.com/ticket/7270