PageRenderTime 59ms CodeModel.GetById 11ms RepoModel.GetById 1ms app.codeStats 0ms

Plain Text | 402 lines | 294 code | 108 blank | 0 comment | 0 complexity | dc66cfa0847aa81f948db19db6cb3179 MD5 | raw file
Possible License(s): BSD-3-Clause
  1. ==========================
  2. Serializing Django objects
  3. ==========================
  4. Django's serialization framework provides a mechanism for "translating" Django
  5. objects into other formats. Usually these other formats will be text-based and
  6. used for sending Django objects over a wire, but it's possible for a
  7. serializer to handle any format (text-based or not).
  8. .. seealso::
  9. If you just want to get some data from your tables into a serialized
  10. form, you could use the :djadmin:`dumpdata` management command.
  11. Serializing data
  12. ----------------
  13. At the highest level, serializing data is a very simple operation::
  14. from django.core import serializers
  15. data = serializers.serialize("xml", SomeModel.objects.all())
  16. The arguments to the ``serialize`` function are the format to serialize the data
  17. to (see `Serialization formats`_) and a :class:`~django.db.models.QuerySet` to
  18. serialize. (Actually, the second argument can be any iterator that yields Django
  19. objects, but it'll almost always be a QuerySet).
  20. You can also use a serializer object directly::
  21. XMLSerializer = serializers.get_serializer("xml")
  22. xml_serializer = XMLSerializer()
  23. xml_serializer.serialize(queryset)
  24. data = xml_serializer.getvalue()
  25. This is useful if you want to serialize data directly to a file-like object
  26. (which includes an :class:`~django.http.HttpResponse`)::
  27. out = open("file.xml", "w")
  28. xml_serializer.serialize(SomeModel.objects.all(), stream=out)
  29. Subset of fields
  30. ~~~~~~~~~~~~~~~~
  31. If you only want a subset of fields to be serialized, you can
  32. specify a ``fields`` argument to the serializer::
  33. from django.core import serializers
  34. data = serializers.serialize('xml', SomeModel.objects.all(), fields=('name','size'))
  35. In this example, only the ``name`` and ``size`` attributes of each model will
  36. be serialized.
  37. .. note::
  38. Depending on your model, you may find that it is not possible to
  39. deserialize a model that only serializes a subset of its fields. If a
  40. serialized object doesn't specify all the fields that are required by a
  41. model, the deserializer will not be able to save deserialized instances.
  42. Inherited Models
  43. ~~~~~~~~~~~~~~~~
  44. If you have a model that is defined using an :ref:`abstract base class
  45. <abstract-base-classes>`, you don't have to do anything special to serialize
  46. that model. Just call the serializer on the object (or objects) that you want to
  47. serialize, and the output will be a complete representation of the serialized
  48. object.
  49. However, if you have a model that uses :ref:`multi-table inheritance
  50. <multi-table-inheritance>`, you also need to serialize all of the base classes
  51. for the model. This is because only the fields that are locally defined on the
  52. model will be serialized. For example, consider the following models::
  53. class Place(models.Model):
  54. name = models.CharField(max_length=50)
  55. class Restaurant(Place):
  56. serves_hot_dogs = models.BooleanField()
  57. If you only serialize the Restaurant model::
  58. data = serializers.serialize('xml', Restaurant.objects.all())
  59. the fields on the serialized output will only contain the `serves_hot_dogs`
  60. attribute. The `name` attribute of the base class will be ignored.
  61. In order to fully serialize your Restaurant instances, you will need to
  62. serialize the Place models as well::
  63. all_objects = list(Restaurant.objects.all()) + list(Place.objects.all())
  64. data = serializers.serialize('xml', all_objects)
  65. Deserializing data
  66. ------------------
  67. Deserializing data is also a fairly simple operation::
  68. for obj in serializers.deserialize("xml", data):
  69. do_something_with(obj)
  70. As you can see, the ``deserialize`` function takes the same format argument as
  71. ``serialize``, a string or stream of data, and returns an iterator.
  72. However, here it gets slightly complicated. The objects returned by the
  73. ``deserialize`` iterator *aren't* simple Django objects. Instead, they are
  74. special ``DeserializedObject`` instances that wrap a created -- but unsaved --
  75. object and any associated relationship data.
  76. Calling ```` saves the object to the database.
  77. This ensures that deserializing is a non-destructive operation even if the
  78. data in your serialized representation doesn't match what's currently in the
  79. database. Usually, working with these ``DeserializedObject`` instances looks
  80. something like::
  81. for deserialized_object in serializers.deserialize("xml", data):
  82. if object_should_be_saved(deserialized_object):
  84. In other words, the usual use is to examine the deserialized objects to make
  85. sure that they are "appropriate" for saving before doing so. Of course, if you
  86. trust your data source you could just save the object and move on.
  87. The Django object itself can be inspected as ``deserialized_object.object``.
  88. .. _serialization-formats:
  89. Serialization formats
  90. ---------------------
  91. Django supports a number of serialization formats, some of which require you
  92. to install third-party Python modules:
  93. ========== ==============================================================
  94. Identifier Information
  95. ========== ==============================================================
  96. ``xml`` Serializes to and from a simple XML dialect.
  97. ``json`` Serializes to and from JSON_ (using a version of simplejson_
  98. bundled with Django).
  99. ``yaml`` Serializes to YAML (YAML Ain't a Markup Language). This
  100. serializer is only available if PyYAML_ is installed.
  101. ========== ==============================================================
  102. .. _json:
  103. .. _simplejson:
  104. .. _PyYAML:
  105. Notes for specific serialization formats
  106. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  107. json
  108. ^^^^
  109. If you're using UTF-8 (or any other non-ASCII encoding) data with the JSON
  110. serializer, you must pass ``ensure_ascii=False`` as a parameter to the
  111. ``serialize()`` call. Otherwise, the output won't be encoded correctly.
  112. For example::
  113. json_serializer = serializers.get_serializer("json")()
  114. json_serializer.serialize(queryset, ensure_ascii=False, stream=response)
  115. The Django source code includes the simplejson_ module. However, if you're
  116. using Python 2.6 or later (which includes a builtin version of the module), Django will
  117. use the builtin ``json`` module automatically. If you have a system installed
  118. version that includes the C-based speedup extension, or your system version is
  119. more recent than the version shipped with Django (currently, 2.0.7), the
  120. system version will be used instead of the version included with Django.
  121. Be aware that if you're serializing using that module directly, not all Django
  122. output can be passed unmodified to simplejson. In particular, :ref:`lazy
  123. translation objects <lazy-translations>` need a `special encoder`_ written for
  124. them. Something like this will work::
  125. from django.utils.functional import Promise
  126. from django.utils.encoding import force_unicode
  127. class LazyEncoder(simplejson.JSONEncoder):
  128. def default(self, obj):
  129. if isinstance(obj, Promise):
  130. return force_unicode(obj)
  131. return super(LazyEncoder, self).default(obj)
  132. .. _special encoder:
  133. .. _topics-serialization-natural-keys:
  134. Natural keys
  135. ------------
  136. .. versionadded:: 1.2
  137. The ability to use natural keys when serializing/deserializing data was
  138. added in the 1.2 release.
  139. The default serialization strategy for foreign keys and many-to-many
  140. relations is to serialize the value of the primary key(s) of the
  141. objects in the relation. This strategy works well for most types of
  142. object, but it can cause difficulty in some circumstances.
  143. Consider the case of a list of objects that have foreign key on
  144. :class:`ContentType`. If you're going to serialize an object that
  145. refers to a content type, you need to have a way to refer to that
  146. content type. Content Types are automatically created by Django as
  147. part of the database synchronization process, so you don't need to
  148. include content types in a fixture or other serialized data. As a
  149. result, the primary key of any given content type isn't easy to
  150. predict - it will depend on how and when :djadmin:`syncdb` was
  151. executed to create the content types.
  152. There is also the matter of convenience. An integer id isn't always
  153. the most convenient way to refer to an object; sometimes, a
  154. more natural reference would be helpful.
  155. It is for these reasons that Django provides *natural keys*. A natural
  156. key is a tuple of values that can be used to uniquely identify an
  157. object instance without using the primary key value.
  158. Deserialization of natural keys
  159. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  160. Consider the following two models::
  161. from django.db import models
  162. class Person(models.Model):
  163. first_name = models.CharField(max_length=100)
  164. last_name = models.CharField(max_length=100)
  165. birthdate = models.DateField()
  166. class Meta:
  167. unique_together = (('first_name', 'last_name'),)
  168. class Book(models.Model):
  169. name = models.CharField(max_length=100)
  170. author = models.ForeignKey(Person)
  171. Ordinarily, serialized data for ``Book`` would use an integer to refer to
  172. the author. For example, in JSON, a Book might be serialized as::
  173. ...
  174. {
  175. "pk": 1,
  176. "model": "",
  177. "fields": {
  178. "name": "Mostly Harmless",
  179. "author": 42
  180. }
  181. }
  182. ...
  183. This isn't a particularly natural way to refer to an author. It
  184. requires that you know the primary key value for the author; it also
  185. requires that this primary key value is stable and predictable.
  186. However, if we add natural key handling to Person, the fixture becomes
  187. much more humane. To add natural key handling, you define a default
  188. Manager for Person with a ``get_by_natural_key()`` method. In the case
  189. of a Person, a good natural key might be the pair of first and last
  190. name::
  191. from django.db import models
  192. class PersonManager(models.Manager):
  193. def get_by_natural_key(self, first_name, last_name):
  194. return self.get(first_name=first_name, last_name=last_name)
  195. class Person(models.Model):
  196. objects = PersonManager()
  197. first_name = models.CharField(max_length=100)
  198. last_name = models.CharField(max_length=100)
  199. birthdate = models.DateField()
  200. class Meta:
  201. unique_together = (('first_name', 'last_name'),)
  202. Now books can use that natural key to refer to ``Person`` objects::
  203. ...
  204. {
  205. "pk": 1,
  206. "model": "",
  207. "fields": {
  208. "name": "Mostly Harmless",
  209. "author": ["Douglas", "Adams"]
  210. }
  211. }
  212. ...
  213. When you try to load this serialized data, Django will use the
  214. ``get_by_natural_key()`` method to resolve ``["Douglas", "Adams"]``
  215. into the primary key of an actual ``Person`` object.
  216. .. note::
  217. Whatever fields you use for a natural key must be able to uniquely
  218. identify an object. This will usually mean that your model will
  219. have a uniqueness clause (either unique=True on a single field, or
  220. ``unique_together`` over multiple fields) for the field or fields
  221. in your natural key. However, uniqueness doesn't need to be
  222. enforced at the database level. If you are certain that a set of
  223. fields will be effectively unique, you can still use those fields
  224. as a natural key.
  225. Serialization of natural keys
  226. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  227. So how do you get Django to emit a natural key when serializing an object?
  228. Firstly, you need to add another method -- this time to the model itself::
  229. class Person(models.Model):
  230. objects = PersonManager()
  231. first_name = models.CharField(max_length=100)
  232. last_name = models.CharField(max_length=100)
  233. birthdate = models.DateField()
  234. def natural_key(self):
  235. return (self.first_name, self.last_name)
  236. class Meta:
  237. unique_together = (('first_name', 'last_name'),)
  238. That method should always return a natural key tuple -- in this
  239. example, ``(first name, last name)``. Then, when you call
  240. ``serializers.serialize()``, you provide a ``use_natural_keys=True``
  241. argument::
  242. >>> serializers.serialize('json', [book1, book2], indent=2, use_natural_keys=True)
  243. When ``use_natural_keys=True`` is specified, Django will use the
  244. ``natural_key()`` method to serialize any reference to objects of the
  245. type that defines the method.
  246. If you are using :djadmin:`dumpdata` to generate serialized data, you
  247. use the `--natural` command line flag to generate natural keys.
  248. .. note::
  249. You don't need to define both ``natural_key()`` and
  250. ``get_by_natural_key()``. If you don't want Django to output
  251. natural keys during serialization, but you want to retain the
  252. ability to load natural keys, then you can opt to not implement
  253. the ``natural_key()`` method.
  254. Conversely, if (for some strange reason) you want Django to output
  255. natural keys during serialization, but *not* be able to load those
  256. key values, just don't define the ``get_by_natural_key()`` method.
  257. Dependencies during serialization
  258. ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
  259. Since natural keys rely on database lookups to resolve references, it
  260. is important that data exists before it is referenced. You can't make
  261. a `forward reference` with natural keys - the data you are referencing
  262. must exist before you include a natural key reference to that data.
  263. To accommodate this limitation, calls to :djadmin:`dumpdata` that use
  264. the :djadminopt:`--natural` option will serialize any model with a
  265. ``natural_key()`` method before it serializes normal key objects.
  266. However, this may not always be enough. If your natural key refers to
  267. another object (by using a foreign key or natural key to another object
  268. as part of a natural key), then you need to be able to ensure that
  269. the objects on which a natural key depends occur in the serialized data
  270. before the natural key requires them.
  271. To control this ordering, you can define dependencies on your
  272. ``natural_key()`` methods. You do this by setting a ``dependencies``
  273. attribute on the ``natural_key()`` method itself.
  274. For example, consider the ``Permission`` model in ``contrib.auth``.
  275. The following is a simplified version of the ``Permission`` model::
  276. class Permission(models.Model):
  277. name = models.CharField(max_length=50)
  278. content_type = models.ForeignKey(ContentType)
  279. codename = models.CharField(max_length=100)
  280. # ...
  281. def natural_key(self):
  282. return (self.codename,) + self.content_type.natural_key()
  283. The natural key for a ``Permission`` is a combination of the codename for the
  284. ``Permission``, and the ``ContentType`` to which the ``Permission`` applies. This means
  285. that ``ContentType`` must be serialized before ``Permission``. To define this
  286. dependency, we add one extra line::
  287. class Permission(models.Model):
  288. # ...
  289. def natural_key(self):
  290. return (self.codename,) + self.content_type.natural_key()
  291. natural_key.dependencies = ['contenttypes.contenttype']
  292. This definition ensures that ``ContentType`` models are serialized before
  293. ``Permission`` models. In turn, any object referencing ``Permission`` will
  294. be serialized after both ``ContentType`` and ``Permission``.