/Doc/library/unicodedata.rst

http://unladen-swallow.googlecode.com/ · ReStructuredText · 170 lines · 109 code · 61 blank · 0 comment · 0 complexity · 2fca6bf728e8a338e6a835282cd5987f MD5 · raw file

  1. :mod:`unicodedata` --- Unicode Database
  2. =======================================
  3. .. module:: unicodedata
  4. :synopsis: Access the Unicode Database.
  5. .. moduleauthor:: Marc-Andre Lemburg <mal@lemburg.com>
  6. .. sectionauthor:: Marc-Andre Lemburg <mal@lemburg.com>
  7. .. sectionauthor:: Martin v. Lรถwis <martin@v.loewis.de>
  8. .. index::
  9. single: Unicode
  10. single: character
  11. pair: Unicode; database
  12. This module provides access to the Unicode Character Database which defines
  13. character properties for all Unicode characters. The data in this database is
  14. based on the :file:`UnicodeData.txt` file version 5.1.0 which is publicly
  15. available from ftp://ftp.unicode.org/.
  16. The module uses the same names and symbols as defined by the UnicodeData File
  17. Format 5.1.0 (see http://www.unicode.org/Public/5.1.0/ucd/UCD.html). It defines
  18. the following functions:
  19. .. function:: lookup(name)
  20. Look up character by name. If a character with the given name is found, return
  21. the corresponding Unicode character. If not found, :exc:`KeyError` is raised.
  22. .. function:: name(unichr[, default])
  23. Returns the name assigned to the Unicode character *unichr* as a string. If no
  24. name is defined, *default* is returned, or, if not given, :exc:`ValueError` is
  25. raised.
  26. .. function:: decimal(unichr[, default])
  27. Returns the decimal value assigned to the Unicode character *unichr* as integer.
  28. If no such value is defined, *default* is returned, or, if not given,
  29. :exc:`ValueError` is raised.
  30. .. function:: digit(unichr[, default])
  31. Returns the digit value assigned to the Unicode character *unichr* as integer.
  32. If no such value is defined, *default* is returned, or, if not given,
  33. :exc:`ValueError` is raised.
  34. .. function:: numeric(unichr[, default])
  35. Returns the numeric value assigned to the Unicode character *unichr* as float.
  36. If no such value is defined, *default* is returned, or, if not given,
  37. :exc:`ValueError` is raised.
  38. .. function:: category(unichr)
  39. Returns the general category assigned to the Unicode character *unichr* as
  40. string.
  41. .. function:: bidirectional(unichr)
  42. Returns the bidirectional category assigned to the Unicode character *unichr* as
  43. string. If no such value is defined, an empty string is returned.
  44. .. function:: combining(unichr)
  45. Returns the canonical combining class assigned to the Unicode character *unichr*
  46. as integer. Returns ``0`` if no combining class is defined.
  47. .. function:: east_asian_width(unichr)
  48. Returns the east asian width assigned to the Unicode character *unichr* as
  49. string.
  50. .. versionadded:: 2.4
  51. .. function:: mirrored(unichr)
  52. Returns the mirrored property assigned to the Unicode character *unichr* as
  53. integer. Returns ``1`` if the character has been identified as a "mirrored"
  54. character in bidirectional text, ``0`` otherwise.
  55. .. function:: decomposition(unichr)
  56. Returns the character decomposition mapping assigned to the Unicode character
  57. *unichr* as string. An empty string is returned in case no such mapping is
  58. defined.
  59. .. function:: normalize(form, unistr)
  60. Return the normal form *form* for the Unicode string *unistr*. Valid values for
  61. *form* are 'NFC', 'NFKC', 'NFD', and 'NFKD'.
  62. The Unicode standard defines various normalization forms of a Unicode string,
  63. based on the definition of canonical equivalence and compatibility equivalence.
  64. In Unicode, several characters can be expressed in various way. For example, the
  65. character U+00C7 (LATIN CAPITAL LETTER C WITH CEDILLA) can also be expressed as
  66. the sequence U+0327 (COMBINING CEDILLA) U+0043 (LATIN CAPITAL LETTER C).
  67. For each character, there are two normal forms: normal form C and normal form D.
  68. Normal form D (NFD) is also known as canonical decomposition, and translates
  69. each character into its decomposed form. Normal form C (NFC) first applies a
  70. canonical decomposition, then composes pre-combined characters again.
  71. In addition to these two forms, there are two additional normal forms based on
  72. compatibility equivalence. In Unicode, certain characters are supported which
  73. normally would be unified with other characters. For example, U+2160 (ROMAN
  74. NUMERAL ONE) is really the same thing as U+0049 (LATIN CAPITAL LETTER I).
  75. However, it is supported in Unicode for compatibility with existing character
  76. sets (e.g. gb2312).
  77. The normal form KD (NFKD) will apply the compatibility decomposition, i.e.
  78. replace all compatibility characters with their equivalents. The normal form KC
  79. (NFKC) first applies the compatibility decomposition, followed by the canonical
  80. composition.
  81. Even if two unicode strings are normalized and look the same to
  82. a human reader, if one has combining characters and the other
  83. doesn't, they may not compare equal.
  84. .. versionadded:: 2.3
  85. In addition, the module exposes the following constant:
  86. .. data:: unidata_version
  87. The version of the Unicode database used in this module.
  88. .. versionadded:: 2.3
  89. .. data:: ucd_3_2_0
  90. This is an object that has the same methods as the entire module, but uses the
  91. Unicode database version 3.2 instead, for applications that require this
  92. specific version of the Unicode database (such as IDNA).
  93. .. versionadded:: 2.5
  94. Examples:
  95. >>> import unicodedata
  96. >>> unicodedata.lookup('LEFT CURLY BRACKET')
  97. u'{'
  98. >>> unicodedata.name(u'/')
  99. 'SOLIDUS'
  100. >>> unicodedata.decimal(u'9')
  101. 9
  102. >>> unicodedata.decimal(u'a')
  103. Traceback (most recent call last):
  104. File "<stdin>", line 1, in ?
  105. ValueError: not a decimal
  106. >>> unicodedata.category(u'A') # 'L'etter, 'u'ppercase
  107. 'Lu'
  108. >>> unicodedata.bidirectional(u'\u0660') # 'A'rabic, 'N'umber
  109. 'AN'