PageRenderTime 44ms CodeModel.GetById 17ms RepoModel.GetById 1ms app.codeStats 0ms

/tags/rel-1-3-25/swigweb/article_cpp.ht

#
Unknown | 275 lines | 225 code | 50 blank | 0 comment | 0 complexity | ac73a5a0e01ee1dbd4005cc687d4999d MD5 | raw file
Possible License(s): LGPL-2.1, Cube, GPL-3.0, 0BSD, GPL-2.0
  1. Thoughts on the Insanity C++ Parsing
  2. <h2>Thoughts on the Insanity of C++ Parsing</h2>
  3. <center>
  4. <em>
  5. "Parsing C++ is simply too complex to do correctly." -- Anonymous
  6. </em>
  7. </center>
  8. <p>
  9. Author: David Beazley (beazley@cs.uchicago.edu)
  10. <p>
  11. August 12, 2002
  12. <p>
  13. A central goal of the SWIG project is to generate extension modules by
  14. parsing the contents of C++ header files. It's not too hard to come up
  15. with reasons why this might be useful---after all, if you've got
  16. several hundred class definitions, do you really want to go off and
  17. write a bunch of hand-crafted wrappers? No, of course not---you're
  18. busy and like everyone else, you've got better things to do with
  19. your time.
  20. <p>
  21. Okay, so there are many reasons why parsing C++ would be nice.
  22. However, parsing C++ is also a nightmare. In fact, C++ would
  23. probably the last language that any normal person would choose to
  24. serve as an interface specification language. It's hard to parse,
  25. hard to analyze, and it involves all sorts
  26. of nasty little problems related to scoping, typenames, templates,
  27. access, and so forth. Because of this, most of the tools that claim
  28. to "parse" C++ don't. Instead, they parse a subset of the language
  29. that happens to match the C++ programming style used by the tool's
  30. creator (believe me, I know---this is how SWIG started). Not
  31. surprisingly, these tools tend to break down when presented with code
  32. that starts to challenge the capabilities of the C++ compiler.
  33. Needless to say, critics see this as opportunity to make bold claims
  34. such as "writing a C++ parser is folly" or "this whole approach is too
  35. hard to ever work correctly."
  36. <p>
  37. Well, one does have to give the critics a little credit---writing a
  38. C++ parser certainly <em>is</em> hard and writing a parser that
  39. actually works correctly is even harder. However, these tasks are
  40. certainly not "impossible." After all, there would be no working C++
  41. compiler if such claims were true! Therefore, the question of whether
  42. or not a wrapper generator can parse C++ is clearly the wrong question
  43. to ask. Instead, the real question is whether or not a wrapper
  44. generation tool that parses C++ can actually do anything useful.
  45. <h3>The problem with using C++ as an interface definition language</h3>
  46. If you cut through all of the low-level details of parsing, the primary
  47. problem of using C++ as an module specification language is that of
  48. ambiguity. Consider a declaration like this:
  49. <blockquote>
  50. <pre>
  51. void foo(double *x, int n);
  52. </pre>
  53. </blockquote>
  54. If you look at this declaration, you can ask yourself the question,
  55. what is "x"? Is it a single input value? Is it an output value
  56. (modified by the function)? Is it an array? Is "n" somehow related?
  57. Perhaps the real problem in this example is that of expressing the
  58. programmer's intent. Yes, the function clearly accepts a pointer to
  59. some object and an integer, but the declaration does not contain
  60. enough additional information to determine the purpose of these
  61. parameters--information that could be useful in generating a suitable
  62. set of a wrappers.
  63. <p>
  64. IDL compilers associated with popular component frameworks (e.g.,
  65. CORBA, COM, etc.) get around this problem by requiring interfaces to
  66. be precisely specified--input and output values are clearly indicated
  67. as such. Thus, one might adopt a similar approach and extend C++
  68. syntax with some special modifiers or qualifiers. For example:
  69. <blockquote>
  70. <pre>
  71. void foo(%output double *x, int n);
  72. </pre>
  73. </blockquote>
  74. The problem with this approach is that it breaks from C++ syntax and
  75. it requires the user to annotate their input files (a task that C++
  76. wrapper generators are supposed to eliminate). Meanwhile, critics sit
  77. back and say "Ha! I told you C++ parsing would never work."
  78. <p>
  79. Another problem with using C++ as an input language is that interface
  80. building often involves more than just blindly wrapping declarations. For instance,
  81. users might want to rename declarations, specify exception handling procedures,
  82. add customized code, and so forth. This suggests that a
  83. wrapper generator really needs to do
  84. more than just parse C++---it must give users the freedom to customize
  85. various aspects of the wrapper generation process. Again, things aren't
  86. looking too good for C++.
  87. <h3>The SWIG approach: pattern matching</h3>
  88. SWIG takes a different approach to the C++ wrapping problem.
  89. Instead of trying to modify C++ with all sorts of little modifiers and
  90. add-ons, wrapping is largely controlled by a pattern matching mechanism that is
  91. built into the underlying C++ type system.
  92. <p>
  93. One part of the pattern matcher is programmed to look for specific sequences of
  94. datatypes and argument names. These patterns, known as typemaps, are
  95. responsible for all aspects of data conversion. They work by simply attaching
  96. bits of C conversion code to specific datatypes and argument names in the
  97. input file. For example, a typemap might be used like this:
  98. <blockquote>
  99. <pre>
  100. %typemap(in) <b>double *items</b> {
  101. // Get an array from the input
  102. ...
  103. }
  104. ...
  105. void foo(<b>double *items</b>, int n);
  106. </pre>
  107. </blockquote>
  108. With this approach, type and argument names are used as
  109. a basis for specifying customized wrapping behavior. For example, if a program
  110. always used an argument of <tt>double *items</tt> to refer to an
  111. array, SWIG can latch onto that and use it to provide customized
  112. processing. It is even possible to write pattern matching rules for
  113. sequences of arguments. For example, you could write the following:
  114. <blockquote>
  115. <pre>
  116. %typemap(in) (<b>double *items, int n</b>) {
  117. // Get an array of items. Set n to number of items
  118. ...
  119. }
  120. ...
  121. void foo(<b>double *items, int n</b>);
  122. </pre>
  123. </blockquote>
  124. The precise details of typemaps are not so important (in fact, most of
  125. this pattern matching is hidden from SWIG users). What is important
  126. is that pattern matching allows customized data handling to be
  127. specified without breaking C++ syntax--instead, a user merely has to
  128. define a few patterns that get applied across the declarations that
  129. appear in C++ header files. In some sense, you might view this
  130. approach as providing customization through naming conventions rather than
  131. having to annotate arguments with extra qualifiers.
  132. <p>
  133. The other pattern matching mechanism used by SWIG is a declaration annotator
  134. that is used to attach properties to specific declarations. A simple example of declaration
  135. annotation might be renaming. For example:
  136. <blockquote>
  137. <pre>
  138. %rename(cprint) print; // Rename all occurrences of 'print' to 'cprint'
  139. </pre>
  140. </blockquote>
  141. A more advanced form of declaration matching would be exception handling.
  142. For example:
  143. <blockquote>
  144. <pre>
  145. %exception Foo::getitem(int) {
  146. try {
  147. $action
  148. } catch (std::out_of_range&amp; e) {
  149. SWIG_exception(SWIG_IndexError,const_cast&lt;char*&gt;(e.what()));
  150. }
  151. }
  152. ...
  153. template&lt;class T&gt; class Foo {
  154. public:
  155. ...
  156. T &amp;getitem(int index); // Exception handling code attached
  157. ...
  158. };
  159. </pre>
  160. </blockquote>
  161. Like typemaps, declaration matching does not break from C++ syntax.
  162. Instead, a user merely specifies special processing rules in advance.
  163. These rules are then attached to any matching C++
  164. declaration that appears later in the input. This means that raw C++
  165. header files can often be parsed and customized with few, if any,
  166. modifications.
  167. <h3>The SWIG difference</h3>
  168. Pattern based approaches to wrapper code generation are not unique to SWIG.
  169. However, most prior efforts have based their pattern matching engines on simple
  170. regular-expression matching. The key difference between SWIG and these systems
  171. is that SWIG's customization features are fully integrated into the
  172. underlying C++ type system. This means that SWIG is able to deal with very
  173. complicated types of C/C++ code---especially code that makes heavy use of
  174. <tt>typedef</tt>, namespaces, aliases, class hierarchies, and more. To
  175. illustrate, consider some code like this:
  176. <blockquote>
  177. <pre>
  178. // A simple SWIG typemap
  179. %typemap(in) int {
  180. $1 = PyInt_AsLong($input);
  181. }
  182. ...
  183. // Some raw C++ code (included later)
  184. namespace X {
  185. typedef int Integer;
  186. class _FooImpl {
  187. public:
  188. typedef Integer value_type;
  189. };
  190. typedef _FooImpl Foo;
  191. }
  192. namespace Y = X;
  193. using Y::Foo;
  194. class Bar : public Foo {
  195. };
  196. void spam(Bar::value_type x);
  197. </pre>
  198. </blockquote>
  199. If you trace your way through this example, you will find that the
  200. <tt>Bar::value_type</tt> argument to function <tt>spam()</tt> is
  201. really an integer. What's more, if you take a close look at the SWIG
  202. generated wrappers, you will find that the typemap pattern defined for
  203. <tt>int</tt> is applied to it--in other words, SWIG does exactly the right thing despite
  204. our efforts to make the code confusing.
  205. <p>
  206. Similarly, declaration annotation is integrated into the type system
  207. and can be used to define properties that span inheritance hierarchies
  208. and more (in fact, there are many similarities between the operation of
  209. SWIG and tools developed for Aspect Oriented Programming).
  210. <h3>What does this mean?</h3>
  211. Pattern-based approaches allow wrapper generation tools to parse C++
  212. declarations and to provide a wide variety of high-level customization
  213. features. Although this approach is quite different than that found
  214. in a typical IDL, the use of patterns makes it possible to work from
  215. existing header files without having to make many (if any) changes to
  216. those files. Moreover, when the underlying pattern matching mechanism
  217. is integrated with the C++ type system, it is possible to build
  218. reliable wrappers to real software---even if that software is filled
  219. with namespaces, templates, classes, <tt>typedef</tt> declarations,
  220. pointers, and other bits of nastiness.
  221. <h3>The bottom line</h3>
  222. Not only is it possible to generate extension modules by parsing C++,
  223. it is possible to do so with real software and with a high degree of
  224. reliability. Don't believe me? Download SWIG-1.3.14 and try it for
  225. yourself.