/fparsec/main/Doc/html/reference/charstream.html

http://github.com/sandersn/fing · HTML · 1839 lines · 1804 code · 29 blank · 6 comment · 0 complexity · 0eb8354014409e44363fac4b55d0b5f9 MD5 · raw file

Large files are truncated click here to view the full file

  1. <!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
  2. <html xmlns="http://www.w3.org/1999/xhtml">
  3. <head>
  4. <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
  5. <title>FParsec.CharStream</title>
  6. <link rel="stylesheet" type="text/css" media="all" href="../css/style.css" />
  7. <link rel="stylesheet" type="text/css" media="screen" href="../css/screen-sidebar.css" title="Sidebar" />
  8. <link rel="alternate stylesheet" type="text/css" media="screen" href="../css/screen-no-sidebar.css" title="No sidebar" />
  9. <!--[if IE]>
  10. <link rel="stylesheet" type="text/css" media="all" href="../css/style-ie.css" />
  11. <![endif]-->
  12. <!--[if IE 6]>
  13. <link rel="stylesheet" type="text/css" media="all" href="../css/style-ie6.css" />
  14. <![endif]-->
  15. <link rel="stylesheet" type="text/css" media="print" href="../css/print.css" />
  16. </head>
  17. <body>
  18. <div id="fixed-layer">
  19. <div id="fixed-wrapper">
  20. <div id="sidebar">
  21. <div id="breadcrumbs-dummy"><span class="breadcrumbs">&nbsp;</span></div>
  22. <div id="nav-tree">
  23. <table class="nav n1">
  24. <tbody class="nav-open n1">
  25. <tr class="nav-entry n1 _1">
  26. <td class="nav-number n1"></td>
  27. <td class="nav-title n1"><a href="../index.html">FParsec Documentation</a></td>
  28. </tr>
  29. <tr class="nav-sub-entries n1 _1">
  30. <td class="nav-sub-entries-number n1"></td>
  31. <td class="nav-sub-entries n1">
  32. <table class="nav n2">
  33. <tbody class="nav-before-open n2">
  34. <tr class="nav-entry n2 _1">
  35. <td class="nav-number n2"><a href="../tutorial.html"><span class="section-number">1</span><span class="nav-space"></span></a></td>
  36. <td class="nav-title n2"><a href="../tutorial.html">Tutorial</a></td>
  37. </tr>
  38. </tbody>
  39. <tbody class="nav-open n2">
  40. <tr class="nav-entry n2 _2">
  41. <td class="nav-number n2"><a href="index.html"><span class="section-number">2</span><span class="nav-space"></span></a></td>
  42. <td class="nav-title n2"><a href="index.html">Reference</a></td>
  43. </tr>
  44. <tr class="nav-sub-entries n2 _2">
  45. <td class="nav-sub-entries-number n2"></td>
  46. <td class="nav-sub-entries n2">
  47. <table class="nav n3">
  48. <tbody class="nav-before-open n3">
  49. <tr class="nav-entry n3 _1">
  50. <td class="nav-number n3"><a href="parser-overview.html"><span class="section-number">1</span><span class="nav-space"></span></a></td>
  51. <td class="nav-title n3"><a href="parser-overview.html">Parser overview</a></td>
  52. </tr>
  53. <tr class="nav-entry n3 _2">
  54. <td class="nav-number n3"><a href="primitives.html"><span class="section-number">2</span><span class="nav-space"></span></a></td>
  55. <td class="nav-title n3"><a href="primitives.html">FParsec.Primitives</a></td>
  56. </tr>
  57. <tr class="nav-entry n3 _3">
  58. <td class="nav-number n3"><a href="charparsers.html"><span class="section-number">3</span><span class="nav-space"></span></a></td>
  59. <td class="nav-title n3"><a href="charparsers.html">FParsec.CharParsers</a></td>
  60. </tr>
  61. <tr class="nav-entry n3 _4">
  62. <td class="nav-number n3">
  63. <a href="operatorprecedenceparser.html"><span class="section-number">4</span><span class="nav-space"></span></a>
  64. </td>
  65. <td class="nav-title n3"><a href="operatorprecedenceparser.html">FParsec.OperatorPrecedenceParser</a></td>
  66. </tr>
  67. <tr class="nav-entry n3 _5">
  68. <td class="nav-number n3"><a href="error.html"><span class="section-number">5</span><span class="nav-space"></span></a></td>
  69. <td class="nav-title n3"><a href="error.html">FParsec.Error</a></td>
  70. </tr>
  71. <tr class="nav-entry n3 _6">
  72. <td class="nav-number n3"><a href="state.html"><span class="section-number">6</span><span class="nav-space"></span></a></td>
  73. <td class="nav-title n3"><a href="state.html">FParsec.State</a></td>
  74. </tr>
  75. <tr class="nav-entry n3 _7">
  76. <td class="nav-number n3"><a href="position.html"><span class="section-number">7</span><span class="nav-space"></span></a></td>
  77. <td class="nav-title n3"><a href="position.html">FParsec.Position</a></td>
  78. </tr>
  79. </tbody>
  80. <tbody class="nav-open selected n3">
  81. <tr class="nav-entry selected n3 _8">
  82. <td class="nav-number selected n3"><a href="#"><span class="section-number">8</span><span class="nav-space"></span></a></td>
  83. <td class="nav-title selected n3"><a href="#">FParsec.CharStream</a></td>
  84. </tr>
  85. <tr class="nav-sub-entries selected n3 _8">
  86. <td class="nav-sub-entries-number selected n3"></td>
  87. <td class="nav-sub-entries selected n3">
  88. <table class="nav n4">
  89. <tbody class="nav-before-open n4">
  90. <tr class="nav-entry n4 _1">
  91. <td class="nav-number n4"><a href="#interface"><span class="section-number">1</span><span class="nav-space"></span></a></td>
  92. <td class="nav-title n4"><a href="#interface">Interface</a></td>
  93. </tr>
  94. <tr class="nav-entry n4 _2">
  95. <td class="nav-number n4"><a href="#remarks"><span class="section-number">2</span><span class="nav-space"></span></a></td>
  96. <td class="nav-title n4"><a href="#remarks">Remarks</a></td>
  97. </tr>
  98. <tr class="nav-entry n4 _3">
  99. <td class="nav-number n4"><a href="#exceptions"><span class="section-number">3</span><span class="nav-space"></span></a></td>
  100. <td class="nav-title n4"><a href="#exceptions">I/O exceptions</a></td>
  101. </tr>
  102. <tr class="nav-entry n4 _4">
  103. <td class="nav-number n4"><a href="#members"><span class="section-number">4</span><span class="nav-space"></span></a></td>
  104. <td class="nav-title n4"><a href="#members">Members</a></td>
  105. </tr>
  106. <tr class="nav-entry n4 _5">
  107. <td class="nav-number n4"><a href="#Iterator"><span class="section-number">5</span><span class="nav-space"></span></a></td>
  108. <td class="nav-title n4"><a href="#Iterator">CharStream.Iterator</a></td>
  109. </tr>
  110. <tr class="nav-sub-entries n4 _5">
  111. <td class="nav-sub-entries-number n4"></td>
  112. <td class="nav-sub-entries n4">
  113. <table class="nav n5">
  114. <tbody class="nav-before-open n5">
  115. <tr class="nav-entry n5 _1">
  116. <td class="nav-number n5">
  117. <a href="#Iterator.interface"><span class="section-number">1</span><span class="nav-space"></span></a>
  118. </td>
  119. <td class="nav-title n5"><a href="#Iterator.interface">Interface</a></td>
  120. </tr>
  121. <tr class="nav-entry n5 _2">
  122. <td class="nav-number n5">
  123. <a href="#Iterator.members"><span class="section-number">2</span><span class="nav-space"></span></a>
  124. </td>
  125. <td class="nav-title n5"><a href="#Iterator.members">Members</a></td>
  126. </tr>
  127. </tbody>
  128. </table>
  129. </td>
  130. </tr>
  131. </tbody>
  132. </table>
  133. </td>
  134. </tr>
  135. </tbody>
  136. </table>
  137. </td>
  138. </tr>
  139. </tbody>
  140. </table>
  141. </td>
  142. </tr>
  143. </tbody>
  144. </table>
  145. </div>
  146. </div>
  147. </div>
  148. </div>
  149. <div id="wrapper">
  150. <div id="main">
  151. <div id="main-content">
  152. <div id="breadcrumbs">
  153. <span class="breadcrumbs">
  154. <span id="breadcrumbs-parents"><a href="../index.html">FParsec Documentation</a><span class="breadcrumbs-sep"> > </span><a href="index.html">Reference</a></span><span class="breadcrumbs-sep"> > </span>FParsec.CharStream
  155. </span>
  156. </div>
  157. <div class="section s2">
  158. <h1 class="title h2"><span class="section-number">2.8</span> FParsec.CharStream</h1>
  159. <div class="intro i2">
  160. <p class="para _1">
  161. The <code class="fsharp"><span class="ci">CharStream</span></code> class provides a unified interface for reading the UTF16 char content from
  162. binary streams or inmemory char buffers.
  163. </p>
  164. </div>
  165. <div id="interface" class="section s3">
  166. <h2 class="title h3"><span class="section-number">2.8.1</span> Interface</h2>
  167. <div class="intro i3">
  168. <div class="para _1">
  169. <pre class="code fsharp"><span class="ck">namespace</span> <span class="ci">FParsec</span>
  170. <span class="cp">[&lt;</span><span class="ci">Sealed</span><span class="cp">&gt;]</span>
  171. <span class="ck">type</span> <span class="ci">CharStream</span> <span class="cp">=</span>
  172. <span class="ck">interface</span> <span class="ci">System</span><span class="cm">.</span><span class="ci">IDisposable</span>
  173. <a id="interface.new_string:B:" href="#members.new_string"><span class="interface-member-marker"><span class="ck">new</span></span></a><span class="cp">:</span> <span class="ci">chars</span><span class="cp">:</span> <span class="ci">string</span> <span class="cp">*</span> <span class="ci">index</span><span class="cp">:</span> <span class="ci">int</span> <span class="cp">*</span> <span class="ci">length</span><span class="cp">:</span> <span class="ci">int</span>
  174. <span class="cr">-&gt;</span> <span class="ci">CharStream</span>
  175. <a id="interface.new_string_offset:B:" href="#members.new_string_offset"><span class="interface-member-marker"><span class="ck">new</span></span></a><span class="cp">:</span> <span class="ci">chars</span><span class="cp">:</span> <span class="ci">string</span> <span class="cp">*</span> <span class="ci">index</span><span class="cp">:</span> <span class="ci">int</span> <span class="cp">*</span> <span class="ci">length</span><span class="cp">:</span> <span class="ci">int</span> <span class="cp">*</span> <span class="ci">streamBeginIndex</span><span class="cp">:</span> <span class="ci">int64</span>
  176. <span class="cr">-&gt;</span> <span class="ci">CharStream</span>
  177. <a id="interface.new_char-array:B:" href="#members.new_char-array"><span class="interface-member-marker"><span class="ck">new</span></span></a><span class="cp">:</span> <span class="ci">chars</span><span class="cp">:</span> <span class="ci">char</span><span class="cp">[]</span> <span class="cp">*</span> <span class="ci">index</span><span class="cp">:</span> <span class="ci">int</span> <span class="cp">*</span> <span class="ci">length</span><span class="cp">:</span> <span class="ci">int</span>
  178. <span class="cr">-&gt;</span> <span class="ci">CharStream</span>
  179. <a id="interface.new_char-array_offset:B:" href="#members.new_char-array_offset"><span class="interface-member-marker"><span class="ck">new</span></span></a><span class="cp">:</span> <span class="ci">chars</span><span class="cp">:</span> <span class="ci">char</span><span class="cp">[]</span> <span class="cp">*</span> <span class="ci">index</span><span class="cp">:</span> <span class="ci">int</span> <span class="cp">*</span> <span class="ci">length</span><span class="cp">:</span> <span class="ci">int</span> <span class="cp">*</span> <span class="ci">streamBeginIndex</span><span class="cp">:</span> <span class="ci">int64</span>
  180. <span class="cr">-&gt;</span> <span class="ci">CharStream</span>
  181. <a id="interface.new_char-pointer:B:" href="#members.new_char-pointer"><span class="interface-member-marker"><span class="ck">new</span></span></a><span class="cp">:</span> <span class="ci">chars</span><span class="cp">:</span> <span class="ci">NativePtr</span><span class="cp">&lt;</span><span class="ci">char</span><span class="cp">&gt;</span> <span class="cp">*</span> <span class="ci">length</span><span class="cp">:</span> <span class="ci">int</span>
  182. <span class="cr">-&gt;</span> <span class="ci">CharStream</span>
  183. <a id="interface.new_char-pointer_offset:B:" href="#members.new_char-pointer_offset"><span class="interface-member-marker"><span class="ck">new</span></span></a><span class="cp">:</span> <span class="ci">chars</span><span class="cp">:</span> <span class="ci">NativePtr</span><span class="cp">&lt;</span><span class="ci">char</span><span class="cp">&gt;</span> <span class="cp">*</span> <span class="ci">length</span><span class="cp">:</span> <span class="ci">int</span> <span class="cp">*</span> <span class="ci">streamBeginIndex</span><span class="cp">:</span> <span class="ci">int64</span>
  184. <span class="cr">-&gt;</span> <span class="ci">CharStream</span>
  185. <a id="interface.new_file-path:B:" href="#members.new_file-path"><span class="interface-member-marker"><span class="ck">new</span></span></a><span class="cp">:</span> <span class="ci">path</span><span class="cp">:</span> <span class="ci">string</span> <span class="cp">*</span> <span class="ci">encoding</span><span class="cp">:</span> <a href="http://msdn.microsoft.com/en-us/library/system.text.encoding.aspx"><span class="ci">System</span><span class="cm">.</span><span class="ci">Text</span><span class="cm">.</span><span class="ci">Encoding</span></a>
  186. <span class="cr">-&gt;</span> <span class="ci">CharStream</span>
  187. <a id="interface.new_file-path_2:B:" href="#members.new_file-path_2"><span class="interface-member-marker"><span class="ck">new</span></span></a><span class="cp">:</span> <span class="ci">path</span><span class="cp">:</span> <span class="ci">string</span>
  188. <span class="cp">*</span> <span class="ci">encoding</span><span class="cp">:</span> <a href="http://msdn.microsoft.com/en-us/library/system.text.encoding.aspx"><span class="ci">System</span><span class="cm">.</span><span class="ci">Text</span><span class="cm">.</span><span class="ci">Encoding</span></a> <span class="cp">*</span> <span class="ci">detectEncodingFromByteOrderMarks</span><span class="cp">:</span> <span class="ci">bool</span>
  189. <span class="cr">-&gt;</span> <span class="ci">CharStream</span>
  190. <a id="interface.new_file-path_3:B:" href="#members.new_file-path_3"><span class="interface-member-marker"><span class="ck">new</span></span></a><span class="cp">:</span> <span class="ci">path</span><span class="cp">:</span> <span class="ci">string</span>
  191. <span class="cp">*</span> <span class="ci">encoding</span><span class="cp">:</span> <a href="http://msdn.microsoft.com/en-us/library/system.text.encoding.aspx"><span class="ci">System</span><span class="cm">.</span><span class="ci">Text</span><span class="cm">.</span><span class="ci">Encoding</span></a> <span class="cp">*</span> <span class="ci">detectEncodingFromByteOrderMarks</span><span class="cp">:</span> <span class="ci">bool</span>
  192. <span class="cp">*</span> <span class="ci">blockSize</span><span class="cp">:</span> <span class="ci">int</span> <span class="cp">*</span> <span class="ci">blockOverlap</span><span class="cp">:</span> <span class="ci">int</span> <span class="cp">*</span> <span class="ci">minRegexSpace</span><span class="cp">:</span> <span class="ci">int</span>
  193. <span class="cp">*</span> <span class="ci">byteBufferLength</span><span class="cp">:</span> <span class="ci">int</span>
  194. <span class="cr">-&gt;</span> <span class="ci">CharStream</span>
  195. <a id="interface.new_stream:B:" href="#members.new_stream"><span class="interface-member-marker"><span class="ck">new</span></span></a><span class="cp">:</span> <span class="ci">stream</span><span class="cp">:</span> <a href="http://msdn.microsoft.com/en-us/library/system.io.stream.aspx"><span class="ci">System</span><span class="cm">.</span><span class="ci">IO</span><span class="cm">.</span><span class="ci">Stream</span></a> <span class="cp">*</span> <span class="ci">encoding</span><span class="cp">:</span> <a href="http://msdn.microsoft.com/en-us/library/system.text.encoding.aspx"><span class="ci">System</span><span class="cm">.</span><span class="ci">Text</span><span class="cm">.</span><span class="ci">Encoding</span></a>
  196. <span class="cr">-&gt;</span> <span class="ci">CharStream</span>
  197. <a id="interface.new_stream_2:B:" href="#members.new_stream_2"><span class="interface-member-marker"><span class="ck">new</span></span></a><span class="cp">:</span> <span class="ci">stream</span><span class="cp">:</span> <a href="http://msdn.microsoft.com/en-us/library/system.io.stream.aspx"><span class="ci">System</span><span class="cm">.</span><span class="ci">IO</span><span class="cm">.</span><span class="ci">Stream</span></a> <span class="cp">*</span> <span class="ci">leaveOpen</span><span class="cp">:</span> <span class="ci">bool</span>
  198. <span class="cp">*</span> <span class="ci">encoding</span><span class="cp">:</span> <a href="http://msdn.microsoft.com/en-us/library/system.text.encoding.aspx"><span class="ci">System</span><span class="cm">.</span><span class="ci">Text</span><span class="cm">.</span><span class="ci">Encoding</span></a>
  199. <span class="cr">-&gt;</span> <span class="ci">CharStream</span>
  200. <a id="interface.new_stream_3:B:" href="#members.new_stream_3"><span class="interface-member-marker"><span class="ck">new</span></span></a><span class="cp">:</span> <span class="ci">stream</span><span class="cp">:</span> <a href="http://msdn.microsoft.com/en-us/library/system.io.stream.aspx"><span class="ci">System</span><span class="cm">.</span><span class="ci">IO</span><span class="cm">.</span><span class="ci">Stream</span></a> <span class="cp">*</span> <span class="ci">leaveOpen</span><span class="cp">:</span> <span class="ci">bool</span>
  201. <span class="cp">*</span> <span class="ci">encoding</span><span class="cp">:</span> <a href="http://msdn.microsoft.com/en-us/library/system.text.encoding.aspx"><span class="ci">System</span><span class="cm">.</span><span class="ci">Text</span><span class="cm">.</span><span class="ci">Encoding</span></a> <span class="cp">*</span> <span class="ci">detectEncodingFromByteOrderMarks</span><span class="cp">:</span> <span class="ci">bool</span>
  202. <span class="cr">-&gt;</span> <span class="ci">CharStream</span>
  203. <a id="interface.new_stream_4:B:" href="#members.new_stream_4"><span class="interface-member-marker"><span class="ck">new</span></span></a><span class="cp">:</span> <span class="ci">stream</span><span class="cp">:</span> <a href="http://msdn.microsoft.com/en-us/library/system.io.stream.aspx"><span class="ci">System</span><span class="cm">.</span><span class="ci">IO</span><span class="cm">.</span><span class="ci">Stream</span></a> <span class="cp">*</span> <span class="ci">leaveOpen</span><span class="cp">:</span> <span class="ci">bool</span>
  204. <span class="cp">*</span> <span class="ci">encoding</span><span class="cp">:</span> <a href="http://msdn.microsoft.com/en-us/library/system.text.encoding.aspx"><span class="ci">System</span><span class="cm">.</span><span class="ci">Text</span><span class="cm">.</span><span class="ci">Encoding</span></a> <span class="cp">*</span> <span class="ci">detectEncodingFromByteOrderMarks</span><span class="cp">:</span> <span class="ci">bool</span>
  205. <span class="cp">*</span> <span class="ci">blockSize</span><span class="cp">:</span> <span class="ci">int</span> <span class="cp">*</span> <span class="ci">blockOverlap</span><span class="cp">:</span> <span class="ci">int</span> <span class="cp">*</span> <span class="ci">minRegexSpace</span><span class="cp">:</span> <span class="ci">int</span>
  206. <span class="cp">*</span> <span class="ci">byteBufferLength</span><span class="cp">:</span> <span class="ci">int</span>
  207. <span class="cr">-&gt;</span> <span class="ci">CharStream</span>
  208. <span class="ck">member</span> <a id="interface.Dispose:B:" href="#members.Dispose"><span class="interface-member-marker"><span class="ci">Dispose</span></span></a><span class="cp">:</span> <span class="ci">unit</span> <span class="cr">-&gt;</span> <span class="ci">unit</span>
  209. <span class="ck">member</span> <a id="interface.Begin:B:" href="#members.Begin"><span class="interface-member-marker"><span class="ci">Begin</span></span></a><span class="cp">:</span> <a href="#Iterator"><span class="ci">Iterator</span></a>
  210. <span class="ck">member</span> <a id="interface.Seek:B:" href="#members.Seek"><span class="interface-member-marker"><span class="ci">Seek</span></span></a><span class="cp">:</span> <span class="ci">int64</span> <span class="cr">-&gt;</span> <a href="#Iterator"><span class="ci">Iterator</span></a>
  211. <span class="ck">member</span> <a id="interface.BeginIndex:B:" href="#members.BeginIndex"><span class="interface-member-marker"><span class="ci">BeginIndex</span></span></a><span class="cp">:</span> <span class="ci">int64</span>
  212. <span class="ck">member</span> <a id="interface.EndIndex:B:" href="#members.EndIndex"><span class="interface-member-marker"><span class="ci">EndIndex</span></span></a><span class="cp">:</span> <span class="ci">int64</span>
  213. <span class="ck">static</span> <span class="ck">member</span> <a id="interface.FoldCase:B:" href="#members.FoldCase"><span class="interface-member-marker"><span class="ci">FoldCase</span></span></a><span class="cp">:</span> <span class="ci">string</span> <span class="cr">-&gt;</span> <span class="ci">string</span>
  214. <span class="ck">static</span> <span class="ck">member</span> <a id="interface.NormalizeNewlines:B:" href="#members.NormalizeNewlines"><span class="interface-member-marker"><span class="ci">NormalizeNewlines</span></span></a><span class="cp">:</span> <span class="ci">string</span> <span class="cr">-&gt;</span> <span class="ci">string</span>
  215. <span class="cp">[&lt;</span><span class="ci">Struct</span><span class="cp">&gt;]</span>
  216. <span class="ck">type</span> <span class="bold"><a href="#Iterator"><span class="ci">Iterator</span></a></span> <span class="cp">=</span> <span class="cbc"><span class="left-delimiter">(*</span> see below <span class="right-delimiter">*)</span></span>
  217. </pre>
  218. </div>
  219. </div>
  220. </div>
  221. <div id="remarks" class="section s3">
  222. <h2 class="title h3"><span class="section-number">2.8.2</span> Remarks</h2>
  223. <div class="intro i3">
  224. <p class="para _1">
  225. The <code class="fsharp"><span class="ci">CharStream</span></code> class provides an <a
  226. href="http://en.wikipedia.org/wiki/Iterator">iteratorbased</a> interface for reading UTF16 chars from a binary stream or an inmemory char
  227. buffer (e.g. a string). It is optimized for use in backtracking parser applications and supports arbitrary <em>charbased</em> seeking, even for
  228. streams larger than physical memory.
  229. </p>
  230. <p class="para _2">
  231. A <code class="fsharp"><span class="ci">CharStream</span></code> constructed from a <code class="fsharp"><a
  232. href="http://msdn.microsoft.com/en-us/library/system.io.stream.aspx"><span class="ci">System</span><span class="cm">.</span><span
  233. class="ci">IO</span><span class="cm">.</span><span class="ci">Stream</span></a></code> or a file path reads the stream blockwise and only holds
  234. the most recently accessed block in memory. The blocks overlap in order to provide efficient access on the boundary between blocks.
  235. </p>
  236. <p class="para _3">
  237. If the char content is already available as a string or a char array, a <code class="fsharp"><span class="ci">CharStream</span></code> can be
  238. directly constructed from the char buffer (without needing to copy the buffer). The overhead of accessing an inmemory char buffer through a
  239. <code class="fsharp"><span class="ci">CharStream</span></code> is minimal.
  240. </p>
  241. <p class="para _4">
  242. You can access the content of a <code class="fsharp"><span class="ci">CharStream</span></code> only through its <code class="fsharp"><a
  243. href="#Iterator"><span class="ci">Iterator</span></a></code> instances. The <code class="fsharp"><a href="#members.Begin"><span
  244. class="ci">Begin</span></a></code> property returns an iterator pointing to the first char in the stream. The <code class="fsharp"><a
  245. href="#members.Seek"><span class="ci">Seek</span></a></code> method returns an iterator pointing to the char with a given index.
  246. </p>
  247. <div class="para _5">
  248. <div class="dl multi-para">
  249. <dl class="dl multi-para">
  250. <dt class="_1"><a id="remarks.low-trust-version"></a>LowTrust version</dt>
  251. <dd class="_1">
  252. <div class="para _1">
  253. <p>
  254. If you compile FParsec with the <code class="fsharp"><span class="ci">LOW_TRUST</span></code> conditional compiler symbol, the <code
  255. class="fsharp"><span class="ci">CharStream</span></code> class differs from the normal version as follows:
  256. </p>
  257. <ul class="l1">
  258. <li class="_1">
  259. No <a href="http://msdn.microsoft.com/en-us/library/t2yzs44b.aspx">unverifiable code</a> involving pointers is used. <em>This allows
  260. FParsec to be executed in an environment with reduced trust</em>, such as medium trust ASP.NET applications.
  261. </li>
  262. <li class="_2">
  263. A <code class="fsharp"><span class="ci">CharStream</span></code> that is constructed from a <code class="fsharp"><a
  264. href="http://msdn.microsoft.com/en-us/library/system.io.stream.aspx"><span class="ci">System</span><span class="cm">.</span><span
  265. class="ci">IO</span><span class="cm">.</span><span class="ci">Stream</span></a></code> or a file path reads the complete file into a single
  266. string during construction. <em>This severely limits the maximum practical stream size.</em>
  267. </li>
  268. <li class="_3">
  269. Although the <code class="fsharp"><span class="ci">CharStream</span></code> class still supports the <code class="fsharp"><span
  270. class="ci">IDisposable</span></code> interface, disposing the <code class="fsharp"><span class="ci">CharStream</span></code> instances is
  271. no longer necessary, since no resources are hold that need to be explicitly released.
  272. </li>
  273. </ul>
  274. </div>
  275. </dd>
  276. <dt class="_2"><a id="remarks.non-sequential-access"></a>Nonsequential access</dt>
  277. <dd class="_2">
  278. <p class="para _1">
  279. <span class="small"><span class="italic">This note does not apply to the <a href="#remarks.low-trust-version">LowTrust version</a> of
  280. FParsec.</span></span><br /> If you construct a <code class="fsharp"><span class="ci">CharStream</span></code> from a <code class="fsharp"><a
  281. href="http://msdn.microsoft.com/en-us/library/system.io.stream.aspx"><span class="ci">System</span><span class="cm">.</span><span
  282. class="ci">IO</span><span class="cm">.</span><span class="ci">Stream</span></a></code> or a file path and you backtrack over a distance long
  283. enough to require the <code class="fsharp"><span class="ci">CharStream</span></code> to reread a previous block, then the underlying
  284. <strong>byte stream needs to support seeking</strong>, otherwise a <code class="fsharp"><a
  285. href="http://msdn.microsoft.com/en-us/library/system.notsupportedexception.aspx"><span class="ci">NotSupportedException</span></a></code> is
  286. thrown. Furthermore, the <strong><a href="http://msdn.microsoft.com/en-us/library/system.text.decoder.aspx">Decoder</a> for the input <a
  287. href="http://msdn.microsoft.com/en-us/library/system.text.encoding.aspx">Encoding</a> must be serializable</strong> if you backtrack to a
  288. block other than the first in the stream. Note that <em>file streams created for regular disk files are always seekable and all the .NET
  289. standard decoders are serializable</em>. In order to support nonseekable streams for applications which dont require extensive
  290. backtracking, no exception will be thrown before an operation actually requires backtracking and the necessary capabilities of the stream or
  291. decoder are not available.
  292. </p>
  293. </dd>
  294. <dt class="_3">Decoder errors</dt>
  295. <dd class="_3">
  296. <p class="para _1">
  297. A <code class="fsharp"><span class="ci">CharStream</span></code> constructed from a binary input stream decodes the input data with the help
  298. of a <code class="fsharp"><a href="http://msdn.microsoft.com/en-us/library/system.text.decoder.aspx"><span
  299. class="ci">Decoder</span></a></code> instance obtained via the <code class="fsharp"><span class="ci">Encodings</span></code>s <code
  300. class="fsharp"><span class="ci">GetDecoder</span></code> method. Depending on the configuration of the encoding the decoder might throw an
  301. exception if it encounters invalid byte sequences, usually a <code class="fsharp"><span class="ci">System</span><span
  302. class="cm">.</span><span class="ci">Text</span><span class="cm">.</span><a
  303. href="http://msdn.microsoft.com/en-us/library/system.text.decoderfallbackexception.aspx"><span
  304. class="ci">DecoderFallbackException</span></a></code> or a <code class="fsharp"><span class="ci">System</span><span class="cm">.</span><span
  305. class="ci">IO</span><span class="cm">.</span><a href="http://msdn.microsoft.com/en-us/library/system.argumentexception.aspx"><span
  306. class="ci">ArgumentException</span></a></code>. <sup class="fn-mark"><a id="remarks.:FN:1:B:" href="#remarks.:FN:1">[1]</a></sup>
  307. </p>
  308. </dd>
  309. <dt class="_4"><a id="remarks.case-insensitive-matching"></a>Caseinsensitive matching</dt>
  310. <dd class="_4">
  311. <p class="para _1">
  312. The <code class="fsharp"><a href="#Iterator.members.MatchCaseFolded"><span class="ci">MatchCaseFolded</span></a></code> member matches the
  313. content of the stream &#x201C;caseinsensitively&#x201D; with a reference string. In this instance &#x201C;caseinsensitive&#x201D; means
  314. that before the chars are matched with the reference string they are <a
  315. href="http://unicode.org/reports/tr21/tr21-5.html#Caseless_Matching">mapped to a canonical form where case differences are erased</a>. For
  316. performance reasons <code class="fsharp"><a href="#Iterator.members.MatchCaseFolded"><span class="ci">MatchCaseFolded</span></a></code> only
  317. applies the (nonTurkic) 1to1 <a href="http://unicode.org/Public/UNIDATA/CaseFolding.txt">case folding mappings</a> (v. 5.1) for Unicode
  318. code points in the Basic Multilingual Plane, i.e. code points below 0x10000. These mappings are sufficient for many caseinsensitive parser
  319. grammars encountered in practice, but they are not appropriate for matching arbitrary natural language content. Also note that the <code
  320. class="fsharp"><span class="ci">CharStream</span></code> class performs no Unicode <a
  321. href="http://unicode.org/reports/tr15/">normalization</a>.
  322. </p>
  323. </dd>
  324. <dt class="_5">Newlines</dt>
  325. <dd class="_5">
  326. <p class="para _1">
  327. The <code class="fsharp"><span class="ci">CharStream</span></code> class itself does not apply any automatic newline conversion to the
  328. processed input. A useful helper function for this purpose is <code class="fsharp"><a href="#members.NormalizeNewlines"><span
  329. class="ci">NormalizeNewlines</span></a></code>, which replaces all ASCII newline character sequences (<code class="fsharp"><span
  330. class="cs"><span class="left-delimiter">"</span><span class="ce">\n</span><span class="right-delimiter">"</span></span></code>, <code
  331. class="fsharp"><span class="cs"><span class="left-delimiter">"</span><span class="ce">\r</span><span class="ce">\n</span><span
  332. class="right-delimiter">"</span></span></code> or <code class="fsharp"><span class="cs"><span class="left-delimiter">"</span><span
  333. class="ce">\r</span><span class="right-delimiter">"</span></span></code>) in a given string with <code class="fsharp"><span class="cs"><span
  334. class="left-delimiter">"</span><span class="ce">\n</span><span class="right-delimiter">"</span></span></code>.
  335. </p>
  336. </dd>
  337. <dt class="_6">Disposable interface</dt>
  338. <dd class="_6">
  339. <p class="para _1">
  340. <span class="small"><span class="italic">This note does not apply to the <a href="#remarks.low-trust-version">LowTrust version</a> of
  341. FParsec.</span></span><br /> A <code class="fsharp"><span class="ci">CharStream</span></code> holds managed and unmanaged resources that need
  342. to be explicitly released. Hence, it is very important that <code class="fsharp"><span class="ci">CharStream</span></code> objects are
  343. promptly disposed after use. Where possible <code class="fsharp"><span class="ci">CharStream</span></code> objects should only be used within
  344. a &#x201C;using&#x201D; block (C#), a &#x201C;use&#x201D; expression( F#) or similar constructs in other languages. Members of <code
  345. class="fsharp"><a href="#Iterator"><span class="ci">CharStream</span><span class="cm">.</span><span class="ci">Iterator</span></a></code>
  346. instances must not be accessed after the corresponding <code class="fsharp"><span class="ci">CharStream</span></code> has been disposed, of
  347. course.
  348. </p>
  349. </dd>
  350. <dt class="_7">Thread safety</dt>
  351. <dd class="_7">
  352. <p class="para _1">
  353. A <code class="fsharp"><span class="ci">CharStream</span></code> constructed from a <code class="fsharp"><a
  354. href="http://msdn.microsoft.com/en-us/library/system.io.stream.aspx"><span class="ci">System</span><span class="cm">.</span><span
  355. class="ci">IO</span><span class="cm">.</span><span class="ci">Stream</span></a></code> or a file path is not thread safe and its members may
  356. not be accessed by multiple threads, except in the <a href="#remarks.low-trust-version">LowTrust version</a> of FParsec. A <code
  357. class="fsharp"><span class="ci">CharStream</span></code> constructed directly from a string, char array or char pointer can always be safely
  358. accessed concurrently using different <code class="fsharp"><span class="ci">Iterators</span></code> (provided that the <code
  359. class="fsharp"><span class="ci">CharStream</span></code> is disposed by only one thread, of course). <span class="italic"><code
  360. class="fsharp"><a href="#Iterator"><span class="ci">CharStream</span><span class="cm">.</span><span class="ci">Iterator</span></a></code>
  361. instances are never thread safe.</span>
  362. </p>
  363. </dd>
  364. </dl>
  365. </div>
  366. </div>
  367. </div>
  368. </div>
  369. <div id="exceptions" class="section s3">
  370. <h2 class="title h3"><span class="section-number">2.8.3</span> I/O exceptions</h2>
  371. <div class="intro i3">
  372. <p class="para _1">
  373. If you construct a <code class="fsharp"><span class="ci">CharStream</span></code> from a <code class="fsharp"><a
  374. href="http://msdn.microsoft.com/en-us/library/system.io.stream.aspx"><span class="ci">System</span><span class="cm">.</span><span
  375. class="ci">IO</span><span class="cm">.</span><span class="ci">Stream</span></a></code> or a file path, any <code class="fsharp"><span
  376. class="ci">CharStream</span></code> operation that requires reading chars from the underlying byte stream may throw one of the following
  377. exceptions.
  378. </p>
  379. <p class="para _2">
  380. In general it is <em>not</em> safe to continue to use a <code class="fsharp"><span class="ci">CharStream</span></code> instance after one of
  381. these exceptions was thrown. Calling <code class="fsharp"><a href="#members.Dispose"><span class="ci">Dispose</span></a><span
  382. class="cp">()</span></code> on an <code class="fsharp"><span class="ci">CharStream</span></code> instance after the instance or one of its
  383. iterators has thrown an exception is always safe.
  384. </p>
  385. <div class="para _3">
  386. <div class="dl multi-para">
  387. <dl class="dl multi-para">
  388. <dt class="_1"><code class="fsharp"><span class="ci">NotSupportedException</span></code></dt>
  389. <dd class="_1">
  390. <p class="para _1">
  391. Seeking of the underlying byte stream is required, but the byte stream does not support seeking or the <code class="fsharp"><a
  392. href="http://msdn.microsoft.com/en-us/library/system.text.encoding.aspx"><span class="ci">Encoding</span></a></code>s <code
  393. class="fsharp"><a href="http://msdn.microsoft.com/en-us/library/system.text.decoder.aspx"><span class="ci">Decoder</span></a></code> is not
  394. serializable. See also the remarks above on <a href="#remarks.non-sequential-access">nonsequential access</a>.
  395. </p>
  396. </dd>
  397. <dt class="_2"><code class="fsharp"><span class="ci">IOException</span></code></dt>
  398. <dd class="_2">
  399. <p class="para _1">An I/O occured while reading data from the underlying byte stream.</p>
  400. </dd>
  401. <dt class="_3"><code class="fsharp"><span class="ci">ArgumentException</span></code></dt>
  402. <dd class="_3">
  403. <p class="para _1">
  404. The underlying byte stream contains invalid bytes and the <code class="fsharp"><a
  405. href="http://msdn.microsoft.com/en-us/library/system.text.encoding.aspx"><span class="ci">Encoding</span></a></code> was constructed with the
  406. <code class="fsharp"><span class="ci">throwOnInvalidBytes</span></code> option.
  407. </p>
  408. </dd>
  409. <dt class="_4"><code class="fsharp"><span class="ci">DecoderFallbackException</span></code></dt>
  410. <dd class="_4">
  411. <p class="para _1">The underlying byte stream contains invalid bytes for which the decoder fallback threw this exception.</p>
  412. <p class="para _2">
  413. The byte index of the invalid bytes in the stream is stored as a boxed <code class="fsharp"><span class="ci">System</span><span
  414. class="cm">.</span><span class="ci">Int64</span></code> in the <code class="fsharp"><span class="cs"><span
  415. class="left-delimiter">"</span>Stream.Position<span class="right-delimiter">"</span></span></code> entry of the <code class="fsharp"><a
  416. href="http://msdn.microsoft.com/en-us/library/system.exception.data.aspx"><span class="ci">Data</span></a></code> member of the exception
  417. instance. The precision of the index depends on the precision of the <code class="fsharp"><a
  418. href="http://msdn.microsoft.com/en-us/library/system.text.decoderfallbackexception.aspx"><span
  419. class="ci">DecoderFallbackException</span></a></code>s <code class="fsharp"><a
  420. href="http://msdn.microsoft.com/en-us/library/system.text.decoderfallbackexception.index.aspx"><span class="ci">Index</span></a></code>
  421. member. If the underlying <code class="fsharp"><a href="http://msdn.microsoft.com/en-us/library/system.io.stream.aspx"><span
  422. class="ci">System</span><span class="cm">.</span><span class="ci">IO</span><span class="cm">.</span><span class="ci">Stream</span></a></code>
  423. is not seekable, the byte index only takes into account the bytes read by the <code class="fsharp"><span class="ci">CharStream</span></code>,
  424. but not any bytes read before the <code class="fsharp"><span class="ci">CharStream</span></code> was constructed.
  425. </p>
  426. </dd>
  427. </dl>
  428. </div>
  429. </div>
  430. </div>
  431. </div>
  432. <div id="members" class="section s3">
  433. <h2 class="title h3"><span class="section-number">2.8.4</span> Members</h2>
  434. <div class="intro i3">
  435. <div class="para _1">
  436. <div class="interface-members">
  437. <div class="interface-member _1" id="members.new_string">
  438. <div class="interface-member-code">
  439. <a class="interface-member-backlink" href="#interface.new_string:B:"></a>
  440. <pre class="code fsharp"><span class="interface-member-marker"><span class="ck">new</span></span><span class="cp">:</span> <span class="ci">chars</span><span class="cp">:</span> <span class="ci">string</span> <span class="cp">*</span> <span class="ci">index</span><span class="cp">:</span> <span class="ci">int</span> <span class="cp">*</span> <span class="ci">length</span><span class="cp">:</span> <span class="ci">int</span>
  441. <span class="cr">-&gt;</span> <span class="ci">CharStream</span>
  442. </pre>
  443. </div>
  444. <div class="interface-member-description">
  445. <p class="para _1">
  446. Is equivalent to <code class="fsharp"><a href="#members.new_string_offset"><span class="ck">new</span> <span
  447. class="ci">CharStream</span></a><span class="cp">(</span><span class="ci">chars</span><span class="cp">,</span> <span
  448. class="ci">index</span><span class="cp">,</span> <span class="ci">length</span><span class="cp">,</span> <span class="cn">0L</span><span
  449. class="cp">)</span></code>.
  450. </p>
  451. </div>
  452. </div>
  453. <div class="interface-member _2" id="members.new_string_offset">
  454. <div class="interface-member-code">
  455. <a class="interface-member-backlink" href="#interface.new_string_offset:B:"></a>
  456. <pre class="code fsharp"><span class="interface-member-marker"><span class="ck">new</span></span><span class="cp">:</span> <span class="ci">chars</span><span class="cp">:</span> <span class="ci">string</span> <span class="cp">*</span> <span class="ci">index</span><span class="cp">:</span> <span class="ci">int</span> <span class="cp">*</span> <span class="ci">length</span><span class="cp">:</span> <span class="ci">int</span> <span class="cp">*</span> <span class="ci">streamBeginIndex</span><span class="cp">:</span> <span class="ci">int64</span>
  457. <span class="cr">-&gt;</span> <span class="ci">CharStream</span>
  458. </pre>
  459. </div>
  460. <div class="interface-member-description">
  461. <p class="para _1">
  462. Constructs a <code class="fsharp"><span class="ci">CharStream</span></code> from the chars in the string argument between the indices <code
  463. class="fsharp"><span class="ci">index</span></code> (inclusive) and <code class="fsharp"><span class="ci">index</span> <span
  464. class="co">+</span> <span class="ci">length</span></code> (exclusive). By directly referencing the chars in the string this constructor
  465. avoids any copy of the string content.
  466. </p>
  467. <p class="para _2">
  468. The first char in the stream is assigned the index <code class="fsharp"><span class="ci">streamBeginIndex</span></code>. A positive <code
  469. class="fsharp"><span class="ci">streamBeginIndex</span></code> allows you for example to create a substream of another <code
  470. class="fsharp"><span class="ci">CharStream</span></code>, i.e. a <code class="fsharp"><span class="ci">CharStream</span></code> instance that
  471. only contains a subsegment of another char stream but is accessible through the same char indices.
  472. </p>
  473. <div class="para _3">
  474. <p>
  475. <code class="fsharp"><span class="ci">chars</span></code> must not be null. An <code class="fsharp"><a
  476. href="http://msdn.microsoft.com/en-us/library/system.argumentoutofrangeexception.aspx"><span
  477. class="ci">ArgumentOutOfRangeException</span></a></code> is thrown if the arguments do not satisfy the following conditions:
  478. </p>
  479. <ul class="l1">
  480. <li class="_1">
  481. <code class="fsharp"><span class="ci">index</span></code> 0, <code class="fsharp"><span class="ci">length</span></code> 0, <code
  482. class="fsharp"><span class="ci">index</span></code> + <code class="fsharp"><span class="ci">length</span></code> <code
  483. class="fsharp"><span class="ci">chars</span><span class="cm">.</span><span class="ci">Length</span></code> and
  484. </li>
  485. <li class="_2">0 <code class="fsharp"><span class="ci">streamBeginIndex</span></code> &lt; 2<sup>60</sup>.</li>
  486. </ul>
  487. </div>
  488. <div class="para _4">
  489. <div class="admonition">
  490. <div class="admonition-title">Important</div>
  491. <div class="admonition-body">
  492. <p class="para _1">
  493. <span class="small"><span class="italic">This note does not apply to the <a href="#remarks.low-trust-version">LowTrust version</a> of
  494. FParsec.</span></span><br /> The given string is &#x201C;<a
  495. href="http://msdn.microsoft.com/en-us/library/83y4ak54.aspx">pinned</a>&#x201D; until the <code class="fsharp"><span
  496. class="ci">CharStream</span></code> is disposed. Pinning the string prevents the GC from moving it around in memory during garbage
  497. collection. On .NET (at least in versions up to and including 4.0) the pinning has no effect if the string is large enough to be allocated
  498. on the Large Object Heap, i.e. has a length of about 42500 chars or more. However, pinning smaller strings does constrain the normal
  499. operations of the GC. Thus, <strong>to minimize the negative impact on the GC, you should dispose <code class="fsharp"><span
  500. class="ci">CharStream</span></code> instances constructed from small strings as soon as youre done parsing it</strong>. If you keep a
  501. large number of <code class="fsharp"><span class="ci">CharStream</span></code> instances constructed from small strings around for an
  502. extended period of time, you risk fragmenting the heap.
  503. </p>
  504. </div>
  505. </div>
  506. </div>
  507. </div>
  508. </div>
  509. <div class="interface-member _3" id="members.new_char-array">
  510. <div class="interface-member-code">
  511. <a class="interface-member-backlink" href="#interface.new_char-array:B:"></a>
  512. <pre class="code fsharp"><span class="interface-member-marker"><span class="ck">new</span></span><span class="cp">:</span> <span class="ci">chars</span><span class="cp">:</span> <span class="ci">char</span><span class="cp">[]</span> <span class="cp">*</span> <span class="ci">index</span><span class="cp">:</span> <span class="ci">int</span> <span class="cp">*</span> <span class="ci">length</span><span class="cp">:</span> <span class="ci">int</span>
  513. <span class="cr">-&gt;</span> <span class="ci">CharStream</span>
  514. </pre>
  515. </div>
  516. <div class="interface-member-description">
  517. <p class="para _1">
  518. <span class="small"><span class="italic">This constructor is not available in the <a href="#remarks.low-trust-version">LowTrust version</a>
  519. of FParsec.</span></span>
  520. </p>
  521. <p class="para _2">
  522. Is equivalent to <code class="fsharp"><a href="#members.new_char-array_offset"><span class="ck">new</span> <span
  523. class="ci">CharStream</span></a><span class="cp">(</span><span class="ci">chars</span><span class="cp">,</span> <span
  524. class="ci">index</span><span class="cp">,</span> <span class="ci">length</span><span class="cp">,</span> <span class="cn">0L</span><span
  525. class="cp">)</span></code>.
  526. </p>
  527. </div>
  528. </div>
  529. <div class="interface-member _4" id="members.new_char-array_offset">
  530. <div class="interface-member-code">
  531. <a class="interface-member-backlink" href="#interface.new_char-array_offset:B:"></a>
  532. <pre class="code fsharp"><span class="interface-member-marker"><span class="ck">new</span></span><span class="cp">:</span> <span class="ci">chars</span><span class="cp">:</span> <span class="ci">char</span><span class="cp">[]</span> <span class="cp">*</span> <span class="ci">index</span><span class="cp">:</span> <span class="ci">int</span> <span class="cp">*</span> <span class="ci">length</span><span class="cp">:</span> <span class="ci">int</span> <span class="cp">*</span> <span class="ci">streamBeginIndex</span><span class="cp">:</span> <span class="ci">int64</span>
  533. <span class="cr">-&gt;</span> <span class="ci">CharStream</span>
  534. </pre>
  535. </div>
  536. <div class="interface-member-description">
  537. <p class="para _1">
  538. <span class="small"><span class="italic">This constructor is not available in the <a href="#remarks.low-trust-version">LowTrust version</a>
  539. of FParsec.</span></span>
  540. </p>
  541. <p class="para _2">
  542. Constructs a <code class="fsharp"><span class="ci">CharStream</span></code> from the chars in the char array argument between the indices
  543. <code class="fsharp"><span class="ci">index</span></code> (inclusive) and <code class="fsharp"><span class="ci">index</span> <span
  544. class="co">+</span> <span class="ci">length</span></code> (exclusive). By directly referencing the chars in the char array this constructor
  545. avoids any copy of the char array content.
  546. </p>
  547. <p class="para _3">
  548. The first char in the stream is assigned the index <code class="fsharp"><span class="ci">streamBeginIndex</span></code>. A positive <code
  549. class="fsharp"><span class="ci">streamBeginIndex</span></code> allows you for example to create a substream of another <code
  550. class="fsharp"><span class="ci">CharStream</span></code>, i.e. a <code class="fsharp"><span class="ci">CharStream</span></code> instance that
  551. only contains a subsegment of another char stream but is accessible through the same char indices.
  552. </p>
  553. <div class="para _4">
  554. <p>
  555. <code class="fsharp"><span class="ci">chars</span></code> must not be null. An <code class="fsharp"><a
  556. href="http://msdn.microsoft.com/en-us/library/system.argumentoutofrangeexception.aspx"><span
  557. class="ci">ArgumentOutOfRangeException</span></a></code> is thrown if the arguments do not satisfy the following conditions:
  558. </p>
  559. <ul class="l1">
  560. <li class="_1">