PageRenderTime 62ms CodeModel.GetById 26ms RepoModel.GetById 0ms app.codeStats 0ms

/doc/development/spam_protection_and_captcha/exploratory_testing.md

https://gitlab.com/gitlab-r2devops/gitlab-foss
Markdown | 360 lines | 284 code | 76 blank | 0 comment | 0 complexity | 19fb0b5ac4d8df3c3cdbe817d870c40f MD5 | raw file
  1. ---
  2. stage: Manage
  3. group: Authentication and Authorization
  4. info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
  5. ---
  6. # Exploratory testing of CAPTCHAs
  7. You can reliably test CAPTCHA on review apps, and in your local development environment (GDK).
  8. You can always:
  9. - Force a reCAPTCHA to appear where it is supported.
  10. - Force a checkbox to display, instead of street sign images to find and select.
  11. To set up testing, follow the configuration on this page.
  12. ## Use appropriate test data
  13. Make sure you are testing a scenario which has spam/CAPTCHA enabled. For example:
  14. make sure you are editing a _public_ snippet, as only public snippets are checked for spam.
  15. ## Enable feature flags
  16. Enable any relevant feature flag, if the spam/CAPTCHA support is behind a feature flag.
  17. ## Set up Akismet and reCAPTCHA
  18. 1. To set up reCAPTCHA:
  19. 1. Review the [GitLab reCAPTCHA documentation](../../integration/recaptcha.md).
  20. 1. Get Google's official test reCAPTCHA credentials using the instructions from
  21. [Google's reCAPTCHA documentation](https://developers.google.com/recaptcha/docs/faq#id-like-to-run-automated-tests-with-recaptcha.-what-should-i-do).
  22. 1. For **Site key**, use: `6LeIxAcTAAAAAJcZVRqyHh71UMIEGNQ_MXjiZKhI`
  23. 1. For **Secret key**, use: `6LeIxAcTAAAAAGG-vFI1TnRWxMZNFuojJ4WifJWe`
  24. 1. Go to **Admin -> Settings -> Reporting** settings: `http://gdk.test:3000/admin/application_settings/reporting#js-spam-settings`
  25. 1. Select **Enable reCAPTCHA**. Enabling for login is not required unless you are testing that feature.
  26. 1. Enter the **Site key** and **Secret key**.
  27. 1. To set up Akismet:
  28. 1. Review the [GitLab documentation on Akismet](../../integration/akismet.md).
  29. 1. Get an Akismet API key. You can sign up for [a testing key from Akismet](https://akismet.com).
  30. You must enter your local host (such as`gdk.test`) and email when signing up.
  31. 1. Go to GitLab Akismet settings page, for example:
  32. `http://gdk.test:3000/admin/application_settings/reporting#js-spam-settings`
  33. 1. Enable Akismet and enter your Akismet **API key**.
  34. 1. To force an Akismet false-positive spam check, refer to the
  35. [Akismet API documentation](https://akismet.com/development/api/#comment-check) and
  36. [Akismet Getting Started documentation](https://docs.akismet.com/getting-started/confirm/) for more details:
  37. 1. You can use `akismet-guaranteed-spam@example.com` as the author email to force spam using the following steps:
  38. 1. Go to user email settings: `http://gdk.test:3000/-/profile/emails`
  39. 1. Add `akismet-guaranteed-spam@example.com` as a secondary email for the administrator user.
  40. 1. Confirm it in the Rails console: `bin/rails c` -> `User.find_by_username('root').emails.last.confirm`
  41. 1. Switch this verified email to be your primary email:
  42. 1. Go to **Avatar dropdown list -> Edit Profile -> Main Settings**.
  43. 1. For **Email**, enter `akismet-guaranteed-spam@example.com` to replace `admin@example.com`.
  44. 1. Select **Update Profile Settings** to save your changes.
  45. ## Test in the web UI
  46. After you have all the above configuration in place, you can test CAPTCHAs. Test
  47. in an area of the application which already has CAPTCHA support, such as:
  48. - Creating or editing an issue.
  49. - Creating or editing a public snippet. Only **public** snippets are checked for spam.
  50. ## Test in a development environment
  51. After you force Spam Flagging + CAPTCHA using the steps above, you can test the
  52. behavior with any spam-protected model/controller action.
  53. ### Test with CAPTCHA enabled (CONDITIONAL_ALLOW verdict)
  54. If CAPTCHA is enabled in these areas, you must solve the CAPTCHA popup modal before you can resubmit the form:
  55. - **Admin -> Settings -> Reporting -> Spam**
  56. - **Anti-bot Protection -> Enable reCAPTCHA**
  57. <!-- vale gitlab.Substitutions = NO -->
  58. ### Testing with CAPTCHA disabled ("DISALLOW" verdict)
  59. <!-- vale gitlab.Substitutions = YES -->
  60. If CAPTCHA is disabled in **Admin -> Settings -> Reporting -> Spam** and **Anti-bot Protection -> Enable reCAPTCHA**,
  61. no CAPTCHA popup displays. You are prevented from submitting the form at all.
  62. ### HTML page to render reCAPTCHA
  63. NOTE:
  64. If you use **Google's official test reCAPTCHA credentials** listed in
  65. [Set up Akismet and reCAPTCHA](#set-up-akismet-and-recaptcha), the
  66. CAPTCHA response string does not matter. It can be any string. If you use a
  67. real, valid key pair, you must solve the CAPTCHA to obtain a
  68. valid CAPTCHA response to use. You can do this once only, and only before it expires.
  69. To directly test the GraphQL API via [GraphQL Explorer](http://gdk.test:3000/-/graphql-explorer),
  70. get a reCAPTCHA response string via this form: `public/recaptcha.html` (`http://gdk.test:3000/recaptcha.html`):
  71. ```html
  72. <html>
  73. <head>
  74. <title>reCAPTCHA demo: Explicit render after an onload callback</title>
  75. <script type="text/javascript">
  76. var onloadCallback = function() {
  77. grecaptcha.render('html_element', {
  78. 'sitekey' : '6Ld05AsaAAAAAMsm1yTUp4qsdFARN15rQJPPqv6i'
  79. });
  80. };
  81. function onSubmit() {
  82. window.document.getElementById('recaptchaResponse').innerHTML = grecaptcha.getResponse();
  83. return false;
  84. }
  85. </script>
  86. </head>
  87. <body>
  88. <form onsubmit="return onSubmit()">
  89. <div id="html_element"></div>
  90. <br>
  91. <input type="submit" value="Submit">
  92. </form>
  93. <div>
  94. <h1>recaptchaResponse:</h1>
  95. <div id="recaptchaResponse"></div>
  96. </div>
  97. <script src="https://www.google.com/recaptcha/api.js?onload=onloadCallback&render=explicit"
  98. async defer>
  99. </script>
  100. </body>
  101. </html>
  102. ```
  103. ## Spam/CAPTCHA API exploratory testing examples
  104. These sections describe the steps needed to perform manual exploratory testing of
  105. various scenarios of the Spam and CAPTCHA behavior for the REST and GraphQL APIs.
  106. For the prerequisites, you must:
  107. 1. Perform all the steps listed above to enable Spam and CAPTCHA in the development environment,
  108. and force form submissions to require a CAPTCHA.
  109. 1. Ensure you have created an HTML page to render CAPTCHA under the `/public` directory,
  110. with a page that contains a form to manually generate a valid CAPTCHA response string.
  111. If you use **Google's official test reCAPTCHA credentials** listed in
  112. [Set up Akismet and reCAPTCHA](#set-up-akismet-and-recaptcha), the contents of the
  113. CAPTCHA response string don't matter.
  114. 1. Go to **Admin -> Settings -> Reporting -> Spam and Anti-bot protection**.
  115. 1. Select or clear **Enable reCAPTCHA** and **Enable Akismet** according to your
  116. scenario's needs.
  117. The following examples use snippet creation as an example. You could also use
  118. snippet updates, issue creation, or issue updates. Issues and snippets are the
  119. only models with full Spam and CAPTCHA support.
  120. ### Initial setup
  121. 1. Create an API token.
  122. 1. Export it in your terminal for the REST commands: `export PRIVATE_TOKEN=<your_api_token>`
  123. 1. Ensure you are logged into GitLab development environment at `localhost:3000` before using GraphiQL explorer,
  124. because it uses your logged-in user as authorization for running GraphQL queries.
  125. 1. For the GraphQL examples, use the GraphiQL explorer at `http://localhost:3000/-/graphql-explorer`.
  126. 1. Use the `--include` (`-i`) option to `curl` to print the HTTP response headers, including the status code.
  127. ### Scenario: Akismet and CAPTCHA enabled
  128. In this example, Akismet and CAPTCHA are enabled:
  129. 1. [Initial request](#initial-request).
  130. <!-- TODO in future edit
  131. Some example videos:
  132. - REST API:
  133. ![CAPTCHA REST API](/uploads/b148cbe45496e6f4a4f63d00bb9fbd8a/captcha_rest_api.mov)
  134. GraphQL API:
  135. ![CAPTCHA GraphQL API](/uploads/3c7ef0fad0b84bd588572bae51519463/captcha_graphql_api.mov)
  136. -->
  137. #### Initial request
  138. This initial request fails because no CAPTCHA response is provided.
  139. REST request:
  140. ```shell
  141. curl --request POST --header "PRIVATE-TOKEN: $PRIVATE_TOKEN" "http://localhost:3000/api/v4/snippets?title=Title&file_name=FileName&content=Content&visibility=public"
  142. ```
  143. REST response:
  144. ```shell
  145. {"needs_captcha_response":true,"spam_log_id":42,"captcha_site_key":"XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX","message":{"error":"Your snippet has been recognized as spam. Please, change the content or solve the reCAPTCHA to proceed."}}
  146. ```
  147. GraphQL request:
  148. ```graphql
  149. mutation {
  150. createSnippet(input: {
  151. title: "Title"
  152. visibilityLevel: public
  153. blobActions: [
  154. {
  155. action: create
  156. filePath: "BlobPath"
  157. content: "BlobContent"
  158. }
  159. ]
  160. }) {
  161. snippet {
  162. id
  163. title
  164. }
  165. errors
  166. }
  167. }
  168. ```
  169. GraphQL response:
  170. ```json
  171. {
  172. "data": {
  173. "createSnippet": null
  174. },
  175. "errors": [
  176. {
  177. "message": "Request denied. Solve CAPTCHA challenge and retry",
  178. "locations": [
  179. {
  180. "line": 22,
  181. "column": 5
  182. }
  183. ],
  184. "path": [
  185. "createSnippet"
  186. ],
  187. "extensions": {
  188. "needs_captcha_response": true,
  189. "spam_log_id": 140,
  190. "captcha_site_key": "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
  191. }
  192. }
  193. ]
  194. }
  195. ```
  196. #### Second request
  197. This request succeeds because a CAPTCHA response is provided.
  198. REST request:
  199. ```shell
  200. export CAPTCHA_RESPONSE="<CAPTCHA response obtained from HTML page to render CAPTCHA>"
  201. export SPAM_LOG_ID="<spam_log_id obtained from initial REST response>"
  202. curl --request POST --header "PRIVATE-TOKEN: $PRIVATE_TOKEN" --header "X-GitLab-Captcha-Response: $CAPTCHA_RESPONSE" --header "X-GitLab-Spam-Log-Id: $SPAM_LOG_ID" "http://localhost:3000/api/v4/snippets?title=Title&file_name=FileName&content=Content&visibility=public"
  203. ```
  204. REST response:
  205. ```shell
  206. {"id":42,"title":"Title","description":null,"visibility":"public", "other_fields": "..."}
  207. ```
  208. GraphQL request:
  209. NOTE:
  210. The GitLab GraphiQL implementation doesn't allow passing of headers, so we must write
  211. this as a `curl` query. Here, `--data-binary` is used to properly handle escaped double quotes
  212. in the JSON-embedded query.
  213. ```shell
  214. export CAPTCHA_RESPONSE="<CAPTCHA response obtained from HTML page to render CAPTCHA>"
  215. export SPAM_LOG_ID="<spam_log_id obtained from initial REST response>"
  216. curl --include "http://localhost:3000/api/graphql" --header "Authorization: Bearer $PRIVATE_TOKEN" --header "Content-Type: application/json" --header "X-GitLab-Captcha-Response: $CAPTCHA_RESPONSE" --header "X-GitLab-Spam-Log-Id: $SPAM_LOG_ID" --request POST --data-binary '{"query": "mutation {createSnippet(input: {title: \"Title\" visibilityLevel: public blobActions: [ { action: create filePath: \"BlobPath\" content: \"BlobContent\" } ] }) { snippet { id title } errors }}"}'
  217. ```
  218. GraphQL response:
  219. ```json
  220. {"data":{"createSnippet":{"snippet":{"id":"gid://gitlab/PersonalSnippet/42","title":"Title"},"errors":[]}}}
  221. ```
  222. ### Scenario: Akismet enabled, CAPTCHA disabled
  223. For this scenario, ensure you clear **Enable reCAPTCHA** in the Admin Area settings as described above.
  224. If CAPTCHA is not enabled, any request flagged as potential spam fails with no chance to resubmit,
  225. even if it could otherwise be resubmitted if CAPTCHA were enabled and successfully solved.
  226. The REST request is the same as if CAPTCHA was enabled:
  227. ```shell
  228. curl --request POST --header "PRIVATE-TOKEN: $PRIVATE_TOKEN" "http://localhost:3000/api/v4/snippets?title=Title&file_name=FileName&content=Content&visibility=public"
  229. ```
  230. REST response:
  231. ```shell
  232. {"message":{"error":"Your snippet has been recognized as spam and has been discarded."}}
  233. ```
  234. GraphQL request:
  235. ```graphql
  236. mutation {
  237. createSnippet(input: {
  238. title: "Title"
  239. visibilityLevel: public
  240. blobActions: [
  241. {
  242. action: create
  243. filePath: "BlobPath"
  244. content: "BlobContent"
  245. }
  246. ]
  247. }) {
  248. snippet {
  249. id
  250. title
  251. }
  252. errors
  253. }
  254. }
  255. ```
  256. GraphQL response:
  257. ```json
  258. {
  259. "data": {
  260. "createSnippet": null
  261. },
  262. "errors": [
  263. {
  264. "message": "Request denied. Spam detected",
  265. "locations": [
  266. {
  267. "line": 22,
  268. "column": 5
  269. }
  270. ],
  271. "path": [
  272. "createSnippet"
  273. ],
  274. "extensions": {
  275. "spam": true
  276. }
  277. }
  278. ]
  279. }
  280. ```
  281. ### Scenario: allow_possible_spam feature flag enabled
  282. With the `allow_possible_spam` feature flag enabled, the API returns a 200 response. Any
  283. valid request is successful and no CAPTCHA is presented, even if the request is considered
  284. spam.