/doc/development/spam_protection_and_captcha/exploratory_testing.md
Markdown | 360 lines | 284 code | 76 blank | 0 comment | 0 complexity | 19fb0b5ac4d8df3c3cdbe817d870c40f MD5 | raw file
1---
2stage: Manage
3group: Authentication and Authorization
4info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://about.gitlab.com/handbook/engineering/ux/technical-writing/#assignments
5---
6
7# Exploratory testing of CAPTCHAs
8
9You can reliably test CAPTCHA on review apps, and in your local development environment (GDK).
10You can always:
11
12- Force a reCAPTCHA to appear where it is supported.
13- Force a checkbox to display, instead of street sign images to find and select.
14
15To set up testing, follow the configuration on this page.
16
17## Use appropriate test data
18
19Make sure you are testing a scenario which has spam/CAPTCHA enabled. For example:
20make sure you are editing a _public_ snippet, as only public snippets are checked for spam.
21
22## Enable feature flags
23
24Enable any relevant feature flag, if the spam/CAPTCHA support is behind a feature flag.
25
26## Set up Akismet and reCAPTCHA
27
281. To set up reCAPTCHA:
29 1. Review the [GitLab reCAPTCHA documentation](../../integration/recaptcha.md).
30 1. Get Google's official test reCAPTCHA credentials using the instructions from
31 [Google's reCAPTCHA documentation](https://developers.google.com/recaptcha/docs/faq#id-like-to-run-automated-tests-with-recaptcha.-what-should-i-do).
32 1. For **Site key**, use: `6LeIxAcTAAAAAJcZVRqyHh71UMIEGNQ_MXjiZKhI`
33 1. For **Secret key**, use: `6LeIxAcTAAAAAGG-vFI1TnRWxMZNFuojJ4WifJWe`
34 1. Go to **Admin -> Settings -> Reporting** settings: `http://gdk.test:3000/admin/application_settings/reporting#js-spam-settings`
35 1. Select **Enable reCAPTCHA**. Enabling for login is not required unless you are testing that feature.
36 1. Enter the **Site key** and **Secret key**.
371. To set up Akismet:
38 1. Review the [GitLab documentation on Akismet](../../integration/akismet.md).
39 1. Get an Akismet API key. You can sign up for [a testing key from Akismet](https://akismet.com).
40 You must enter your local host (such as`gdk.test`) and email when signing up.
41 1. Go to GitLab Akismet settings page, for example:
42 `http://gdk.test:3000/admin/application_settings/reporting#js-spam-settings`
43 1. Enable Akismet and enter your Akismet **API key**.
441. To force an Akismet false-positive spam check, refer to the
45 [Akismet API documentation](https://akismet.com/development/api/#comment-check) and
46 [Akismet Getting Started documentation](https://docs.akismet.com/getting-started/confirm/) for more details:
47 1. You can use `akismet-guaranteed-spam@example.com` as the author email to force spam using the following steps:
48 1. Go to user email settings: `http://gdk.test:3000/-/profile/emails`
49 1. Add `akismet-guaranteed-spam@example.com` as a secondary email for the administrator user.
50 1. Confirm it in the Rails console: `bin/rails c` -> `User.find_by_username('root').emails.last.confirm`
51 1. Switch this verified email to be your primary email:
52 1. Go to **Avatar dropdown list -> Edit Profile -> Main Settings**.
53 1. For **Email**, enter `akismet-guaranteed-spam@example.com` to replace `admin@example.com`.
54 1. Select **Update Profile Settings** to save your changes.
55
56## Test in the web UI
57
58After you have all the above configuration in place, you can test CAPTCHAs. Test
59in an area of the application which already has CAPTCHA support, such as:
60
61- Creating or editing an issue.
62- Creating or editing a public snippet. Only **public** snippets are checked for spam.
63
64## Test in a development environment
65
66After you force Spam Flagging + CAPTCHA using the steps above, you can test the
67behavior with any spam-protected model/controller action.
68
69### Test with CAPTCHA enabled (CONDITIONAL_ALLOW verdict)
70
71If CAPTCHA is enabled in these areas, you must solve the CAPTCHA popup modal before you can resubmit the form:
72
73- **Admin -> Settings -> Reporting -> Spam**
74- **Anti-bot Protection -> Enable reCAPTCHA**
75
76<!-- vale gitlab.Substitutions = NO -->
77
78### Testing with CAPTCHA disabled ("DISALLOW" verdict)
79
80<!-- vale gitlab.Substitutions = YES -->
81
82If CAPTCHA is disabled in **Admin -> Settings -> Reporting -> Spam** and **Anti-bot Protection -> Enable reCAPTCHA**,
83no CAPTCHA popup displays. You are prevented from submitting the form at all.
84
85### HTML page to render reCAPTCHA
86
87NOTE:
88If you use **Google's official test reCAPTCHA credentials** listed in
89[Set up Akismet and reCAPTCHA](#set-up-akismet-and-recaptcha), the
90CAPTCHA response string does not matter. It can be any string. If you use a
91real, valid key pair, you must solve the CAPTCHA to obtain a
92valid CAPTCHA response to use. You can do this once only, and only before it expires.
93
94To directly test the GraphQL API via [GraphQL Explorer](http://gdk.test:3000/-/graphql-explorer),
95get a reCAPTCHA response string via this form: `public/recaptcha.html` (`http://gdk.test:3000/recaptcha.html`):
96
97```html
98<html>
99<head>
100 <title>reCAPTCHA demo: Explicit render after an onload callback</title>
101 <script type="text/javascript">
102 var onloadCallback = function() {
103 grecaptcha.render('html_element', {
104 'sitekey' : '6Ld05AsaAAAAAMsm1yTUp4qsdFARN15rQJPPqv6i'
105 });
106 };
107 function onSubmit() {
108 window.document.getElementById('recaptchaResponse').innerHTML = grecaptcha.getResponse();
109 return false;
110 }
111 </script>
112</head>
113<body>
114<form onsubmit="return onSubmit()">
115 <div id="html_element"></div>
116 <br>
117 <input type="submit" value="Submit">
118</form>
119<div>
120 <h1>recaptchaResponse:</h1>
121 <div id="recaptchaResponse"></div>
122</div>
123<script src="https://www.google.com/recaptcha/api.js?onload=onloadCallback&render=explicit"
124 async defer>
125</script>
126</body>
127</html>
128```
129
130## Spam/CAPTCHA API exploratory testing examples
131
132These sections describe the steps needed to perform manual exploratory testing of
133various scenarios of the Spam and CAPTCHA behavior for the REST and GraphQL APIs.
134
135For the prerequisites, you must:
136
1371. Perform all the steps listed above to enable Spam and CAPTCHA in the development environment,
138 and force form submissions to require a CAPTCHA.
1391. Ensure you have created an HTML page to render CAPTCHA under the `/public` directory,
140 with a page that contains a form to manually generate a valid CAPTCHA response string.
141 If you use **Google's official test reCAPTCHA credentials** listed in
142 [Set up Akismet and reCAPTCHA](#set-up-akismet-and-recaptcha), the contents of the
143 CAPTCHA response string don't matter.
1441. Go to **Admin -> Settings -> Reporting -> Spam and Anti-bot protection**.
1451. Select or clear **Enable reCAPTCHA** and **Enable Akismet** according to your
146 scenario's needs.
147
148The following examples use snippet creation as an example. You could also use
149snippet updates, issue creation, or issue updates. Issues and snippets are the
150only models with full Spam and CAPTCHA support.
151
152### Initial setup
153
1541. Create an API token.
1551. Export it in your terminal for the REST commands: `export PRIVATE_TOKEN=<your_api_token>`
1561. Ensure you are logged into GitLab development environment at `localhost:3000` before using GraphiQL explorer,
157 because it uses your logged-in user as authorization for running GraphQL queries.
1581. For the GraphQL examples, use the GraphiQL explorer at `http://localhost:3000/-/graphql-explorer`.
1591. Use the `--include` (`-i`) option to `curl` to print the HTTP response headers, including the status code.
160
161### Scenario: Akismet and CAPTCHA enabled
162
163In this example, Akismet and CAPTCHA are enabled:
164
1651. [Initial request](#initial-request).
166
167<!-- TODO in future edit
168
169Some example videos:
170
171- REST API:
172
173
174
175GraphQL API:
176
177
178
179-->
180
181#### Initial request
182
183This initial request fails because no CAPTCHA response is provided.
184
185REST request:
186
187```shell
188curl --request POST --header "PRIVATE-TOKEN: $PRIVATE_TOKEN" "http://localhost:3000/api/v4/snippets?title=Title&file_name=FileName&content=Content&visibility=public"
189```
190
191REST response:
192
193```shell
194{"needs_captcha_response":true,"spam_log_id":42,"captcha_site_key":"XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX","message":{"error":"Your snippet has been recognized as spam. Please, change the content or solve the reCAPTCHA to proceed."}}
195```
196
197GraphQL request:
198
199```graphql
200mutation {
201 createSnippet(input: {
202 title: "Title"
203 visibilityLevel: public
204 blobActions: [
205 {
206 action: create
207 filePath: "BlobPath"
208 content: "BlobContent"
209 }
210 ]
211 }) {
212 snippet {
213 id
214 title
215 }
216 errors
217 }
218}
219```
220
221GraphQL response:
222
223```json
224{
225 "data": {
226 "createSnippet": null
227 },
228 "errors": [
229 {
230 "message": "Request denied. Solve CAPTCHA challenge and retry",
231 "locations": [
232 {
233 "line": 22,
234 "column": 5
235 }
236 ],
237 "path": [
238 "createSnippet"
239 ],
240 "extensions": {
241 "needs_captcha_response": true,
242 "spam_log_id": 140,
243 "captcha_site_key": "XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX"
244 }
245 }
246 ]
247}
248```
249
250#### Second request
251
252This request succeeds because a CAPTCHA response is provided.
253
254REST request:
255
256```shell
257export CAPTCHA_RESPONSE="<CAPTCHA response obtained from HTML page to render CAPTCHA>"
258export SPAM_LOG_ID="<spam_log_id obtained from initial REST response>"
259curl --request POST --header "PRIVATE-TOKEN: $PRIVATE_TOKEN" --header "X-GitLab-Captcha-Response: $CAPTCHA_RESPONSE" --header "X-GitLab-Spam-Log-Id: $SPAM_LOG_ID" "http://localhost:3000/api/v4/snippets?title=Title&file_name=FileName&content=Content&visibility=public"
260```
261
262REST response:
263
264```shell
265{"id":42,"title":"Title","description":null,"visibility":"public", "other_fields": "..."}
266```
267
268GraphQL request:
269
270NOTE:
271The GitLab GraphiQL implementation doesn't allow passing of headers, so we must write
272this as a `curl` query. Here, `--data-binary` is used to properly handle escaped double quotes
273in the JSON-embedded query.
274
275```shell
276export CAPTCHA_RESPONSE="<CAPTCHA response obtained from HTML page to render CAPTCHA>"
277export SPAM_LOG_ID="<spam_log_id obtained from initial REST response>"
278curl --include "http://localhost:3000/api/graphql" --header "Authorization: Bearer $PRIVATE_TOKEN" --header "Content-Type: application/json" --header "X-GitLab-Captcha-Response: $CAPTCHA_RESPONSE" --header "X-GitLab-Spam-Log-Id: $SPAM_LOG_ID" --request POST --data-binary '{"query": "mutation {createSnippet(input: {title: \"Title\" visibilityLevel: public blobActions: [ { action: create filePath: \"BlobPath\" content: \"BlobContent\" } ] }) { snippet { id title } errors }}"}'
279```
280
281GraphQL response:
282
283```json
284{"data":{"createSnippet":{"snippet":{"id":"gid://gitlab/PersonalSnippet/42","title":"Title"},"errors":[]}}}
285```
286
287### Scenario: Akismet enabled, CAPTCHA disabled
288
289For this scenario, ensure you clear **Enable reCAPTCHA** in the Admin Area settings as described above.
290If CAPTCHA is not enabled, any request flagged as potential spam fails with no chance to resubmit,
291even if it could otherwise be resubmitted if CAPTCHA were enabled and successfully solved.
292
293The REST request is the same as if CAPTCHA was enabled:
294
295```shell
296curl --request POST --header "PRIVATE-TOKEN: $PRIVATE_TOKEN" "http://localhost:3000/api/v4/snippets?title=Title&file_name=FileName&content=Content&visibility=public"
297```
298
299REST response:
300
301```shell
302{"message":{"error":"Your snippet has been recognized as spam and has been discarded."}}
303```
304
305GraphQL request:
306
307```graphql
308mutation {
309 createSnippet(input: {
310 title: "Title"
311 visibilityLevel: public
312 blobActions: [
313 {
314 action: create
315 filePath: "BlobPath"
316 content: "BlobContent"
317 }
318 ]
319 }) {
320 snippet {
321 id
322 title
323 }
324 errors
325 }
326}
327```
328
329GraphQL response:
330
331```json
332{
333 "data": {
334 "createSnippet": null
335 },
336 "errors": [
337 {
338 "message": "Request denied. Spam detected",
339 "locations": [
340 {
341 "line": 22,
342 "column": 5
343 }
344 ],
345 "path": [
346 "createSnippet"
347 ],
348 "extensions": {
349 "spam": true
350 }
351 }
352 ]
353}
354```
355
356### Scenario: allow_possible_spam feature flag enabled
357
358With the `allow_possible_spam` feature flag enabled, the API returns a 200 response. Any
359valid request is successful and no CAPTCHA is presented, even if the request is considered
360spam.