PageRenderTime 35ms CodeModel.GetById 2ms app.highlight 28ms RepoModel.GetById 1ms app.codeStats 0ms

/duckduckhack/spice/spice_basic_tutorial.md

https://github.com/DavidMascio/duckduckgo-documentation
Markdown | 410 lines | 259 code | 151 blank | 0 comment | 0 complexity | e2252c2abede1be1422847e15d142023 MD5 | raw file
  1## Basic Spice Tutorial
  2
  3In this tutorial, we'll be making a Spice instant answer that lets you search for Node.js packages, using the [Node Packaged Modules API](http://registry.npmjs.org/uglify-js/latest). The end result works [like this](https://next.duckduckgo.com/?q=npm+http-server) and the first part, the "backend" component, will look like this:
  4
  5<!-- /summary -->
  6
  7# NPM Spice - Backend (Perl)
  8
  9###### Npm.pm
 10
 11```perl
 12package DDG::Spice::Npm;
 13# ABSTRACT: Returns package information from npm package manager's registry.
 14
 15use DDG::Spice;
 16
 17triggers startend => 'npm';
 18
 19spice to => 'http://registry.npmjs.org/$1/latest';
 20spice wrap_jsonp_callback => 1;
 21
 22handle remainder => sub {
 23    return $_ if $_;
 24    return;
 25};
 26
 271;
 28```
 29
 30## Naming our Spice Package
 31
 32To begin, open your favorite text editor like [gedit](http://projects.gnome.org/gedit/), notepad or [emacs](http://www.gnu.org/software/emacs/) and type the following:
 33
 34```perl
 35package DDG::Spice::Npm;
 36# ABSTRACT: Returns package information from npm package manager's registry.
 37```
 38
 39<!-- /summary -->
 40
 41Each instant answer is a [Perl package](https://duckduckgo.com/?q=perl+package), so we start by declaring the package namespace. For a new Spice (or any new instant answer), you would change **npm** to the name of the instant answer (written in [CamelCase](https://duckduckgo.com/?q=camelcase) format).
 42
 43The second line is a special comment line that is used for documentation purposes.
 44
 45## Import the Spice Class
 46
 47Next, type the following [use statement](https://duckduckgo.com/?q=perl+use) to import [the magic behind](https://github.com/duckduckgo/duckduckgo/tree/master/lib/DDG) our instant answer system.
 48
 49```perl
 50use DDG::Spice;
 51```
 52
 53**\*\*Note:** Right after the above line, you should include any Perl modules that you'll be leveraging to help generate the answer. **Make sure** you add those modules to the `dist.ini` file in this repository. If you're not using any additional modules, carry on!
 54
 55## Define the Trigger Word(s)
 56
 57On the next line, type:
 58
 59```perl
 60triggers startend => 'npm';
 61```
 62
 63**triggers** are keywords/phrases that tell us when to make the instant answer run. When a particular *trigger word* (or phrase) is part of a search query, it tells DuckDuckGo to *trigger* the instant answer(s) that have indicated they should trigger for the given word (or phrase).
 64
 65<!-- /summary -->
 66
 67In this case there is one trigger word: "**npm**". 
 68
 69Let's say someone searched "**npm uglify-js**". "**npm**" is the *first* word, so it would trigger our Spice because the **startend** keyword says, "Make sure the *trigger word* is at the *start*, or the *end*, of the query." 
 70
 71There are several other keywords like **startend** which will be covered shortly. The **=>** symbol is there to separate the trigger words from the keywords (for readability).
 72
 73## Define the API Call
 74
 75On the next line enter:
 76
 77```perl
 78spice to => 'http://registry.npmjs.org/$1/latest';
 79```
 80
 81The **spice to** attribute indicates the API endpoint we'll be using, which means a `GET` request will be made to this URL. The URL has a **$1** placeholder, which will eventually be replaced with whatever string our `handle` function (which we'll define shortly) returns. Generally, Spice instant answers use a search endpoint for a given API and so, we'll need to pass along our search term(s) in the API request. We do this by returning the desired search terms in the `handle` function and then the **$1** placeholder, in the `spice to` URL, will be replaced accordingly.
 82
 83Please note that we prefer to use **HTTP** endpoints, even when an HTTPS endpoint is available.  The faster connections make for a better user experience.  User security and privacy is not violated because all requests are proxied through an HTTPS connection to the DuckDuckGo servers.
 84
 85<!-- /summary -->
 86
 87Using the previous example, if we wanted to search for "**uglify-js**" with the NPM API, we'd need to replace `$1` with `uglify-js` which would result in this URL: <http://registry.npmjs.org/uglify-js/latest>. If you follow that link, you'll notice the API returns a JSON object containing all the data pertaining to the "uglify-js" NPM Package.
 88
 89## Indicate our Callback Function
 90
 91In most cases, APIs allow for a "callback" parameter, which usually looks like this:
 92
 93```perl
 94http://www.api.awesome.com/?q=<search_term>&callback=<function_name>
 95```
 96
 97This parameter is used to wrap the JSON object being returned, in a JavaScript function call to the function named in the `callback` parameter. Unfortunately, the NPM API doesn't allow for this parameter to be specified, so we force this manually by using the **spice wrap\_jsonp\_callback** function. Enter this text on the next line to do this:
 98
 99```perl
100spice wrap_jsonp_callback => 1;
101```
102
103<!-- /summary -->
104
105Now, when the JSON is returned by the API, it will be wrapped in a call to our Spice's JavaScript callback function, which we'll define later.
106
107------
108
109**If the API *did* support the callback parameter**, our `spice to` would look like this:
110
111```perl
112spice to => 'http://registry.npmjs.org/$1/latest?callback={{callback}}';
113```
114
115Where `{{callback}}` is another special placeholder which will be **automatically** replaced with the correct Spice callback function name.
116
117**\*\*Note:** Not every API uses the word "callback" for their callback parameter. Some use "jsonp" (i.e.`&jsonp=myCallbackFn`), but these names are simply convention. Therefore it's best to read the documentation for the API you're using to ensure the correct callback parameter name is used.
118
119## Define the Handle Function
120
121Moving on, enter this on the next line:
122
123```perl
124handle remainder => sub {
125```
126
127Once triggers are specified, we define how to *handle* the query. `handle` is another keyword, similar to **triggers**.
128
129You can *handle* different parts of the search query, but the most common is the **remainder**, which refers to the remainder of the query, after the first matched trigger word/phrase has been removed. 
130
131<!-- /summary -->
132
133For example, if the query was "**npm uglify-js**", the trigger would be *npm* and the **remainder** would be *uglify-js*.
134
135Now let's add a few more lines to complete the handle function:
136
137```perl
138handle remainder => sub {
139    return $_ if $_;
140    return;
141};
142```
143
144This function (the part within the **{}**, after **sub**) is a very important part of the Spice. It defines what our Spice instant answer will replace the **$1** placeholder with.
145
146Whatever you are *handling* is passed to the `handle` function in the **$\_** variable ( **$\_** is a special default variable in Perl that is commonly used to store temporary values). For example, if you searched DuckDuckGo for *"npm uglify-js"*, the value of **$\_** will be *"uglify-js"*, i.e. the **remainder** of the query.
147
148Let's take a closer look at the first line of the function:
149
150```perl
151return $_ if $_;
152```
153
154The heart of the function is just this one line. The **remainder** is in the **$\_** variable as discussed. If it is not blank ( **if $\_** ), we return the **$\_** variable, which as we just saw, contains the **remainder** of the query.
155
156If we are unable to provide a good instant answer, we simply **return nothing**. That's exactly what the second line in the function does:
157
158```perl
159return;
160```
161
162This line is only run if **$\_** contained nothing, because otherwise the line before it would return something and end the function execution.
163
164If for example you searched DuckDuckGo for "npm", this would trigger our NPM Spice, and then in the `handle` function the value of **$\_**, the **remainder** of the query, would be blank (because "npm" is the trigger word and would have been stripped out) and so our handle function would **return nothing**, meaning no API call would be made, and no result will be displayed.
165
166## Return True at EOF
167
168Finally, all Perl packages that load correctly should [return a true value](http://stackoverflow.com/questions/5293246/why-the-1-at-the-end-of-each-perl-package) so add a `1;` on the very last line.
169
170```perl
1711;
172```
173
174And that's it! At this point you have written a functional backend for a Spice instant answer. Now, the frontend components need to written so that we can display our instant answer result on the page.
175
176## Recap (So Far)
177
178The instant answer system works like this at the highest level:
179
180- First, DuckDuckGo breaks down the query (search terms) into separate words.
181
182- Then we see if any of those words or groups of words are **triggers** for any instant answers. These **triggers** are defined by the developer when creating an instant answer. In the example we used above, the trigger word is "**npm**".
183
184- If a Spice is triggered, we run its `handle` function.
185
186- If the Spice's `handle` function returns a value, it is used to replace our **$1** placeholder in the **spice to** URL, and then a request is made to that URL. When the API responds with a JSON object, it is wrapped in a function call, making the JSON object the input to our JavaScript callback function.
187<!-- /summary -->
188
189------
190
191# NPM Spice - Frontend (JavaScript)
192
193As mentioned, every instant answer requires a Spice callback function. For the *NPM* instant answer, the callback function will be named `ddg_spice_npm()` and will be defined in the **npm.js** file. 
194
195The name of the callback function is determined by the **Npm.pm** Perl module we wrote, which specifies the name of the package, `DDG::Spice::Npm`. The portion after `DDG::Spice::` is lower cased and converted from camel case to underscore separated (i.e. `DDG::Spice::CamelCase` -> `ddg_spice_camel_case`) in order to create the name of our callback function. This generated name is what the previous `{{callback}}` placeholder will be replaced with. Similarly, if we instead had to use `spice wrap_jsonp_callback`, that will also wrap the JSON returned by the API with a function call to this generated callback name.
196
197To clarify:
198
199```perl
200{{callback}} => ddg_spice_npm
201
202or
203
204{ JSON_API_RESPONSE: 1, ... } => ddg_spice_npm( { JSON_API_RESPONSE: 1, ... } );
205```
206
207The final **npm.js** will look like this:
208
209###### npm.js
210
211```javascript
212(function (env) {
213    "use strict";
214    env.ddg_spice_npm = function(api_result){
215
216        if (api_result.error) {
217            return Spice.failed('npm');
218        }
219
220        Spice.add({
221            id: "npm",
222            name: "NPM",
223            data: api_result,
224            meta: {
225                sourceName: "npmjs.org",
226                sourceUrl: 'http://npmjs.org/package/' + api_result.name
227            },
228            templates: {
229                group: 'base',
230                options:{
231                    content: Spice.npm.content,
232                    moreAt: true
233                }
234            }
235        });
236    };
237}(this));
238```
239
240Let's go through it line-by-line:
241
242## Invoke an Immediate Function
243
244###### npm.js (continued)
245
246```javascript
247(function (env) {
248    "use strict";
249```
250
251<!-- /summary -->
252
253We begin by invoking an anonymous, immediately-executing function, which takes an object as input (i.e. the environment). This is better known as the JavaScript "[Module Pattern](http://www.addyosmani.com/resources/essentialjsdesignpatterns/book/#modulepatternjavascript)" and it ensures that any variables or functions created within our anonymous function do not affect the global scope. It also allows us to explicitly import anything from the outside environment, which we use to pass along the global scope into our `env` variable (you'll see this at the end), so we can still access and modify the global scope as needed. After creating our module, we then we turn on JavaScript's [Strict Mode](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Functions_and_function_scope/Strict_mode?redirectlocale=en-US&redirectslug=JavaScript%2FReference%2FFunctions_and_function_scope%2FStrict_mode) which also offers various benefits. You don't really need to understand the purpose of these first two lines to create a Spice instant answer; however, they're both necessary and must be included when defining a Spice callback function.
254
255## Define our Callback Function
256
257```javascript
258    env.ddg_spice_npm = function (api_result) {
259```
260
261<!-- /summary -->
262
263We now define our callback function, `ddg_spice_npm`, using a function expression and set it as a property of the `env` object that was passed into our anonymous function. The name we specify here ***must*** match the name of our Spice package, which we discussed above. We also specify that the callback function takes one input parameter, which we have named `api_result`. All Spice callback functions should look like this. The name for the input should **always** be called `api_result`. This is part of our naming convention and helps to standardize the JavaScript code.
264
265## Validate our API Response
266
267###### npm.js (continued)
268
269```javascript
270        if (api_result.error) {
271            return Spice.failed('npm');
272        }
273```
274
275<!-- /summary -->
276
277Here we specify that if the `error` property in the API result is defined (meaning there are no results to use) we `return` a call to `Spice.failed('npm')`, which stops the execution of the function and as a result, won't display an instant answer. It's important to use `Spice.failed();` because this lets the rest of the Spice system know our Spice isn't going to display so that other relevant answers can be given an opportunity to display.
278
279In other cases, because most APIs return an array of results, a similar check should be made to see if the results array has a `length`. This way, if no results were returned from the API we can stop and display nothing.
280
281Moving on, the next part is very important, it defines how the Spice result should look and specifies which parts of the API result are important. This piece of code tells DuckDuckGo to show the instant answer:
282
283## Display our Spice Result (and Specify some Details)
284
285###### npm.js (continued)
286
287```javascript
288        Spice.add({
289            id: "npm",
290            name: "NPM",
291            data: api_result,
292            meta: {
293                sourceName: "npmjs.org",
294                sourceUrl: 'http://npmjs.org/package/' + api_result.name
295            },
296            templates: {
297                group: 'base',
298                options:{
299                    content: Spice.npm.content,
300                    moreAt: true
301                }
302            }
303        });
304```
305
306Here we make a call to the `Spice.add()` function, which operates on an input object that we normally define inside the function call. Let's go over each of the parameters specified in the object we give to `Spice.add()`:
307
308<!-- /summary -->
309
310- `id` is the unique identifier for your Spice instant answer, as convention, we use the name of the spice, just like we do for the callback function.
311
312- `name` is the text to be used for the Spice's AnswerBar tab.
313
314- `data` is the object (meaning it can be an array, because they're technically objects too) that is passed along to the Handlebars template. In this case, the *context* of the NPM template will be the `api_result` object. This is very important to understand because **only the data passed along to the template is accessible to the template**. In most cases the `data` parameter should be set to `api_result` so all the data returned from the API is accessible to the template. 
315
316- `meta:{...}` is used to provide meta-data about the instant answer. This is used for the MetaBar (which appears above the tiles when multiple results are returned) and also for the "More at" link which appears in the detail area of a single result instant answer:
317    
318    + `sourceName` is the name of the source for the "**More at <source>**" link that's displayed for attribution purposes.
319
320    + `sourceUrl` is the target of the "**More at**" link. It's the page that the user will click through to and is intended to provide the user with further information related to the instant answer.
321
322- `templates:{...}` is used to specify which template group, templates and sub-templates are to be used for displaying your Spice instant answer:
323    
324    + `group` is used to specify the template group that we are using. Here', we're using the built-in, **'base'** template group. Doing this lets the templating system know which default, built-in templates should be used for the tile view and detail area, when they are present.
325    
326    + `options:{...}` is used to specify certain things about our templates. Some templates have components which can be turned on and off and some have the ability to include a sub-template, called `content`. In our case, we're indicating the template to be used for `content` and that the `moreAt` component should be turned on.
327        
328        * For `content` we pass along a reference to our handlebars template, `Spice.npm.content`. The templates contained in each Spice's share directory are pre-compiled and added to the global `Spice` object whenever a Spice instant answer is triggered. In order to namespace all the templates, we use a naming hierarchy, so `Spice.npm.content` references the **content.handlebars** template located in the **zeroclickinfo-spice/share/spice/npm/** directory.
329        
330        * Seeing as we've provided a `sourceName` and `sourceURL`, we have the necessary information to create a "**More at**" link and so we tell the template system to enable the creation of it by setting the `moreAt` flag to `true`.
331
332## Global Import
333
334```javascript
335    }
336}(this));
337```
338
339Lastly, we close our callback function expression, as well as our module, and we "import" the global scope, via `this`, as the input to our module. This means within the module, we can access the global scope using the `env` variable defined at the very beginning.
340
341## Define our Handlebars Template
342
343At this point, the rendering of the Spice instant answer changes context from JavaScript to Handlebars.js. As mentioned, our `Spice.add()` call specifies our template and the **context** for the template, so now `Spice.add()` executes the template function using `data` as the input. Let's look at the NPM instant answer's Handlebars template to see how it displays the instant answer result:
344
345<!-- /summary -->
346
347###### content.handlebars
348
349```html
350<h5>
351    {{name}} ({{version}})
352</h5>
353<div>{{description}}</div>
354<pre>$ npm install {{name}}</pre>
355```
356
357As you can see, this is a special type of HTML template where all of `api_result`'s properties (e.g., `version`, `description`) can be accessed by wrapping their respective names in double curly braces. This is possible, because we passed along the `api_result` object (containing all the JSON) to the `data` parameter, which becomes the **context** for our template.
358
359This is what our JSON response looks like, and our template refers by name to the properties of our JSON object:
360
361###### Sample API response from NPM
362
363```json
364{
365
366    "name": "http-server",
367    "version": "0.6.1",
368    "author": {
369        "name": "Nodejitsu",
370        "email": "support@nodejitsu.com"
371    },
372    "description": "a simple zero-configuration command-line http server",
373    ...
374}
375```
376
377For the NPM Spice, we have created a basic HTML skeleton and filled it in with the some useful information, by indicating which variables should go where. `{{name}}`, `{{version}}` and `{{description}}` can also be thought of as placeholders, which will be replaced by their respective values in `api_result`.
378
379**\*\*Note:** When dealing with the detail area, an `<h5>` tag is the biggest heading tag you should use.
380
381## All Done!
382
383And that's it, you're done! You now have a working Spice instant answer! The backend (Perl) tells DuckDuckGo when to trigger our Spice and the frontend (JavaScript & Handlebars) tells DuckDuckGo what to put on the page. Before moving on, lets do a quick recap...
384
385## What We've Accomplished
386
387We've created **one** file in the Spice lib directory, `lib/DDG/Spice/`, named `Npm.pm`, which defines the API to use and the triggers for our Spice and we've created **two** files in the Spice share directory, `share/spice/npm/`:
388
3891. `npm.js` - which delegates the API's response and calls `Spice.add()`
3902. `content.handlebars` - which specifies the instant answer's HTML structure and determines which properties of the API response are placed into the HTML result
391
392Again, the Spice instant answer system works like so:
393
394- First, DuckDuckGo breaks down the query (search terms) into separate words.
395
396- Then we see if any of those words or groups of words are **triggers** for any instant answers. These **triggers** are defined by the developer when creating an instant answer. In the example we used above, the trigger word is "**npm**".
397
398- If a Spice is triggered, we run its `handle` function.
399
400- If the Spice's `handle` function returns a value, it is used to replace our **$1** placeholder in the **spice to** URL, and then a request is made to that URL. When the API responds with a JSON object, it is wrapped in a function call, making the JSON object the input to our JavaScript callback function.
401
402- Our JavaScript callback function takes the API result (JSON Object), and passes it along to our `Spice.add()` call which specifies, using the elements of the JSON object, which parts are to be used in creating the instant answer "More at" link and most importantly, passes along our JSON object to our Spice's Handlebars template.
403
404- The template then defines, using HTML and variables, what the actual content for the Spice result will be
405
406- This content is then created and rendered onto the page by the Spice system's backend.
407
408## Next Steps
409
410Congratulations! You have now written your first Spice instant answer. Now, let's move on to testing, so we can make sure our instant answer functions as expected!