index.html | searchcode

/index.html

https://gitlab.com/vschmidt94/CS494-HowTo-Leaflet-map
HTML | 236 lines | 218 code | 13 blank | 5 comment | 0 complexity | b92e76a4ac7088fb46efc49164d9489a MD5 | raw file

<!DOCTYPE html>
<!-- Name:       C. Vaughan Schmidt
email:      schmicar@onid.oregonstate.edu
course:     CS494-400 Fall 2014
Assignment: Week 5 - Intro to PHP
-->
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<title>schmicar HowTo</title>
<link rel="stylesheet" href="css/style.css">
<link rel="stylesheet" href="css/bootstrap.css">
<link rel="stylesheet" href="http://cdn.leafletjs.com/leaflet-0.7.3/leaflet.css" />
<script src="http://ajax.googleapis.com/ajax/libs/jquery/1.11.1/jquery.min.js"></script>
<script src="js/bootstrap.js"></script>
<script src="http://cdn.leafletjs.com/leaflet-0.7.3/leaflet.js"></script>
<script src="js/Leaflet.MakiMarkers.js"></script>
<script src="js/getLoc.js"></script>
<script src="js/leaflet.GeometryUtil.js"></script>
<script type="text/javascript" src="//cdnjs.cloudflare.com/ajax/libs/gist-embed/1.9/gist-embed.min.js"></script>
  </head>
</head>
<body data-spy="scroll" data-target="#myScollspy">
<div class="container">
  <div class="jumbotron">
    <h2>HowTo: Web Scraping + Leaflet.js</h2>
  </div>
  
  <div class="row" >
    <div class="col-xs-3" id="myScrollspy">
      <ul class="nav nav-tabs nav-stacked" data-spy="affix" data-offset-top="150" data-offset-bottom="0">
        <li><a href="#section-1">Motivation</a></li>
        <li><a href="#section-2">Web Scraping</a></li>
        <li><a href="#section-3">Leaflet.js Intro</a></li>
        <li><a href="#section-4">Markers &amp; Popups</a></li>
        <li><a href="#section-5">Where You At?</a></li>
        <li><a href="#section-6">Nearest Neighbor</a></li>
        <li><a href="#section-7">Recap</a></li>
      </ul>
    </div>
    <div class="col-xs-9" id="content" style="background-color:white;">
      <h2 id="section-1">Motivation</h2>
      
      <h4 class="highlighted2">Note: This guide is not a deep dive into any one single library or technique, but rather the intent is to show how to utilize several libraries and techniques in concert to achieve the result of mapping landmarks on the OSU campus - starting from scratch.</h4>
      <p>
      This how-to guide was developed for my CS494 Web Development class requirements.  My personal objective was to combine what was learned in CS494 Databases with the database project I am also working on.
      </p>
      <p>
      The database project is working towards producing a useful database of all the public artwork installed across the Oregon State University campus in Corvallis, OR.  There is a substantial amount of uncatalogged artwork across campus that has been installed as student projects, or even as part of funded projects, but very few of these pieces are even identified or contain the artist's name or other details.
      </p>
      <p>
      Ultimately, it would be a nice to have a mash-up of picture handling, mapping &amp; GIS services so that the end user could locate nearby artwork from their mobile device.  Additionally, in the case of user-submitted pictures as content, the pictures could be geographically located as well as possibly validated that the picture was indeed taken in the vicinity of Corvallis.  
      </p>
      </p>
      The general steps shown in this HowTo document achieves much of this functionality.
      </p>
      
      <h3>Relevant XKCD</h3>
      <img src="http://imgs.xkcd.com/comics/tasks.png" class="img-responsive inline-block" alt="xkcd comic">
      <p style="text-align:center;">
      Image credit: http://xkcd.com/1425/
      <hr>
      <h2 id="section-2">Web Scraping the OSU Map</h2>
      <p>
      In discussing the database project for CS275, I immediately found myself using the
      <a href="http://oregonstate.edu/campusmap/">OSU Campus Map</a> to orient myself and try to get some idea of how the artwork data could be located and visualized.
      </p>
      <p>
      As soon as I began using the online campus map, it's quickly apparent there is already a LOT of data published to the web for all the major (and minor) campus landmarks.  Exploring the OSU online map leads to
      several discoveries:
      </p>
      
      <ul><li>OSU has unique location codes for *everything*
        <ul><li>Parking lots</li>
        <li>Buildings</li>
        <li>WiFi Hotspots</li>
      </ul>
    </li>
    <li>OSU uses Google's map api's</li>
    <li><strong>Somewhere, there must also be a database with additional information, such as building accessibility details, as some buildings have additional information available.</strong>
      <ul><li>Example: Selecting the Women's Building on the online map</li></ul>
    </li>
  </ul>
  <p id='video1'>
  It's much easier to show in a video how I went about discovering / uncovering the AJAX requests and JSON responses that the OSU Campus Map uses.  
  </p>
  <iframe width="420" height="315" src="//www.youtube.com/embed/QhdgAH7yw2A"
  allowfullscreen style="display: block; margin: 0 auto;"></iframe>
  <p>
  Once the AJAX URL request format &amp; JSON response has been discovered (example: <a href="http://oregonstate.edu/campusmap/map/element?locations=814">http://oregonstate.edu/campusmap/map/element?locations=814</a>), it is straightforward to build a little PHP script to iterate through successive calls to the URL, parse the returned JSON and then recombine just the portions I am interested in.
  </p>
  The results of the crawler script can be seen here:
  <a href="http://web.engr.oregonstate.edu/~schmicar/cs494/HowTo/bldgcrawler.php">
    http://web.engr.oregonstate.edu/~schmicar/cs494/HowTo/bldgcrawler.php
  </a>.
  <p>
  The PHP code for the web scraping is listed below.
  </p>
  
  <p>
  <em>
  Note that what I really am after here is a format that can just be appended to a MySQL
  </em>
      <pre>
        INSERT INTO ... VALUES .... 
</pre>
  <em>
  query statement, so that is why I've formatted the output a certain way.  You could just as easily format it into JSON or something else.  (Also it took a few initial attemps to determine I really was only interested in landmark id's 447-1038...)
  </em>
  </p>
  <h3>bldgcrawler.php Script Listing</h3>
  
  <code data-gist-id="bf53163182bfcbfab579"></code>
  
  <p>
  <strong>Disclaimer</strong>: I <em>really</em> should go about getting official OK to use the OSU campus map website in this manner.  However, due to expediency of the class projects, and the fact this is education-based &amp; non-commercial, I believe it falls under fair-use provisions of applicable copyright laws.  I mention this as a reminder that there might be potential legal issues when it comes to web-scraping and data harvesting.
  </p>
  <hr>
  <h2 id="section-3">Introducing Leaflet.js</h2>
  <p>
  <a href="http://leafletjs.com/">Leaflet.js</a> "is a modern open-source JavaScript library for mobile-friendly interactive maps." (and from the various mapping API's I checked into, Leaflet looks very user-friendly and understandable...)
  </p>
  <p>
  Before you can use Leaflet.js - you need to go sign up at <a href="http://www.mapbox.com">mapbox.com</a> for your unique Map ID, which Leaflet needs to have to produce maps for the web application.  Grab your new Map ID, and you will be ready to go.
  </p>
  <div id="map1">
  </div>
  <script src="./js/tutMap1.js"></script>
  <p>
  Adding the above map to the web page was as simple as adding a div for the map and calling a .js script in the .html file:
  </p>
  
  <code data-gist-id="e51148f749953299375a"></code>
  
  <p>
  The javascript file contains:
  </p>
  <code data-gist-id="4f1e97dd1860bcdab14a"></code>
  
  Note that I plugged in some initial latitude / longitude based on the data I had scraped from the OSU online campus map.
  <hr>
  <h2 id="section-4">Markers &amp; Popups</h2>
  <p>
  At this point, I would like to add some of my harvested landmarks as
  pushpins to the map.  Additionally, I am going to have the landmark
  open as a popup if a user clicks on the pushpin marker.
  </p>
  <div id="map2">
  </div>
  <div>
    <script src="./js/tutMap2.js"></script>
  </div>
  The Javascript code to accomplish this is very straightforward:
  <ul>
    <li>Add a few landmarks as array of objects (real application would
      be using a database for this, just a simple array for demo).
    </li>
    <li>Loop through the array:
      <ul>
        <li>Add a leaflet marker (L.marker) for every landmark object.
        </li>
        <li>Bind a popup to each of the markers.
        </li>
      </ul>
    </li>
  </ul>
  <code data-gist-id="5b4b82787cd94cbd48a3" data-gist-highlight-line="30-31"></code>
  <hr>
  <h2 id="section-5">Where You At?</h2>
  <p>
Now, let's incorporate the user's position on the map.  Most modern browsers are capable of geolocation using something along the lines of:
</p> 
<code data-gist-id="d190ba0ec26097726683" data-gist-line="11-18" data-gist-highlight-line="14"></code>
<p> Once we get the user's cordinates (lat, long), now we have everything we need to start putting all the parts together.  I've attached the getLocation() to the button below, and when clicked, it will:
</p>
<ol>
<li>Get user location using the navigator.geolocation interface (if available) </li>
<li>Above will set a callback to my showPosition() function where:</li>
<ol><li>Set the innerHTML to display the user's position in text form on web page</li>
<li>Add a leaflet.js map to the page (map3)</li>
<li>Add markers for all my destination landmarks to the map, using the MakiMarkers.js plug-in (gives more colors and glyphs than stock leaflet.js marker)</li>
<li>Add a different MakiMarker (rocket glyph) to the map for the user's location.</li>
<li>Set the map's boundaries such that the proper bounds are selected that will allow ALL markers to display on the map. </li>
<ol><li> To do this, I add all markers to an array, and add the array of markers to a leaflet feature group.</li>
<li>Then, I can call method .fitBounds(group.getBounds().pad(0.5)) to have the map calculate the approriate bounds to encompass the markers.</li>
</ol>
<li>As long as I am here, I go ahead and calculate the nearest neighbor (see below for more on nearest neighbor)</li>
</ol>
</ol>
<h4 class="highlighted2">
<strong>Note that the user should get a popup asking to allow location services, etc.</strong>
</h4>
  <div id="getLoc">
<button onclick="getLocation()" class="highlighted">Get My Location!</button>
  </div>
  <div id="map3">
  </div>
<p>
<br>
If you noticed the more colorful markers in the new map, this was done with the Leaflet.MakiMarkers.js plugin.  This JavaScript file is
  <a href="https://github.com/jseppi/Leaflet.MakiMarkers">available here on GitHub.</a>  The bit of code that sets the MakiMarker user location icon (rocket) on the Leaflet map is here:
</p>
<code data-gist-id="d190ba0ec26097726683" data-gist-line="43-46" data-gist-highlight-line="44"></code>
<hr>
<h2 id="section-6">Nearest Neighbor</h2>
  <div id="nearestMarker">
  </div>
  <p>
    Finally, as long as I was working with the user's geo location, I went ahead and used another GeometryUtil.js method to return the closest marker to the user's location.  
  </p>
  <p>
    It's worth noting that the closest() method returns a simple object just containing the latitude and longitude of the closest marker.  I have to take that information, and loop back through my collection of points comparing the lat, long values of each point to the lat, long of the closest to determine which one of my points actually *was* the closest point.  Due to the floating point calculations involved in the various manipulations, I test for similarities by allowing a delta variance of under 1E-14 degrees - which in terms of actually feet and inches would be much smaller than a gnat's ass... so I should not be accidently confusing marker points unless they are literally the exact same spot. 
  </p>
    <code data-gist-id="d190ba0ec26097726683" data-gist-line="58-79" data-gist-highlight-line="59"></code>
  <hr>
  <h2 id="section-7">Recap</h2>
  The intent of this section is to provide a quick summary of all the various libraries, tools, etc that were employed to achieve the final result - a map of OSU campus landmarks + the current user geolocation, with the landmark closest to the user identified and distance computed.

  <ul>
  <li><strong>Chrome Dev Tools</strong> - used to uncover the AJAX calls and JSON responses the official campus map web page is using.</li>
  <li><strong>PHP Script</strong> - quick and dirty PHP script to replicate the AJAX calls discovered above.  Scans the range of location id's and produces a dataset useable elsewhere.</li>
  <li><strong><a href="http://leafletjs.com/">leaflet.js</a></strong> - JavaScript library for easy to use mapping.</li>
  <li><strong><a href="https://www.mapbox.com/">mapbox.com</a></strong> - Have to register here to get a map id code.  Supplies the map tiles to our leaflet map.</li>
  <li><strong><a href="https://developer.mozilla.org/en-US/docs/Web/API/Geolocation">HTML5 Geolocation API</a></strong> - A HTML5 API providing geolocation information ability when used with most modern browsers.</li>
  <li><strong><a href="https://github.com/jseppi/Leaflet.MakiMarkers">MakiMarkers.js</a></strong> - A plugin library for leaflet that enhances the markers (pushpins) on the map.  Adds color and glyphs</li>
  <li><strong><a href="http://makinacorpus.github.io/Leaflet.GeometryUtil/">Leaflet.GeometryUtil.js</a></strong> - A plugin library for leaflet adds a lot of computational function.  Used to find the nearest neighbor and distance between a pair of markers.</li>
  <li><strong><a href="https://github.com/blairvanderhoof/gist-embed">gist-embed.js</a></strong> - JavaScript Library used on this webpage to provide line highlighting, line slection on embedded GitHub Gist code listings.</li>
  <li><strong><a href="http://getbootstrap.com/">bootstrap.js</a></strong> - JavaScript Library used on this webpage to provide the basic webpage framework.</li>
  </ul>

  <hr>
  <h2 id="section-7">Code Listing</h2>
  <p>All code used in this project is available at <a href="https://github.com/vschmidt94/CS494-HowTo-Leaflet-map.git">this GitHub Repository</a>.  Some GitHub Gists have been used for convenience in the body of this document.

</div>
</div>
</body>