/atom.xml

<?xml version="1.0" encoding="utf-8"?>
<feed xmlns="http://www.w3.org/2005/Atom">

  <title><![CDATA[Trey Hunner]]></title>
  <link href="http://treyhunner.com/atom.xml" rel="self"/>
  <link href="http://treyhunner.com/"/>
  <updated>2014-06-16T18:24:10-07:00</updated>
  <id>http://treyhunner.com/</id>
  <author>
    <name><![CDATA[Trey Hunner]]></name>
    
  </author>
  <generator uri="http://octopress.org/">Octopress</generator>

  
  <entry>
    <title type="html"><![CDATA[CLI for Finding DRM-free Audiobooks]]></title>
    <link href="http://treyhunner.com/2014/05/cli-for-finding-drm-free-audiobooks/"/>
    <updated>2014-05-14T12:31:00-07:00</updated>
    <id>http://treyhunner.com/2014/05/cli-for-finding-drm-free-audiobooks</id>
    <content type="html"><![CDATA[<p>I recently acquired an appreciation for audiobooks.  I listen to multiple
audiobooks every month and I prefer DRM-free audiobooks so that I can listen
through my favorite audiobook reader on each of my devices.</p>

<p><a href="http://www.downpour.com/">Downpour</a> and <a href="http://www.emusic.com/">eMusic</a> are the only services I know that provide a large
selection of DRM-free audiobooks.  Unfortunately not all audiobooks are
available on either of these sites, so I often end up searching both sites for
each book to discover what versions (if any) are available from each.  I got
sick of searching both websites all the time, so I just created a Python script
to do that work for me.</p>

<div><script src='https://gist.github.com/21f307b975027be5162d.js'></script>
<noscript><pre><code>#!/usr/bin/env python
&quot;&quot;&quot;
Search downpour.com and emusic.com for DRM-free audiobooks

Usage::

    ./find_audiobooks.py &lt;title&gt;...

File released to the public domain under CC0 license:
http://creativecommons.org/publicdomain/zero/1.0/deed

&quot;&quot;&quot;
import sys
from itertools import chain, izip_longest
import urllib2

from bs4 import BeautifulSoup
from purl import URL


def unescape(text):
    &quot;&quot;&quot;Return string without smart apostrophes&quot;&quot;&quot;
    return text.replace(u'\u2019', &quot;'&quot;)


def get_downpour_url(book_name):
    &quot;&quot;&quot;Return search URL for downpour.com&quot;&quot;&quot;
    base = URL(&quot;http://www.downpour.com/catalogsearch/result/&quot;)
    return base.query_param('q', book_name).as_string()


def get_emusic_url(book_name):
    &quot;&quot;&quot;Return search URL for emusic.com&quot;&quot;&quot;
    base_url = URL(&quot;http://www.emusic.com/search/book/&quot;)
    return base_url.query_param('s', book_name).as_string()


def search_downpour(book_name):
    &quot;&quot;&quot;Search Downpour and return list of parsed results&quot;&quot;&quot;
    response = urllib2.urlopen(get_downpour_url(book_name))
    page = BeautifulSoup(response)
    books = page.find_all('li', attrs={'class': &quot;item&quot;})
    results = []
    for book in books:
        header = book.find(attrs={'class': &quot;product-name&quot;})
        link_tag = header.find('a')
        title = ' '.join(unescape(x)
                         for x in link_tag.stripped_strings)
        link = link_tag['href']
        for author in book.find_all(attrs={'class': 'author'}):
            author_text = author.text
            if author_text.startswith('By '):
                author = author_text[3:]
        results.append({
            'title': title,
            'link': link,
            'author': author,
        })
    return results


def search_emusic(book_name):
    &quot;&quot;&quot;Search eMusic and return list of parsed results&quot;&quot;&quot;
    response = urllib2.urlopen(get_emusic_url(book_name))
    page = BeautifulSoup(response)
    books = page.find_all('li', attrs={'class': &quot;bundle&quot;})
    results = []
    for book in books:
        link_tag = book.find('h4').find('a')
        author_tag = book.find('h5')
        results.append({
            'title': link_tag.text,
            'link': link_tag['href'],
            'author': author_tag.text,
        })
    return results


def print_result(result):
    &quot;&quot;&quot;Print title, author, and link for audiobook result&quot;&quot;&quot;
    print &quot;Title: {}&quot;.format(result['title'])
    print &quot;Author: {}&quot;.format(result['author'])
    print &quot;Link: {}&quot;.format(result['link'])
    print


def merge_lists(*lists):
    &quot;&quot;&quot;Return merge of lists by alternating elements of each&quot;&quot;&quot;
    combined_lists = chain.from_iterable(izip_longest(*lists))
    return list(filter(bool, combined_lists))


def main(*args):
    &quot;&quot;&quot;Search audiobook sites and return search results&quot;&quot;&quot;
    for book_name in args:
        results1 = search_emusic(book_name)
        results2 = search_downpour(book_name)
        results = merge_lists(results1[:3], results2[:3])
        for result in results:
            print_result(result)


if __name__ == &quot;__main__&quot;:
    main(*sys.argv[1:])
</code></pre></noscript></div>


<h3>Future Improvements</h3>

<p>Some ideas for future improvements:</p>

<ul>
<li>Add <a href="http://www.audible.com/">Audible</a> results when the book cannot be found in a DRM-free format</li>
<li>Rewrite the script in JavaScript and create a Chrome extension out of it</li>
<li>Use <a href="https://github.com/kennethreitz/clint">clint</a> to colorize the command-line output</li>
</ul>

]]></content>
  </entry>
  
  <entry>
    <title type="html"><![CDATA[Supporting Both Django 1.7 and South]]></title>
    <link href="http://treyhunner.com/2014/03/migrating-to-django-1-dot-7/"/>
    <updated>2014-03-27T13:05:00-07:00</updated>
    <id>http://treyhunner.com/2014/03/migrating-to-django-1-dot-7</id>
    <content type="html"><![CDATA[<p>Have an open source Django app with South migrations?  Adding support for Django 1.7 might be a little painful.  In this post I will discuss the difficulty of supporting Django 1.7 while maintaining South migrations for users of Django 1.6 and below.</p>

<p>Django 1.7 uses the <code>migrations</code> sub-package in your app for database migrations and South relies on the same package.  Unfortunately, you can&rsquo;t store both packages in the same place.  At first glance, it seems we cannot support both Django 1.7 and previous versions of Django using South.  However, as I explain below, we can support both at once.</p>

<h2>Assessing your options</h2>

<p>In order to support both Django 1.7 and Django 1.6 with South we can rename the <code>migrations</code> package and instruct users to reference the new package in their settings module.  We can do this with the <a href="https://docs.djangoproject.com/en/1.7/ref/settings/#std:setting-MIGRATION_MODULES">MIGRATION_MODULES</a> or <a href="http://south.readthedocs.org/en/latest/settings.html#south-migration-modules">SOUTH_MIGRATION_MODULES</a> settings.  There are three options:</p>

<ol>
<li>Move existing <code>migrations</code> directory to <code>south_migrations</code> and create Django 1.7 migrations in <code>migrations</code> package</li>
<li>Create new Django 1.7 migrations package in <code>django_migrations</code> directory and leave existing South migrations package</li>
<li>Move existing <code>migrations</code> directory to <code>south_migrations</code> and create Django 1.7 migrations in <code>django_migrations</code> directory</li>
</ol>


<p>The first option requires existing users either switch to Django 1.7 or update their settings module before upgrading to the new version of your app.  The second option requires all Django 1.7 users to customize their settings module to properly install your app.  The third option requires everyone (both Django 1.7 and South users) to update their settings module.</p>

<p>Out of those options I prefer the first one.  When you eventually drop support for South, you will probably want your Django 1.7 migrations to live in the <code>migrations</code> directory.  If you don&rsquo;t force that switch now, you would eventually need to break backwards-compatibility or maintain two duplicate migrations directories.</p>

<p>So our plan is to move the South migrations to <code>south_migrations</code> and create Django 1.7 migrations.  An example with the <a href="https://github.com/treyhunner/django-email-log">django-email-log</a> app:</p>

<figure class='code'><figcaption><span></span></figcaption><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
<span class='line-number'>2</span>
<span class='line-number'>3</span>
</pre></td><td class='code'><pre><code class='bash'><span class='line'><span class="nv">$ </span>git mv email_log/migrations email_log/south_migrations
</span><span class='line'><span class="nv">$ </span>python manage.py makemigrations email_log
</span><span class='line'><span class="nv">$ </span>git add email_log/migrations
</span></code></pre></td></tr></table></div></figure>


<h2>Breaking South support</h2>

<p>If you move <code>migrations</code> to <code>south_migrations</code> and make a Django 1.7 <code>migrations</code> package, what happens to existing users with South?</p>

<p>Your new app upgrade will break backwards compatibility for South users and you want to make sure they <em>know</em> they need to make a change immediately after upgrading.  Users should see a loud and clear error message instructing them what they need to do.  This can be done by hijacking their use of the <strong>migrate</strong> command with South.</p>

<p>Existing users will run <strong>migrate</strong> when upgrading your app.  If they don&rsquo;t migrate immediately, they will when they notice a problem and realize they need to run <strong>migrate</strong>.  Upon migrating, we want to show a clear error message telling the user what to do.</p>

<h2>Failing loudly and with a clear error message</h2>

<p>When South looks for app migrations it will import our <code>migrations</code> package.  Our <code>migrations</code> package contains Django 1.7 migrations, which South won&rsquo;t understand.  So we want to make sure that if our <code>migrations</code> package is imported either Django 1.7 is installed or a proper error message is displayed.  Upon importing this package, we can check for the presence of the new <code>django.db.migrations</code> module and if not found we will raise an exception with a descriptive error message.</p>

<p>For example, this is the code I plan to add to the <code>email_log/migrations/__init__.py</code> file for <a href="https://github.com/treyhunner/django-email-log">django-email-log</a> to add Django 1.7 support:</p>

<figure class='code'><figcaption><span></span></figcaption><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
<span class='line-number'>2</span>
<span class='line-number'>3</span>
<span class='line-number'>4</span>
<span class='line-number'>5</span>
<span class='line-number'>6</span>
<span class='line-number'>7</span>
<span class='line-number'>8</span>
<span class='line-number'>9</span>
<span class='line-number'>10</span>
<span class='line-number'>11</span>
<span class='line-number'>12</span>
<span class='line-number'>13</span>
<span class='line-number'>14</span>
<span class='line-number'>15</span>
<span class='line-number'>16</span>
<span class='line-number'>17</span>
<span class='line-number'>18</span>
<span class='line-number'>19</span>
<span class='line-number'>20</span>
<span class='line-number'>21</span>
</pre></td><td class='code'><pre><code class='python'><span class='line'><span class="sd">&quot;&quot;&quot;</span>
</span><span class='line'><span class="sd">Django migrations for email_log app</span>
</span><span class='line'>
</span><span class='line'><span class="sd">This package does not contain South migrations.  South migrations can be found</span>
</span><span class='line'><span class="sd">in the ``south_migrations`` package.</span>
</span><span class='line'><span class="sd">&quot;&quot;&quot;</span>
</span><span class='line'>
</span><span class='line'><span class="n">SOUTH_ERROR_MESSAGE</span> <span class="o">=</span> <span class="s">&quot;&quot;&quot;</span><span class="se">\n</span><span class="s"></span>
</span><span class='line'><span class="s">For South support, customize the SOUTH_MIGRATION_MODULES setting like so:</span>
</span><span class='line'>
</span><span class='line'><span class="s">    SOUTH_MIGRATION_MODULES = {</span>
</span><span class='line'><span class="s">        &#39;email_log&#39;: &#39;email_log.south_migrations&#39;,</span>
</span><span class='line'><span class="s">    }</span>
</span><span class='line'><span class="s">&quot;&quot;&quot;</span>
</span><span class='line'>
</span><span class='line'><span class="c"># Ensure the user is not using Django 1.6 or below with South</span>
</span><span class='line'><span class="k">try</span><span class="p">:</span>
</span><span class='line'>    <span class="kn">from</span> <span class="nn">django.db</span> <span class="kn">import</span> <span class="n">migrations</span>  <span class="c"># noqa</span>
</span><span class='line'><span class="k">except</span> <span class="ne">ImportError</span><span class="p">:</span>
</span><span class='line'>    <span class="kn">from</span> <span class="nn">django.core.exceptions</span> <span class="kn">import</span> <span class="n">ImproperlyConfigured</span>
</span><span class='line'>    <span class="k">raise</span> <span class="n">ImproperlyConfigured</span><span class="p">(</span><span class="n">SOUTH_ERROR_MESSAGE</span><span class="p">)</span>
</span></code></pre></td></tr></table></div></figure>


<p>Now when we run <strong>migrate</strong> with Django 1.6 and South, we&rsquo;ll see the following exception raised:</p>

<figure class='code'><figcaption><span></span></figcaption><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
<span class='line-number'>2</span>
<span class='line-number'>3</span>
<span class='line-number'>4</span>
<span class='line-number'>5</span>
<span class='line-number'>6</span>
<span class='line-number'>7</span>
</pre></td><td class='code'><pre><code class='python'><span class='line'><span class="n">django</span><span class="o">.</span><span class="n">core</span><span class="o">.</span><span class="n">exceptions</span><span class="o">.</span><span class="n">ImproperlyConfigured</span><span class="p">:</span>
</span><span class='line'>
</span><span class='line'><span class="n">For</span> <span class="n">South</span> <span class="n">support</span><span class="p">,</span> <span class="n">customize</span> <span class="n">the</span> <span class="n">SOUTH_MIGRATION_MODULES</span> <span class="n">setting</span> <span class="n">like</span> <span class="n">so</span><span class="p">:</span>
</span><span class='line'>
</span><span class='line'>    <span class="n">SOUTH_MIGRATION_MODULES</span> <span class="o">=</span> <span class="p">{</span>
</span><span class='line'>        <span class="s">&#39;email_log&#39;</span><span class="p">:</span> <span class="s">&#39;email_log.south_migrations&#39;</span><span class="p">,</span>
</span><span class='line'>    <span class="p">}</span>
</span></code></pre></td></tr></table></div></figure>


<h2>Conclusion</h2>

<p>This breaks backwards compatibility, but our users should immediately understand what has broken and how to fix it.  Remember to upgrade the major number of your package version to note this backwards-incompatible change.</p>

<p>I would love to hear your thoughts about this approach in the comments below.  Let me know if you have other ideas about how to handle supporting Django 1.7 migrations and South at the same time.</p>
]]></content>
  </entry>
  
  <entry>
    <title type="html"><![CDATA[TDD With Django Tutorial]]></title>
    <link href="http://treyhunner.com/2013/11/tdd-with-django-workshop/"/>
    <updated>2013-11-04T00:00:00-08:00</updated>
    <id>http://treyhunner.com/2013/11/tdd-with-django-workshop</id>
    <content type="html"><![CDATA[<p>I helped host a free Test-Driven Django Web Development workshop on <time date="2013-11-02">Saturday November 2</time> with <a href="http://pythonsd.org/">San Diego Python</a>.  We created a series of tutorials demonstrating how to create a Django-powered blog while practicing test-driven development.  The <a href="http://python.org/psf/">Python Software Foundation</a> sponsored the event and the <a href="http://aicenterca.com/">Ansir Innovation Center</a> provided a venue.</p>

<p>You can find the tutorials at <a href="http://bit.ly/pysd-tdd">http://bit.ly/pysd-tdd</a> .  The tutorials are provided under a <a href="https://creativecommons.org/licenses/by-sa/3.0/">CC BY-SA license</a> so you can reuse and modify them for your own purposes.</p>

<p>Tutorial markdown files and working source code may be found on <a href="https://github.com/pythonsd/test-driven-django-development">Github</a>.  We plan to improve and extend these tutorials for a future workshop.  If you have ideas for improvements/additions or if you notice a bug, please submit an issue or open a pull request.</p>
]]></content>
  </entry>
  
  <entry>
    <title type="html"><![CDATA[Visual Integration Tests for Django]]></title>
    <link href="http://treyhunner.com/2013/10/visual-integration-tests-for-django/"/>
    <updated>2013-10-03T15:19:00-07:00</updated>
    <id>http://treyhunner.com/2013/10/visual-integration-tests-for-django</id>
    <content type="html"><![CDATA[<p>I recently added a new type of test to my testing arsenal: visual tests.  Visual tests ensure the CSS, markup, and JavaScript produce a webpage that looks right.</p>

<h2>Visual testing frameworks</h2>

<p>Visual testing tools compare screenshots to ensure tested webpages look pixel perfect.  Capturing webpage screenshots requires a full-featured web browser to render CSS and execute JavaScript.  All three of the visual testing tools I found rely on Selenium or PhantomJS for rendering.</p>

<h3>PhantomCSS</h3>

<p><a href="https://github.com/Huddle/PhantomCSS">PhantomCSS</a> uses PhantomJS for screenshot differencing.  PhantomCSS won&rsquo;t integrate directly with the Django live server or your Python test suite, so if you want to run a visual integration test, you&rsquo;d need to manually start and stop the test server between tests.  I might eventually try out PhantomCSS for CSS unit tests, but I wanted to visually test my full website so I needed integration with the Django live server.</p>

<h3>Django-casper</h3>

<p><a href="https://github.com/dobarkod/django-casper">Django-casper</a> uses Django live server tests to execute CasperJS test files (which use PhantomJS) to compare screenshots.  Each test requires an additional Python test which references a JavaScript file that executes the navigation and screenshotting code.  I found this approach messy and difficult to setup.</p>

<h3>Needle</h3>

<p>The <a href="https://github.com/bfirsh/needle">needle</a> Python library uses Selenium to navigate your website and screenshot rendered pages.  Unfortunately needle has poor test coverage, a seemingly failing test suite, and no change log.  Despite these shortcomings, I went with needle for my visual integration tests because it got the job done.</p>

<h2>Django and Needle</h2>

<p>I used the following mixin to integrate the Django live server with needle.  I used PhantomJS, but Firefox or another Selenium web driver should work as well.</p>

<figure class='code'><figcaption><span></span></figcaption><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
<span class='line-number'>2</span>
<span class='line-number'>3</span>
<span class='line-number'>4</span>
<span class='line-number'>5</span>
<span class='line-number'>6</span>
<span class='line-number'>7</span>
<span class='line-number'>8</span>
<span class='line-number'>9</span>
<span class='line-number'>10</span>
<span class='line-number'>11</span>
<span class='line-number'>12</span>
<span class='line-number'>13</span>
<span class='line-number'>14</span>
</pre></td><td class='code'><pre><code class='python'><span class='line'><span class="kn">from</span> <span class="nn">django.test</span> <span class="kn">import</span> <span class="n">LiveServerTestCase</span>
</span><span class='line'><span class="kn">from</span> <span class="nn">needle.cases</span> <span class="kn">import</span> <span class="n">NeedleTestCase</span>
</span><span class='line'><span class="kn">from</span> <span class="nn">selenium.webdriver</span> <span class="kn">import</span> <span class="n">PhantomJS</span>
</span><span class='line'>
</span><span class='line'>
</span><span class='line'><span class="k">class</span> <span class="nc">DjangoNeedleTestCase</span><span class="p">(</span><span class="n">NeedleTestCase</span><span class="p">,</span> <span class="n">LiveServerTestCase</span><span class="p">):</span>
</span><span class='line'>
</span><span class='line'>    <span class="sd">&quot;&quot;&quot;Needle test case for use with Django live server&quot;&quot;&quot;</span>
</span><span class='line'>
</span><span class='line'>    <span class="n">driver</span> <span class="o">=</span> <span class="n">PhantomJS</span>
</span><span class='line'>
</span><span class='line'>    <span class="nd">@classmethod</span>
</span><span class='line'>    <span class="k">def</span> <span class="nf">get_web_driver</span><span class="p">(</span><span class="n">cls</span><span class="p">):</span>
</span><span class='line'>        <span class="k">return</span> <span class="nb">type</span><span class="p">(</span><span class="s">&#39;NeedleWebDriver&#39;</span><span class="p">,</span> <span class="p">(</span><span class="n">NeedleWebDriverMixin</span><span class="p">,</span> <span class="n">cls</span><span class="o">.</span><span class="n">driver</span><span class="p">),</span> <span class="p">{})()</span>
</span></code></pre></td></tr></table></div></figure>


<p>Unfortunately the above code only works with the version of needle on Github.  The PyPI version does not yet include the <code>NeedleWebDriverMixin</code> (which I contributed recently for Django support).  I have created <a href="https://github.com/bfirsh/needle/issues/13">an issue</a> suggesting a new PyPI release be made to resolve this problem.</p>

<h2>Room for improvement</h2>

<p>Currently I only run my visual tests manually.  Visual tests are very brittle and occasionally they just break without any changes.  If I manage to stabilize my visual tests so that they pass consistently on different platforms, I may run them during continuous integration.</p>

<p>Do you have another solution for visual integration testing?  Let me know in the comments.</p>
]]></content>
  </entry>
  
  <entry>
    <title type="html"><![CDATA[Extensible JSON Encoder Using Single-dispatch Functions]]></title>
    <link href="http://treyhunner.com/2013/09/singledispatch-json-serializer/"/>
    <updated>2013-09-27T00:00:00-07:00</updated>
    <id>http://treyhunner.com/2013/09/singledispatch-json-serializer</id>
    <content type="html"><![CDATA[<p>Single-dispatch generic functions will be added to Python 3.4 (as proposed in <a href="http://www.python.org/dev/peps/pep-0443/">PEP 443</a>).  When reading about single-dispatch functions I immediately thought of the difficulties I&rsquo;ve had with custom JSON encoders.  Below I explain why custom JSON encoders can complicate your application and how single-dispatch functions could be used to create a simpler JSON encoder.</p>

<h2>JSON Encoding using the json library</h2>

<p>With the current Python <code>json</code> library, using an extensible JSON encoder in your generic application may require some/all of the following:</p>

<ul>
<li>Allowing specification of custom encoder class by client applications</li>
<li>Overriding the default JSON encoder class (or a client-specified one) for any further extensions</li>
<li>Passing JSON encoder classes into other serialization libraries used by your application</li>
</ul>


<h3>Combining JSON encoders</h3>

<p>If you need to compose two custom JSON encoders specified in two different packages, you may need to:</p>

<ul>
<li>Use multiple inheritance and hope the encoders play nicely together</li>
<li>Duplicate code from one of the packages and create a new serializer with single inheritance</li>
<li>Monkey patch one or both of the libraries</li>
</ul>


<h2>JSON encoder using single-dispatch generic functions</h2>

<p>I created a wrapper around the <code>json</code> library to make a JSON encoder using single-dispatch generic functions.  Here&rsquo;s how to use it:</p>

<div><script src='https://gist.github.com/6734816.js?file=example.py'></script>
<noscript><pre><code>from decimal import Decimal

from json_singledispatch import encode


@encode.register(set)
def encode_set(obj):
    return encode(list(obj))


@encode.register(Decimal)
def encode_dict(obj):
    return encode(str(obj))


print encode({'key': &quot;value&quot;})
print encode({5, 6})
print encode(Decimal(&quot;5.6&quot;))</code></pre></noscript></div>


<p>As you can see, it&rsquo;s fairly easy to extend the encoder to understand serialization rules for new data types.</p>

<p>The impementation is fairly simple, albeit a hack:</p>

<div><script src='https://gist.github.com/6734816.js?file=json_singledispatch.py'></script>
<noscript><pre><code>import json
from singledispatch import singledispatch


class _CustomEncoder(json.JSONEncoder):
    def default(self, obj):
        for obj_type, handler in encode.registry.items():
            if isinstance(obj, obj_type) and obj_type is not object:
                return handler(obj)
        return super(_CustomEncoder, self).default(obj)


@singledispatch
def encode(obj, **kwargs):
    return json.dumps(obj, cls=_CustomEncoder, **kwargs)</code></pre></noscript></div>


<p>This code is intended as a proof-of-concept to demonstrate the power of single-dispatch generic functions.  Feel free to use it however you like.</p>

<h2>Related Links</h2>

<ul>
<li><a href="http://lukasz.langa.pl/8/single-dispatch-generic-functions/">What single-dispatch generic functios mean for you</a></li>
<li><a href="http://julien.danjou.info/blog/2013/python-3.4-single-dispatch-generic-function">Python 3.4 single dispatch, a step into generic functions</a></li>
</ul>

]]></content>
  </entry>
  
  <entry>
    <title type="html"><![CDATA[Test-Inspired Development]]></title>
    <link href="http://treyhunner.com/2013/07/test-inspired-development/"/>
    <updated>2013-07-28T00:00:00-07:00</updated>
    <id>http://treyhunner.com/2013/07/test-inspired-development</id>
    <content type="html"><![CDATA[<p>You need to write tests for your code to demonstrate that it works as expected.  In this post I will note the method I usually use for writing code and writing tests.</p>

<p>I follow a method of development similar to test-driven development, which I call &ldquo;test-inspired development&rdquo;.  Here&rsquo;s roughly what I do:</p>

<ol>
<li>Write a bit of code</li>
<li>Write a bit of a unit test for your code</li>
<li>Ensure the test succeeds, but fails when the code is commented out or git stash-ed</li>
<li>Repeat steps 1, 2, and 3 as needed</li>
<li>Start writing an integration test, fleshing out as far as you can</li>
<li>Make sure the integration test fails at the correct point</li>
<li>Repeat steps 5 and 6 as needed</li>
<li>Make sure no commit you make along the way contains both tests and code</li>
<li>Use git rebase if necessary to ensure the test commits come before the code commits</li>
</ol>


<p>Ideally I would write all of my test code first and then write my application code and ensure my tests pass.  That isn&rsquo;t what usually happens though.  I often find it tricky to write full tests before writing any code.  Even when writing code in small parts, sometimes my code gets written before the corresponding test code.</p>

<p>I write all of my code in feature branches (short-lived branches forked from master) and I often create a pull request (even against my own repository) when creating a new feature.  Because I commit the tests first, a code reviewer can easily confirm that all the new tests actually fail correctly without the code present by simply rolling back to the last test commit and running the tests.</p>

<p>I usually write my unit tests first and at or near the same time as my code because they help often help shape my code heavily.  The functional/integration tests are important for ensuring that the application actually works all the way through (working parts are useless on their own).</p>

<p>Have questions about my testing methodology?  Do you use a different technique?  Feel free to share your thoughts in the comments below.</p>
]]></content>
  </entry>
  
  <entry>
    <title type="html"><![CDATA[Maintaining an Open Source Project]]></title>
    <link href="http://treyhunner.com/2013/07/maintaining-an-open-source-project/"/>
    <updated>2013-07-21T00:00:00-07:00</updated>
    <id>http://treyhunner.com/2013/07/maintaining-an-open-source-project</id>
    <content type="html"><![CDATA[<p>This post contains some of my opinions on how to maintain an open source project well.  I&rsquo;ve built up my opinions on this topic from maintaining and contributing to open source projects.  Keep in mind that I am not an expert on this topic.</p>

<h2>Structure your project logically</h2>

<p>You shouldn&rsquo;t need to worry about structuring your project because others have done that work for you.  <strong>Base your project&rsquo;s design on other well-organized projects.</strong></p>

<p>I compiled some tips I try to follow for open source Django projects I maintain in my <a href="https://github.com/treyhunner/django-skeleton-app">django-skeleton-app</a> repository.  Check it out if you&rsquo;re starting a new Python project.</p>

<h2>Make it very easy to contribute</h2>

<p>Document your process for contributing code, filing bugs, and maintaining other feedback.  Reference this process in your README file, your documentation, and your <a href="https://github.com/blog/1184-contributing-guidelines">CONTRIBUTING</a> file.  If you need a place to discuss project development outside of the issue tracker, consider creating a mailing list.</p>

<h2>Be kind</h2>

<p>Thank your contributors, appologize if you make a big mistake or say something rude, and be very courteous.  Add all your contributors to the AUTHORS file and grant push access to active contributors.</p>

<h2>Be Responsive</h2>

<p>Don&rsquo;t have time to look over a new pull request this week?  Make a comment noting when you&rsquo;ll give your feedback.  This way your contributor can add a friendly reminder later without feeling rude.</p>

<p>Realized you forgot to respond to a 12 month old pull request?  Respond right now and ask whether the code or problem is still relevant.  A late response is better than none at all.</p>

<p>Don&rsquo;t know if you&rsquo;ll ever have time to look at an issue?  Say so!  <strong>Do not leave your contributors in the dark!</strong></p>

<h2>Automate all the things</h2>

<p>Use:</p>

<ul>
<li><a href="https://travis-ci.org/">Travis CI</a>: run continuous integration tests for all pushes and pull requests</li>
<li><a href="https://coveralls.io/">Coveralls</a>: measure code coverage while running your CI tests</li>
<li>A documentation site like <a href="https://readthedocs.org/">Read The Docs</a> which auto-updates on pushes</li>
</ul>


<h2>Learn from others</h2>

<p>I&rsquo;m not the only one with opinions about open source.</p>

<p>Read more opinions on maintaining an open source project:</p>

<ul>
<li><a href="https://segment.io/blog/tips-for-maintaining-an-open-source-library/">Tips for maintaining an Open-Source library</a></li>
<li><a href="http://www.codesimplicity.com/post/open-source-community-simplified/">Open Source Community, Simplified</a></li>
<li><a href="https://hacks.mozilla.org/2013/05/how-to-spread-the-word-about-your-code/">How to Spead The Word About Your Code</a></li>
</ul>


<p>Watch some videos about maintaining an open source project:</p>

<ul>
<li><a href="http://www.youtube.com/watch?v=xgWFTrXn0_U">Maintaining Your Sanity While Maintaining Your Open Source App</a></li>
<li><a href="http://pyvideo.org/video/1795/write-the-docs">Write the Docs</a></li>
</ul>

]]></content>
  </entry>
  
  <entry>
    <title type="html"><![CDATA[Log All Outgoing Emails in Django]]></title>
    <link href="http://treyhunner.com/2013/05/django-email-log/"/>
    <updated>2013-05-20T00:00:00-07:00</updated>
    <id>http://treyhunner.com/2013/05/django-email-log</id>
    <content type="html"><![CDATA[<p>Ever needed to determine whether an email was sent from a Django project?  I
made a Django application that does exactly that: <a href="https://github.com/treyhunner/django-email-log">django-email-log</a>.</p>

<p>I got the idea from <a href="http://stackoverflow.com/a/7553759/98187">a StackOverflow answer</a> and I decided to make a real
application out of it.  All emails are stored in a single model which can
easily be viewed, searched, sorted, and filtered from the admin site.</p>

<p>I used test-driven development when making the app and I baked in Python 3
support from the beginning.  I found the process of TDD for a standalone
Python package fairly easy and enjoyable.</p>
]]></content>
  </entry>
  
  <entry>
    <title type="html"><![CDATA[Django-simple-history Is Back]]></title>
    <link href="http://treyhunner.com/2013/05/django-simple-history/"/>
    <updated>2013-05-01T00:00:00-07:00</updated>
    <id>http://treyhunner.com/2013/05/django-simple-history</id>
    <content type="html"><![CDATA[<p>I wrote <a href="http://treyhunner.com/2011/09/django-and-model-history/">a post</a> over a year ago about recording a history of changes for Django model instances.  I evaluated three different Django packages to record model history.  My favorite of the options, django-simple-history, was abandoned and development continued through multiple forks.</p>

<p>I recently attempted to revive <a href="https://github.com/treyhunner/django-simple-history">django-simple-history</a>.  I added tests, put it <a href="https://pypi.python.org/pypi/django-simple-history/">on PyPI</a>, and made it easier to use with newer versions of Django.  I moved my fork of the project to git and Github, added Travis and Coveralls support for continuous integration and code coverage tracking, and noted future features on the issue tracker.</p>

<p>Soon after I started writing tests for the project I received feature requests, issues, pull requests, and emails with words of encouragement.  I appreciate all of the help I&rsquo;ve had while reviving the project.  I plan to remain responsive to the suggestions for my fork of the code.  If you&rsquo;d like to help out with the project please feel free to submit an issue, make a pull request, or comment on the code commits on the <a href="https://github.com/treyhunner/django-simple-history">Github page</a>.</p>
]]></content>
  </entry>
  
  <entry>
    <title type="html"><![CDATA[Random Name Generator Website]]></title>
    <link href="http://treyhunner.com/2013/03/pseudorandom.name/"/>
    <updated>2013-03-10T00:00:00-08:00</updated>
    <id>http://treyhunner.com/2013/03/pseudorandom.name</id>
    <content type="html"><![CDATA[<p>In my <a href="http://treyhunner.com/2013/02/random-name-generator/">last post</a> I discussed the <strong>names</strong> Python library that generates
random names.  I&rsquo;ve now used this Python library to make a basic Flask website
that generates random names.  You can visit it at:
<a href="http://www.pseudorandom.name">http://www.pseudorandom.name</a></p>

<h2>It&rsquo;s responsive</h2>

<p>The font size and margin on pseudorandom.name change based on the web browser
width and height.  I determined the font sizes I wanted to use for each screen
width by using the longest first and last name in the name files I&rsquo;m using (11
and 13 characters respectively).  The website looks reasonable on various
desktop resolutions and on phone screens.</p>

<h2>It&rsquo;s an HTTP API</h2>

<p>If curl is used to access the site, plain text is returned instead of an HTML
webpage.  For example:</p>

<pre><code>curl www.pseudorandom.name
</code></pre>

<p>The site is hosted on Heroku which <a href="https://devcenter.heroku.com/articles/custom-domains#apex-domains-examplecom">doesn&rsquo;t support bare domains</a>.
Currently <a href="http://pseudorandom.name">http://pseudorandom.name</a> redirects to <a href="http://www.pseudorandom.name">http://www.pseudorandom.name</a> so
using the bare domain requires telling curl to follow redirects with <code>-L</code>:</p>

<pre><code>curl -L pseudorandom.name
</code></pre>

<h2>More to come?</h2>

<p>It might be nice to allow generating multiple names at once, generating
gender-specific names, and maybe providing other content types (JSON).  The
HTML version could also use a cleaner, more feature-full design.</p>

<p>Also it would probably be more efficient to use a SQL database for querying
random names so I may eventually abandon the names library and use a database
for querying.</p>

<p>The website&rsquo;s source code is <a href="https://github.com/treyhunner/pseudorandom.name">hosted on Github</a> and provided under an
<a href="http://th.mit-license.org/2013">MIT license</a>.  Feel free to fork it or submit an issue.</p>
]]></content>
  </entry>
  
  <entry>
    <title type="html"><![CDATA[Random Name Generator in Python]]></title>
    <link href="http://treyhunner.com/2013/02/random-name-generator/"/>
    <updated>2013-02-17T00:00:00-08:00</updated>
    <id>http://treyhunner.com/2013/02/random-name-generator</id>
    <content type="html"><![CDATA[<p>I&rsquo;ve used multiple websites to generate random names for my test data when
running manual or automated QA tests.</p>

<p>Since discovering DuckDuckGo&rsquo;s <a href="https://duckduckgo.com/?q=random+word">random word</a> generation, I&rsquo;ve
<a href="https://duckduckhack.uservoice.com/forums/5168-ideas-for-duckduckgo-instant-answer-plugins/suggestions/2850418-random-name">hoped</a> that someone would make a DuckDuckGo plugin to generate
random names also.</p>

<p>I didn&rsquo;t want to write my own DuckDuckGo plugin yet so I made a Python-powered
command line tool instead using <a href="https://www.census.gov/genealogy/www/data/1990surnames/index.html">1990 Census data</a>.</p>

<p>The program is called <code>names</code> and can be found on <a href="https://github.com/treyhunner/names">Github</a> and <a href="http://pypi.python.org/pypi/names/0.1">PyPI</a>.</p>

<h3>It&rsquo;s really simple</h3>

<p>It&rsquo;s basically just one file currently that&rsquo;s <a href="https://github.com/treyhunner/names/blob/f99542dc21f48aa82da4406f8ce408e92639430d/names/__init__.py">about 40 lines long</a>.
There is only one feature available from the command line currently: generate a
single random full name.  There&rsquo;s a few more features if importing as a Python
package: generate random last name or generate random first name (with or
without specifying gender), generate random full name (also without or without
gender).</p>

<p>The random name picker relies on the cumulative frequencies listed in the
included Census data files.  Here&rsquo;s the steps that are taken:
1. A random floating point number is chosen between 0.0 and 90.0
2. Name file lines are iterated through until a cumulative frequency is found
that is less than the randomly generated number
3. The name on that line is chosen and returned (or printed out)</p>

<h3>Examples</h3>

<p>Here&rsquo;s how you use it from the command line:</p>

<pre><code>$ names
Kara Lopes
</code></pre>

<p>Here&rsquo;s how you use it as a Python package:</p>

<pre><code>&gt;&gt;&gt; import names
&gt;&gt;&gt; names.get_full_name()
u'Patricia Halford'
&gt;&gt;&gt; names.get_full_name(gender='male')
u'Patrick Keating'
&gt;&gt;&gt; names.get_first_name()
'Bernard'
&gt;&gt;&gt; names.get_first_name(gender='female')
'Christina'
&gt;&gt;&gt; names.get_last_name()
'Szczepanek'
</code></pre>
]]></content>
  </entry>
  
  <entry>
    <title type="html"><![CDATA[Tmuxstart]]></title>
    <link href="http://treyhunner.com/2012/12/tmuxstart/"/>
    <updated>2012-12-15T00:00:00-08:00</updated>
    <id>http://treyhunner.com/2012/12/tmuxstart</id>
    <content type="html"><![CDATA[<p>I&rsquo;ve been using a helper script to manage all of my tmux sessions for the last
few months (nearly since the time I switched from screen to tmux).</p>

<h3>A tmuxinator alternative</h3>

<p>I use a separate session for each project I work on and I wanted a way to
easily connect to project sessions with the same default windows and
environment variables each time.  I tried using <a href="https://github.com/aziz/tmuxinator">tmuxinator</a>, but I found it
to be too complicated for my needs.  I had trouble working within the
limitations of the yaml files and I found the installation to be overly
complicated (especially when working on a new server that didn&rsquo;t have ruby
installed).  After I realized how simple it would be to solve my own problem I
made the tmuxstart script.</p>

<h3>Now available on Github</h3>

<p>While updating my <a href="https://github.com/treyhunner/dotfiles">dotfiles</a> repository I realized I hadn&rsquo;t yet incorporated
my tmuxstart workflow into my <code>.tmux.conf</code> file.  Yesterday I decided to make
README and LICENSE files for my script and put it under version control.  I
also added some helper functions to make the script even easier to use.  It&rsquo;s
now available in my <a href="https://github.com/treyhunner/tmuxstart">tmuxstart</a> repository on Github.</p>

<h3>Example usage</h3>

<p>Below is an example of how to use tmuxstart.</p>

<ol>
<li>Add the following line to your <code>~/.tmux.conf</code> file:</li>
</ol>


<figure class='code'><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
</pre></td><td class='code'><pre><code class=''><span class='line'>bind S command-prompt -p "Make/attach session:" "new-window 'tmuxstart \'%%\''"</span></code></pre></td></tr></table></div></figure>


<ol>
<li>Create a <code>~/.tmuxstart</code> directory:</li>
</ol>


<figure class='code'><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
</pre></td><td class='code'><pre><code class=''><span class='line'>mkdir ~/.tmuxstart</span></code></pre></td></tr></table></div></figure>


<ol>
<li>Create a file <code>~/.tmuxstart/main</code> with the following contents:</li>
</ol>


<figure class='code'><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
<span class='line-number'>2</span>
</pre></td><td class='code'><pre><code class=''><span class='line'>new_session htop
</span><span class='line'>new_window</span></code></pre></td></tr></table></div></figure>


<p>Now you can create a tmux session called &ldquo;main&rdquo; with <a href="http://htop.sourceforge.net/">htop</a> in the first
window and a shell in the second window by executing:</p>

<pre><code>tmuxstart main
</code></pre>

<p>Or from an existing tmux server type <code>&lt;PREFIX&gt; S</code> (<code>&lt;PREFIX&gt;</code> is Ctrl-B by
default), type <code>main</code> and hit Enter.</p>

<p>When using either of these methods if a session called &ldquo;main&rdquo; already exists
the existing session will be used instead.</p>

<h3>How I use this</h3>

<p>I have a separate tmuxstart session file for each of my Django projects and
some of my open source projects also.  I use a &ldquo;main&rdquo; session file similar to
the one above and I have Gnome Terminal set to start with the custom command
&ldquo;tmuxstart main&rdquo; instead of a shell.</p>
]]></content>
  </entry>
  
  <entry>
    <title type="html"><![CDATA[Maintaining Consistent Coding Conventions With EditorConfig]]></title>
    <link href="http://treyhunner.com/2012/02/editorconfig/"/>
    <updated>2012-02-17T00:00:00-08:00</updated>
    <id>http://treyhunner.com/2012/02/editorconfig</id>
    <content type="html"><![CDATA[<p>I often have indentation issues when coding because I work with many open
source (and closed source) projects that have different identation styles.  For
example, my <a href="https://github.com/treyhunner/jquery-formrestrict">jQuery formrestrict</a> uses 4 spaces for indentation,
<a href="https://github.com/kswedberg/jquery-expander">jQuery expander</a> uses 2 spaces, and <a href="https://github.com/gbirke/jquery_pagination">jQuery pagination</a> uses hard tabs.
I don&rsquo;t want to change my text editor&rsquo;s indentation settings every time I open
one of these files and using a plugin to auto-detect indentation is only a
partial solution.  To solve this problem I started a project I call
EditorConfig.</p>

<p><a href="http://editorconfig.org">EditorConfig</a> defines coding conventions for groups of files and instructs
text editors to adhere to these conventions.  To solve my problem, I just
create a file called <code>.editorconfig</code> that sets indentation for my files and
with my EditorConfig plugin installed Vim takes care of the rest.  Here&rsquo;s an
example <code>.editorconfig</code> file I could add to my project that uses
<a href="https://github.com/kswedberg/jquery-expander">jQuery expander</a> and <a href="https://github.com/gbirke/jquery_pagination">jQuery pagination</a>:</p>

<figure class='code'><figcaption><span>.editorconfig</span></figcaption><div class="highlight"><table><tr><td class="gutter"><pre class="line-numbers"><span class='line-number'>1</span>
<span class='line-number'>2</span>
<span class='line-number'>3</span>
<span class='line-number'>4</span>
<span class='line-number'>5</span>
<span class='line-number'>6</span>
<span class='line-number'>7</span>
<span class='line-number'>8</span>
<span class='line-number'>9</span>
<span class='line-number'>10</span>
<span class='line-number'>11</span>
</pre></td><td class='code'><pre><code class='ini'><span class='line'><span class="k">[*.js]</span>
</span><span class='line'><span class="na">indent_style</span> <span class="o">=</span> <span class="s">space</span>
</span><span class='line'><span class="na">indent_size</span> <span class="o">=</span> <span class="s">4</span>
</span><span class='line'>
</span><span class='line'><span class="k">[jquery.expander.js]</span>
</span><span class='line'><span class="na">indent_style</span> <span class="o">=</span> <span class="s">space</span>
</span><span class='line'><span class="na">indent_size</span> <span class="o">=</span> <span class="s">2</span>
</span><span class='line'>
</span><span class='line'><span class="k">[jquery.pagination.js]</span>
</span><span class='line'><span class="na">indent_style</span> <span class="o">=</span> <span class="s">tab</span>
</span><span class='line'><span class="na">indent_size</span> <span class="o">=</span> <span class="s">4</span>
</span></code></pre></td></tr></table></div></figure>


<p>With this <code>.editorconfig</code> file, all JavaScript files use 4 spaces for
indentation except for <code>jquery.expander.js</code> which uses 2 spaces and
<code>jquery.pagination.js</code> which uses hard tabs with a column width of 4.  If I put
my <code>.editorconfig</code> file under version control, other developers working on my
project can see the coding conventions I defined and if their text editor has
an EditorConfig plugin installed their editor will automatically use the
correct indentation as well.</p>

<p>Example EditorConfig files can be seen in my own projects (such as
<a href="https://github.com/treyhunner/dotfiles/blob/master/.editorconfig">in my dotfiles repo</a>) and in the
<a href="https://github.com/treyhunner/dotfiles/blob/master/.editorconfig">various EditorConfig plugin codebases</a>.  More
information on EditorConfig can be found at the
<a href="http://editorconfig.org">EditorConfig website</a>.</p>

<p>EditorConfig plugins are <a href="http://editorconfig.org/#download">available for 6 different editors</a>
now.  If you like the idea, <a href="http://editorconfig.org">try out EditorConfig</a> for your
favorite editor and <a href="https://groups.google.com/forum/?fromgroups#!forum/editorconfig">tell us what you think</a>.  If you don&rsquo;t like
the idea or have a problem with EditorConfig please <a href="https://github.com/editorconfig/editorconfig/issues">submit an issue</a>,
add a thread to <a href="https://groups.google.com/forum/?fromgroups#!forum/editorconfig">the mailing list</a>, or send me an email voicing
your concern.</p>
]]></content>
  </entry>
  
  <entry>
    <title type="html"><![CDATA[Migrating From Subversion to Git]]></title>
    <link href="http://treyhunner.com/2011/11/migrating-from-subversion-to-git/"/>
    <updated>2011-11-17T00:00:00-08:00</updated>
    <id>http://treyhunner.com/2011/11/migrating-from-subversion-to-git</id>
    <content type="html"><![CDATA[<p>I recently migrated multiple Subversion repositories to Git.  I found
<a href="http://blog.woobling.org/2009/06/git-svn-abandon.html" title="Migrating from Subversion to Git">this blog post</a> very helpful during the process.  Here are
some tips I found helpful that were not mentioned in that post.</p>

<h3>Generating an authors file</h3>

<p>Subversion denotes authors by usernames while Git uses name and email.  An
authors file can be used to map subversion usernames to Git authors when
creating the Git repository, like so:</p>

<pre><code>git-svn clone --stdlayout --authors-file=authors.txt http://your/svn/repo/url
</code></pre>

<p>The trickest part about <a href="http://triptico.com/notes/8d4510bb.html">making an authors file</a> was finding
all the authors.  I found this command useful for finding usernames of all
Subversion committers:</p>

<pre><code>svn log | awk '($0 ~ /^r/) {print $3}' | sort -u
</code></pre>

<p>A similar method of creating an authors file is
<a href="http://technicalpickles.com/posts/creating-a-svn-authorsfile-when-migrating-from-subversion-to-git/">presented here</a>.</p>

<h3>Removing git-svn-id messages</h3>

<p>When migrating from Subversion, every commit messages has a <code>git-svn-id</code> line
appended to it like this one:</p>

<blockquote><p>git-svn-id: <a href="http://svn/repo/url/trunk@9837">http://svn/repo/url/trunk@9837</a> 1eab27b1-3bc6-4acd-4026-59d9a2a3569e</p></blockquote>

<p>If you are planning on migrating away from your old Subversion repository
entirely, there&rsquo;s no need to keep these.  The following command (taken from the
<a href="http://linux.die.net/man/1/git-filter-branch">git filter-branch man page</a>) removes these <code>git-svn-id</code>
lines from all commit messages in the current branch:</p>

<pre><code>git filter-branch -f --msg-filter 'sed -e "/git-svn-id:/d"'
</code></pre>

<h3>Removing empty commit messages</h3>

<p>Subversion allows empty commit messages, but Git does not.  Any empty commit
messages in your newly migrated git repository should be replaced so commands
like <code>git rebase</code> will work on these commits.</p>

<p>This command will replace all empty commit messages with <nobr>
&ldquo;&lt;empty commit message&gt;&rdquo;</nobr>:</p>

<pre><code>git filter-branch -f --msg-filter '
read msg
if [ -n "$msg" ] ; then
    echo "$msg"
else
    echo "&lt;empty commit message&gt;"
fi'
</code></pre>

<h3>Poking around</h3>

<p>After you&rsquo;ve cleaned up your new <em>master</em> branch, you should cleanup other
branches you plan to keep.  Just <code>git checkout</code> each branch and repeat the same
steps.  To find the remote subversion branches available for checkout use
<code>git branch -a</code>.</p>

<p>Migrating between version control systems may be a good time to permanently
cleanup commit history with a <code>git rebase</code> or eliminate large files
that were never used with a <code>git filter-branch</code>.  Just remember to make backups of
previous working versions of branches before changing them, just in case.</p>
]]></content>
  </entry>
  
  <entry>
    <title type="html"><![CDATA[Django and Model History]]></title>
    <link href="http://treyhunner.com/2011/09/django-and-model-history/"/>
    <updated>2011-09-29T00:00:00-07:00</updated>
    <id>http://treyhunner.com/2011/09/django-and-model-history</id>
    <content type="html"><![CDATA[<p><strong>May 2013 Update:</strong> <a href="http://treyhunner.com/2013/05/django-simple-history/">django-simple-history is back</a></p>

<p>Recently I had a need to store a snapshot of every state of particular model instance in a Django application.  I basically needed version control for the rows in my database tables.  When searching for applications that provided this feature, which I call <strong>model history</strong>, I found <a href="http://djangopackages.com/grids/g/model-audit/">many different approaches</a> but few good comparisons of them.  In an attempt to fill that void, I&rsquo;m going to detail some of my findings here.</p>

<h3>django-reversion</h3>

<p>The <a href="https://github.com/etianen/django-reversion">django-reversion</a> application was started in 2008 by Dave Hall.  Reversion uses only one table to store data for all version-tracked models.  Each version of a model adds a new row to this table, using a JSON object representing the model state.  Models can be reverted to previous versions from the admin interface.  This single-table version structure makes django-reversion very easy to install and to uninstall, but it also creates <a href="http://groups.google.com/group/django-reversion/browse_thread/thread/922b4e42d9577e0b">problems when model fields are changed</a>.</p>

<h3>django-revisions</h3>

<p>The <a href="https://github.com/stdbrouw/django-revisions">django-revisions</a> application was created by Stijn Debrouwere in 2010 because the existing Django model history applications at the time were abandoned or suffered from fundamental design problems.  Revisions uses a model history method called same-table versioning (<a href="http://stdbrouw.github.com/django-revisions/design.html">design details outlined here</a>).  Same-table versioning adds a few fields to each version-tracked model which allows it to record the most recent version of each model as well as old versions in the original model table.  Model changes are simplified because they change all versions at once and no new tables need to be added to use revisions (just new fields on existing tables).  The only problem I found with revisions was that it does not currently support database-level uniqueness constraints.  Adding <code>unique=True</code> to a model field or a <code>unique_together</code> Meta attribute will result in an error.  Currently uniqueness constraints must be specified in a separate way for Revisions to honor them when saving models.</p>

<h3>django-simple-history</h3>

<p>The <a href="https://bitbucket.org/q/django-simple-history/">django-simple-history</a> application was based on code originally written by Marty Alchin, author of Pro Django.  Marty Alchin posted <a href="https://code.djangoproject.com/wiki/AuditTrail">AuditTrail</a> on the Django trac wiki in 2007 and later revised and republished the code in his book Pro Django in 2008, renaming it to HistoricalRecords.  Corey Bertram created django-simple-history from this code and put it online in 2010.</p>

<p>Simple History works by creating a separate &ldquo;historical&rdquo; model for each model that requires an audit trail and storing a snapshot of each changed model instance in that historical model.  For example, a Book model would have a HistoricalBook created from it that would store a new HistoricalBook instance every time a Book instance was changed.  Collisions are avoided by disabling uniqueness constraints and model schema changes are accepted by automatically changing historical models as well.  This method comes at the cost of creating an extra table in the database for each model that needs history.</p>

<h3>My conclusions</h3>

<p>When testing these three applications myself, I immediately eliminated django-reversion because I needed to allow easy model schema changes for my project.  I found that both django-revisions and django-simple-history worked well with schema migrations through <a href="http://south.aeracode.org/docs/about.html">South</a> (which I use on everything).  Django-revisions worked better for data migrations in South (due to only needing to change one model), but the uniqueness constraint problems with django-revisions would have been problematic for some of my models.  So eventually I settled on <a href="https://bitbucket.org/q/django-simple-history/">django-simple-history</a>.</p>
]]></content>
  </entry>
  
  <entry>
    <title type="html"><![CDATA[Sharing Screenshots in Linux]]></title>
    <link href="http://treyhunner.com/2011/04/sharing-screenshots-in-linux/"/>
    <updated>2011-04-03T00:00:00-07:00</updated>
    <id>http://treyhunner.com/2011/04/sharing-screenshots-in-linux</id>
    <content type="html"><![CDATA[<p>I have been using Github Issues recently and loving its simplicity.  Unfortunately, I&rsquo;ve found that I often need to upload screenshots to demonstrate bugs and Issues does not support file uploads.  There are <a href="http://wiki.dropbox.com/DropboxAddons/DropboxScreenGrabber">Windows</a> and <a href="http://www.getcloudapp.com/">Mac</a> applications that solve this problem by capturing a screenshot, uploading it, and copying a URL to access the screenshot to the clipboard.</p>

<p>I did not find any Linux applications that will capture/upload a screenshot and copy the URL but I discovered <a href="http://forums.dropbox.com/topic.php?id=21735">a thread in the Dropbox forums</a> with a script that does just that.  I added comments to the script, changed the variable names, removed the need for a temporary file, and added a <code>notify-send</code> call as a visual cue (should work on Ubuntu).  I have the script mapped to <kbd>Ctrl-PrtScrn</kbd> in Ubuntu.</p>

<div><script src='https://gist.github.com/892492.js'></script>
<noscript><pre><code>#!/bin/sh
# Ubuntu-specific modification of http://wiki.dropbox.com/TipsAndTricks/ShareScreenshots

# Change these
DB_USER_ID=YOURDBUSERID
BITLY_USERNAME=YOURBITLYUSERNAME
BITLY_API_KEY=YOURBITLYKEYHERE
DROPBOX_PUBLIC_DIR=~/Dropbox/Public
SCREENSHOT_DIR=screenshots

CAPTURE_DELAY=0
PICTURE_QUALITY=50
FILE_EXTENSION=png
TIME=$(date +%Y%m%d%H%M%S)
FILENAME=$TIME.$FILE_EXTENSION

# Move to the directory where screenshots will be stored
mkdir -p $DROPBOX_PUBLIC_DIR/$SCREENSHOT_DIR
cd $DROPBOX_PUBLIC_DIR/$SCREENSHOT_DIR

# Take screenshot and save in screenshot directory
scrot -d $CAPTURE_DELAY -q $PICTURE_QUALITY $FILENAME

# Get Dropbox public URL for screenshot
DB_URL=&quot;http://dl.dropbox.com/u/$DB_USER_ID/$SCREENSHOT_DIR/$FILENAME&quot;

# Get bit.ly shortened URL for Dropbox URL
ESCAPED_DB_URL=&quot;$(echo $DB_URL | sed 's,:,%3A,g;s,/,%2F,g')&quot;
BITLY_API_CALL=&quot;http://api.bit.ly/v3/shorten?login=$BITLY_USERNAME&amp;apiKey=$BITLY_API_KEY&amp;longUrl=$ESCAPED_DB_URL&amp;format=txt&quot;
SHORT_URL=$(curl -s -S $BITLY_API_CALL)

# Copy shortened URL to clipboard
echo $SHORT_URL | xclip -sel clip

# Display message to user (requires libnotify)
notify-send &quot;Screenshot added&quot; &quot;Screenshot link copied to clipboard: $SHORT_URL&quot;
</code></pre></noscript></div>



]]></content>
  </entry>
  
  <entry>
    <title type="html"><![CDATA[Encrypted Private Keys in Django]]></title>
    <link href="http://treyhunner.com/2010/12/encrypted-private-keys-in-django/"/>
    <updated>2010-12-11T00:00:00-08:00</updated>
    <id>http://treyhunner.com/2010/12/encrypted-private-keys-in-django</id>
    <content type="html"><![CDATA[<p>Uniquely identifiable URLs are necessary for many web applications.  For example, a website that provides book reviews may identify the URL of a specific book like this: <strong>www.example.com/books/8839/</strong>.  The easiest way to identify entities in Django is to use the unique primary key of each object, which by default is an auto-incremented positive integer.</p>

<p>Revealing the primary key of an entity is often not desirable.  An astute visitor of the website mentioned above may be able to guess information from the URL such as how many book reviews are available on the website or how old specific reviews are.</p>

<p>The code snippet below demonstrates one way to use a unique but cryptic identifier for an object without needing to change the way primary keys are generated.  There are two notable extensions to the basic Django Model in the below code:</p>

<ol>
<li>The encrypted_pk and encrypted_id model properties return an AES-encrypted version of the primary key as a 13 character base-36 string.</li>
<li>The get method of the default manager can be queried with an encrypted primary key by using the keyword argument encrypted_pk.</li>
</ol>


<p>Feel free to use this code however you want.</p>

<div><script src='https://gist.github.com/735861.js'></script>
<noscript><pre><code># This code is under the MIT license.
# Inspired by this StackOverflow question:
http://stackoverflow.com/questions/3295405/creating-django-objects-with-a-random-primary-key

import struct
from Crypto.Cipher import DES
from django.db import models


def base36encode(number):
    &quot;&quot;&quot;Encode number to string of alphanumeric characters (0 to z). (Code taken from Wikipedia).&quot;&quot;&quot;
    if not isinstance(number, (int, long)):
        raise TypeError('number must be an integer')
    if number &lt; 0:
        raise ValueError('number must be positive')

    alphabet = '0123456789abcdefghijklmnopqrstuvwxyz'
    base36 = ''
    while number:
        number, i = divmod(number, 36)
        base36 = alphabet[i] + base36

    return base36 or alphabet[0]


def base36decode(numstr):
    &quot;&quot;&quot;Convert a base-36 string (made of alphanumeric characters) to its numeric value.&quot;&quot;&quot;
    return int(numstr,36)


class EncryptedPKModelManager(models.Manager):
    &quot;&quot;&quot;This manager allows models to be identified based on their encrypted_pk value.&quot;&quot;&quot;
    def get(self, *args, **kwargs):
        encrypted_pk = kwargs.pop('encrypted_pk', None)
        if encrypted_pk:
            # If found, decrypt encrypted_pk argument and set pk argument to the appropriate value
            kwargs['pk'] = struct.unpack('&lt;Q', self.model.encryption_obj.decrypt(
                struct.pack('&lt;Q', base36decode(encrypted_pk))
            ))[0]
        return super(EncryptedPKModelManager, self).get(*args, **kwargs)


class EncryptedPKModel(models.Model):
    &quot;&quot;&quot;Adds encrypted_pk property to children which returns the encrypted value of the primary key.&quot;&quot;&quot;
    encryption_obj = DES.new('8charkey') # This 8 character secret key should be changed!

    def __init__(self, *args, **kwargs):
        super(EncryptedPKModel, self).__init__(*args, **kwargs)
        setattr(
            self.__class__,
            &quot;encrypted_%s&quot; % (self._meta.pk.name,),
            property(self.__class__._encrypted_pk)
        )

    def _encrypted_pk(self):
        return base36encode(struct.unpack('&lt;Q', self.encryption_obj.encrypt(
            str(struct.pack('&lt;Q', self.pk))
        ))[0])

    encrypted_pk = property(_encrypted_pk)

    class Meta:
        abstract = True


class ExampleModelManager(EncryptedPKModelManager):
    pass


class ExampleModel(EncryptedPKModel):
    objects = ExampleModelManager()
    example_field = models.CharField(max_length=32)


# Example usage:
# example_instance = ExampleModel.objects.get(pk=1)
# url_pk = example_instance.encrypted_pk
# ExampleModel.objects.get(encrypted_pk=url_pk)
</code></pre></noscript></div>



]]></content>
  </entry>
  
  <entry>
    <title type="html"><![CDATA[Replacement for jQuery AlphaNumeric Plugin]]></title>
    <link href="http://treyhunner.com/2010/10/replacement-for-jquery-alphanumeric-plugin/"/>
    <updated>2010-10-18T00:00:00-07:00</updated>
    <id>http://treyhunner.com/2010/10/replacement-for-jquery-alphanumeric-plugin</id>
    <content type="html"><![CDATA[<p>I recently inherited a codebase that used the <a href="http://plugins.jquery.com/project/aphanumeric">jQuery AlphaNumeric plugin</a> extensively.  This plugin can be used to restrict users from entering certain characters into a form field.   The functions included for this plugin (<strong>alphanumeric</strong>, <strong>alpha</strong>, and <strong>numeric</strong>) claim to allow only alphabetic and/or numeric characters to be entered in the form field being acted on.</p>

<p>Unfortunately, this plugin is ineffectual.  I have witnessed unexpected behaviors of varying significance associated with this plugin:</p>

<ol>
<li>Forbidden characters can be input by pasting with CTRL+V in Chrome</li>
<li>Forbidden characters can be input by selecting Edit-&gt;Paste in Firefox and IE</li>
<li>Forbidden characters can be input by middle-click pasting in Linux</li>
<li>The arrow keys, Home button, End button, and Delete button do not work in input fields using this plugin&rsquo;s functions in Firefox</li>
<li>The context menu is disabled (right clicking on an input field does nothing)</li>
<li>And most importantly, instead of using a list of allowed characters, a list of disallowed characters is used so these are the only characters that are actually forbidden:

<blockquote><p>!@#$%^&amp;*()+=[]&lsquo;;,/{}|&ldquo;:?~`.&ndash;</p></blockquote></li>
</ol>


<p>The code is so brief that patching the current plugin would be pointless, so instead I wrote <a href="http://github.com/treyhunner/jquery-formrestrict">a replacement</a> that acts similar enough that I did not have to change any of our pre-existing code that depended on the AlphaNumeric plugin.</p>

<p>First I created <strong>restrict</strong>, a very basic modifier function that takes a sanitizer function as an argument.  The sanitizer function should manipulate the string to be valid input (if it was not already) and return the valid version of this input.  The <strong>restrict</strong> function is triggered whenever the input field is altered and will immediately replace the text in the field with sanitized text.</p>

<p>Most of the restricts I would want to use on an input field can be represented by a regular expression, so I created the <strong>regexRestrict</strong> function that takes a regular expression as input and uses <strong>restrict</strong> to replace matches to this regular expression found in the string.</p>

<p>The <strong>restrict</strong> and <strong>regexRestrict</strong> functions provide every feature that the AlphaNumeric plugin promises, but they don&rsquo;t use the same syntax as the AlphaNumeric plugin.  To be able to drop this plugin into a codebase that currently uses the AlphaNumeric plugin, we&rsquo;d need an equivalent to the <strong>alphanumeric</strong>, <strong>alpha</strong>, and <strong>numeric</strong> functions with all of their <a href="http://itgroup.com.ph/alphanumeric/">stated features</a>.  To allow the code that relied on the AlphaNumeric plugin to continue functioning, I created replacements for all three of these functions.  These functions take the same inputs as their AlphaNumeric plugin equivalents.</p>

<p>The restrict and regexRestrict functions and the alphanumeric plugin replacement that uses these functions can be <a href="http://github.com/treyhunner/jquery-formrestrict">found on github</a>.</p>
]]></content>
  </entry>
  
  <entry>
    <title type="html"><![CDATA[Multiple Monitors With Multiple Workspaces]]></title>
    <link href="http://treyhunner.com/2009/06/multiple-monitors-with-multiple-workspaces/"/>
    <updated>2009-06-15T00:00:00-07:00</updated>
    <id>http://treyhunner.com/2009/06/multiple-monitors-with-multiple-workspaces</id>
    <content type="html"><![CDATA[<p>In most window managers (WMs) that allow for multiple workspaces, additional monitors simply increase the size of each workspace. Since January I have been using a window manager that handles multiple monitors very differently, <a href="http://www.xmonad.org/">xmonad</a>.  Instead of increasing the workspace size to fit onto two monitors, each monitor displays a separate workspace, so the number of visible workspaces is increased.</p>

<p><em>Paradigm difference between these two WM styles:</em></p>

<ol>
<li>Each additional monitor extends the workspace size

<ul>
<li>One large workspace is visible at a time (ex: workspace 1 spans across all monitors)</li>
<li>When the workspace changes, both monitors change</li>
<li>When removing a monitor, workspaces must shrink in size, bunching windows together</li>
</ul>
</li>
<li>Each additional monitor allows another workspace to be visible

<ul>
<li>Each monitor displays one workspace at a time (ex: monitor 1 currently showing workspace 3 and monitor 2 currently showing workspace 1)</li>
<li>When the workspace on one monitor changes the workspace on the other monitor does not need to change</li>
<li>When removing a monitor, one less workspace will be displayed</li>
</ul>
</li>
</ol>


<p>There are many times when I want to be able to keep one monitor static while changing the other monitor.  For example I may want a video to stay on one monitor while I work on the other monitor.  In most window managers this restricts me to one workspace because changing workspaces would change both monitors.  In xmonad, the workspace that contains the video can be placed on one monitor and the visible workspace on the other monitor can changed freely without interfering with the first monitor.</p>

<p>Since using xmonad, I have found &ldquo;1 workspace per monitor&rdquo; window management much more productive and comfortable.  I wish more window managers would at least make this kind of workspace/monitor handling an option.  I have had problems with xmonad recently and I have been trying to switch back to a more popular window manager with free-floating windows like Gnome, KDE, Xfce, or Openbox.</p>

<p>So far my biggest problem with switching to other window managers has been the lack of &ldquo;1 workspace per monitor&rdquo; support.  Xmonad has greatly increased my productivity with two monitors and it&rsquo;s hard for me to switch away from it for this reason.  Hopefully I will find out how to effectively emulate this behavior in other window managers.</p>
]]></content>
  </entry>
  
  <entry>
    <title type="html"><![CDATA[Ubuntu Now Boots in 10 Seconds]]></title>
    <link href="http://treyhunner.com/2009/05/ubuntu-now-boots-in-10-seconds/"/>
    <updated>2009-05-01T00:00:00-07:00</updated>
    <id>http://treyhunner.com/2009/05/ubuntu-now-boots-in-10-seconds</id>
    <content type="html"><![CDATA[<p>I upgraded my Thinkpad to Ubuntu 9.04 (Jaunty Jackalope) recently.  My laptop has a solid-state drive and I had tweaked the boot process previously so it was down to 15 seconds.  I had heard that Jaunty had drastically decrease the boot time, but I figured my computer could not boot much faster than it already had since I had modified the boot process drastically.</p>

<p>I was wrong.  The first time I rebooted I was amazed at the speed of the boot.  My boot logger recorded the boot as taking 8 seconds.  I have rebooted once more since I installed Jaunty and that boot only took 7 seconds.</p>

<p>The main reason the boot time is so fast is due to my solid state drive.  That is how I got my boot down to 15 seconds before.  However, 7 seconds is a ridiculously fast boot time in my opinion.  Especially since it seems like it takes only about 3 seconds until my login manager is loaded and waiting for me.</p>
]]></content>
  </entry>
  
</feed>