Silver Lining
=============

.. contents::

.. toctree::

    design
    terminology
    services
    appconfig
    deploymentconfig
    envvariables
    devpatterns
    management-scripts
    php
    processes
    providers
    examples
    django-quickstart
    disabling-sites
    docswanted
    todo
    comparisons
    dns
    filelayout
    devel-silverlining
    license
    contributors
    Repository <http://bitbucket.org/ianb/silverlining>
    Mailing List <http://groups.google.com/group/silverlining-dev>
    Continuous Integration Server <http://ci.cloudsilverlining.org>
    Presentation on Silver Lining <http://bit.ly/azvW7t>

.. comment: not working

    Pre-built zip files <http://cloudsilverlining.org/zip/>

This is a script and server layout to manage Python-based web
applications.  And PHP.

Author, license, etc
--------------------

This is written by `Ian Bicking <http://ianbicking.org>`_; you might
`join the list <http://groups.google.com/group/silverlining-dev>`_ or
`open a ticket <http://bitbucket.org/ianb/silverlining/issues/>`_, or
maybe head on over to FreeNode, ``#silverlining``.  There's a
`bitbucket repository <http://bitbucket.org/ianb/silverlining>`_.  It
was written with support by `The Open Planning Project
<http://openplans.org>`_.

It's under the `GPL <license.html>`_, though that **does not** imply
anything with respect to the licensing on applications that use this
for deployment.  All files in ``silverlining/server-files/support``
should be considered to be MIT-licensed (these files actually come
close to coexisting with deployed files, though even those don't come
very close).

.. comment:

   Maybe this is predicting pushback unnecessarily... so consider this
   a possible but not necessary bit of documentation.  But if you give
   me grief about the GPL, consider this the response...

   If you have a problem with the GPL for this application you need to
   get over yourself.  The GPL isn't viral to your application, and
   that's the only reasonable interpretation of the GPL.  I don't care
   about what stupid lawyers say about the GPL; if a lawyer is telling
   you or your employer scary stories about what the GPL does, your
   lawyer is a bad lawyer and I'm not going to work around your
   organizational bug.  If you really have a valid problem with *this*
   program being under the GPL, then I guess you want to make
   proprietary extensions to it, then eh, I'm not really interested.

Who Should Use Silver Lining
----------------------------

Maybe not you!  Silver Lining is not a supported product.  If you want
to use Silver Lining, then Silver Lining is not for you.  If you want
to help develop Silver Lining, then maybe Silver Lining is for you.
If you want to fix problems, provide patches, and collaborate on the
project, then great.  Maybe at a later date this will be a more
usable, reliable tool; but that time is not right now.

To repeat: Silver Lining is for collaborators, not users!

What Does It Do?
----------------

This is a script that:

1. Can create and destroy virtual servers, using a Cloud service API
   (something supported by `libcloud <http://libcloud.org>`_).
2. Sets up an Ubuntu server to a known configuration.
3. Deploys Python web applications to these servers.

(Also it can deploy PHP applications, maybe other languages in the
future.)

You shouldn't concern yourself too much with the server at this
point.  Instead, let us ask: what does the application look like?

The Application
---------------

Silver Lining is inspired by Google App Engine in this respect; an
application is a bunch of files that are uploaded to the server.  You
don't "build" an application on the server.  To upload or update an
application, you run::

    $ silver update path/to/app/ myapp.com

This uploads your files, makes sure requests to myapp.com go to your
new application.  Your application has a file ``app.ini`` (located, in
this example, in ``path/to/app/app.ini``) which describes the basics
of the application.

This also sets up any *services* that you need for your application
(which you have listed in ``app.ini``).  **Services** include any kind
of persistence your application requires.  This would be a database,
or even just a location where you can store files (which isn't
available unless you ask for it).  These services then *inject* their
configuration into your application though environmental variables.
So when you start your application you might look for
``os.environ['CONFIG_PG_DBNAME']`` to see what PostgreSQL database you
should connect to.  (What each service adds to the environment is
covered in the `services <services>`_ document.)

You might also ask: what about binary packages?  For instance, `lxml
<http://codespeak.net/lxml/>`_.  With Silver Lining only *portable*
files can go in your application directory; typically ``.py`` files,
templates, and other plain-text data files.  So if you want lxml you
have to have the Ubuntu package installed; you'd add (to ``app.ini``)
the configuration ``packages = lxml``.

A lot of this is covered in `application configuration <appconfig>`_.

And what kind of frameworks can you use with Silver Lining?  Anything
really.  The only real exception is Zope 2, which has strict Python
version requirements, lots of C extensions, and all sorts of other
complexities.  `Repoze <http://repoze.org/>`_ will work fine though,
and a `ZEO
<http://wiki.zope.org/ZODB/guide/node2.html#SECTION000230000000000000000>`_
service is easy to imagine (though not yet implemented).

The Server
----------

The server is setup thusly:

* Ubuntu Lucid 10.04 LTS

* Python 2.6.

* Apache with mod_wsgi.

* But you don't directly interact with either of these.

* Right now the persistence handling is minimal;
  this is where the most additional support needs to be added to the
  system (backups, migration, cloning, separating servers, and
  eventually stuff like replication). Currently there is basic setup for:

  * PostgreSQL with the PostGIS extensions.
  * CouchDB served locally
  * MongoDB served locally

* Nothing cron-like has been implemented yet, though probably it will
  be implemented very similar to App Engine (with internal requests).
  App setup and migration has to be hand-coded, but can be configured
  to be run.

* Each app lives at its own domain; per-directory applications are not
  currently implemented (but would be nice).  Multiple hostnames
  pointing to a single application is not implemented, but would be
  easy.  Wildcards require some more thought (simply in terms of
  Apache configuration).

Provider support
----------------

This package currently has only been tested with `Rackspace
<http://www.rackspace.com/cloud/?id=918>`_.  The only specific
Rackspace extension is the use of "files" on server creation (this is
part of the node-creation API that uploads files on the newly created
node/server).  The use of the files extension is minimal at this point
(just setting up ``/root/.ssh/authorized_keys``), and it is usable
without that.  If you are signing up for an account by clicking
through `this affiliate link
<http://www.rackspace.com/cloud/?id=918>`_ you can throw a few bucks
my way.

Silver Lining uses `libcloud`_, which is a library abstracting several
provider APIs.  It should be usable with other providers, but it's
only been tried with Rackspace, expect bumps with other services.

Language Support
----------------

There is some experimental support for `PHP <php>`_; it's not
incredibly graceful, but it has many of the same deployment advantages
as with Python.

Other languages would also be possible.  Ruby is a prime candidate of
course, probably using `Passenger <http://www.modrails.com/>`_ instead
of mod_wsgi.  If I had any clue how Perl was best deployed, it might
also seem reasonable.  But I (Ian) only have a personal interest in
the two supported platforms and *maybe* Java.

Installing
----------

.. comment: broken

 First, grab the newest zip file snapshot from `the zip/ directory </zip/>`_.

 Unpack the zip file, and run ``silver.py``.  You can't move this
 file out of the directory with the other library files, but you can
 alias it or something.

Window clients are not supported.  The only way Windows will be supported
is if someone volunteers to maintain the Windows support.  Individual
patches won't be accepted until that time -- it would only be a
disappointment to Windows users to get a half-working and unsupported client.

If you don't want to install from the zip file, you can install Silver
Lining into a virtualenv environment pretty easily too:

First get `virtualenv <http://virtualenv.openplans.org>`_, which also
gives you `pip <http://pip.openplans.org>`_.  Also be sure you have
Mercurial (``hg``) and Subversion (``svn``) installed.  Then run::

    $ virtualenv -p python2.6 silver
    $ silver/bin/pip install -r http://bitbucket.org/ianb/silverlining/raw/tip/requirements.txt

This installs a bunch of stuff, most importantly
``silver/bin/silver``.  Your applications will go in a separate
environment, so you should be sure that ``silver`` works regardless of
what virtualenv environment is activated.  I recommend::

    $ alias silver="/path/to/silver/bin/silver"

Cloud Server Management
-----------------------

There's several commands just to manage servers (also called nodes).
These are really just frontends to what libcloud provides.

Each "node" has a hostname, which is the name of the node, and also
the hostname setup in ``/etc/hosts``.

``silver list-images``:
    Lists all the images available (these are starter systems that can
    be cloned).

``silver list-sizes``:
    Lists all the sizes (plans) you can get for your server (a
    combination of RAM/disk and sometimes bandwidth).

``silver create-node``:
    Creates a new node (aka server).

``silver destroy-node``:
    Deletes a node. It is advisable to remove the node from
    ``~/.ssh/known_hosts`` after destroying it to avoid SSH warnings
    in future.

``silver setup-node``:
    This takes a bare Ubuntu Lucid system and sets it up.