/docs/porting.txt
Plain Text | 371 lines | 263 code | 108 blank | 0 comment | 0 complexity | 86106f578681fd6ecc029a543b5ae818 MD5 | raw file
Possible License(s): GPL-2.0
1Porting An Application To Silver Lining 2======================================= 3 4This describes some of the steps of taking an existing application and 5"porting" it to Silver Lining. 6 7First steps for a simple Python application 8------------------------------------------- 9 10Imagine you've been developing an application for a while, using your 11Python framework of choice. It doesn't matter much *what* framework 12you used. You might have to do a little cleanup to start: 13 14* Make sure everything runs okay with Python 2.6. 15* Make sure you've identified all your requirements. 16 17It's not required to use a ``setup.py`` file or specify your 18requirements in ``setup.py`` with ``install_requires``. These files 19are good practice, but not necessary. 20 21Clean up your ``site-packages`` 22------------------------------- 23 24Some people have installed things into their computer as root, e.g., 25by using ``sudo easy_install`` or ``sudo pip install``. This is a bad 26idea. This will cause all sorts of weird problems and complications 27for you, not just with Silver Lining but with everything. 28 29I recommend deleting as much from ``/usr/lib/python2.6/site-packages`` 30as you can. I recommend *keeping* some select stable libraries like 31PIL, or Database drivers (psycopg, MySQLdb, etc). 32 33Create the file layout 34---------------------- 35 36Now you are ready to create a location for storing your application. 37With Silver Lining applications all the code and libraries are stored 38in a single directory. I usually name this directory ``mything-app`` 39(where ``mything`` is the name of my application). So run:: 40 41 $ silver init mything-app 42 $ cd mything-app 43 $ source bin/activate 44 45Now you have an activated `virtualenv 46<http://virtualenv.openplans.org>`_ with some basic Silver Lining 47layout, though no application. 48 49Next, put your application in place:: 50 51 $ cd src 52 $ hg/git/svn clone/checkout your-repository mything 53 54If you already have your code checked out into, say, ``~/src/mything`` 55then unfortunately you'll either have to move it or deal with two 56separate checkouts -- the code has to be physically located inside the 57``mything-app`` directory, because only that directory will be 58uploaded to the server. Though you can *probably* do something like 59this, but I haven't tried it:: 60 61 $ cd $VIRTUAL_ENV/src 62 $ ln -s ~/src/mything mything 63 64anyway, you probably want to do this work on a branch or separate 65cloned repository. 66 67Next create a new repository for your application's libraries. You 68won't install libraries during the regular workflow of developing 69applications with Silver Lining, even if you are collaborating with 70other developers or using multiple machines -- you'll only install 71things when you've actually added, updated, or removed a library that 72your application uses. To do that you'll put all your *installed* 73libraries in another repository. You can do that like:: 74 75 $ cd $VIRTUAL_ENV/lib 76 $ rm -r python 77 $ hg init python 78 79or use ``git init``, or create a repository and clone/check it out. 80This repository will be separate from your main application. 81 82Note that everything gets installed into ``lib/python/``, not into 83``lib/python2.6/site-packages/`` (which exists, but won't be added 84to). 85 86Installing Stuff 87---------------- 88 89Now you have some files in place, but nothing is really "installed". 90 91If you have a ``setup.py`` file, then run:: 92 93 $ cd $VIRTUAL_ENV 94 $ pip install -e src/mything/ 95 96This will make the package importable and install any requirements 97you've specified. 98 99If you *have not* written a ``setup.py`` file (common for Django 100applications) then you'll have to make sure the appropriate directory 101is on the path in a more manual fashion. Let's say you want 102``src/mything/`` directly on the path, then you'll do:: 103 104 $ cd $VIRTUAL_ENV/lib/python/ 105 $ echo ../../src/mything/ > mything.pth 106 107If you need to install some additional things, go ahead now:: 108 109 $ pip install SomeLibrary 110 111If you are developing a library concurrently with your application 112then you should also check that library out into ``src/`` and use 113``pip install -e``. 114 115You should use **pip** specifically and not easy_install, as pip 116cleans up after itself, uninstalling libraries and old versions of 117libraries. 118 119Non-Python Libraries 120-------------------- 121 122If you have any libraries that you need that aren't pure-Python, you 123should install them system-wide. Ideally you should use your 124operating system facilities to install these libraries (e.g., 125``apt-get install python-lxml``). 126 127Make note of these libraries. Database connection libraries will be 128installed automatically, but any other libraries will have to be 129explicitly added (more on that later). 130 131Describe your application with ``app.ini`` 132------------------------------------------ 133 134Now you have to tell Silver Lining how to start up your application, 135and possibly some other details. This is done through a configuration 136file ``app.ini`` in the root of your app. 137 138There will be an example file in place. You might just want to edit 139it, making note of these settings:: 140 141 [production] 142 app_name = mything 143 runner = src/mything/silver-runner.py 144 145Those are really the most important settings. Also if you are using a 146database (likely!) then find the appropriate line and uncomment it. 147In our example we'll imagine you are using PostgreSQL:: 148 149 service.postgresql = 150 151Note that you don't need any value there. (Maybe someday there will 152be a service that isn't just "enabled" but is actually configured, at 153which point it will take some value.) 154 155The most important thing you'll note is that now you have a "runner" 156defined in ``src/mything/silver-runner.py``. You better write that! 157 158The specific code you'll use depends on your web framework. This file 159will basically look like a mod_wsgi ``.wsgi`` file: it's some Python 160code that creates an object ``application``. 161 162You shouldn't need to set up your path in this file. In fact, you 163shouldn't need to set up any of your environment, you just find the 164application object. You can also use a `Paste Deploy 165<http://pythonpaste.org/deploy/>`_ ini file if you want (but even if 166you are using a framework such as Pylons, you can if you want import 167your ``make_app`` function directly and call it). 168 169Here's an example for Django:: 170 171 import django.core.handlers.wsgi 172 application = django.core.handlers.wsgi.WSGIHandler() 173 174Setting Up Your Environment 175--------------------------- 176 177One nice thing about Silver Lining is that your environment can 178*always* be setup. That means, everything importable all the time, 179everything configurable all the time, all data available. 180 181To do this there is a specific file for you to customize (well, a file 182for you to create): ``lib/python/silvercustomize.py`` (similar to 183``sitecustomize.py``). 184 185If you are using Django, you should do at least this:: 186 187 import os 188 os.environ['DJANGO_SETTINGS_MODULE'] = 'mything.settings' 189 190On other systems you might import or configure other things. It is 191preferable to put things here rather than in your runner, as this file 192is always loaded, so it will be available in the interactive 193interpreter, in tests, in scripts, etc. 194 195Configuration 196------------- 197 198Silver Lining uses a different model of configuration than is 199typically used. You can, if you want, ignore configuration entirely 200-- but this is about *porting* an application, so you certainly 201haven't ignored configuration before. So you need to adapt that 202configuration. 203 204Silver Lining tells the application about the environment through 205`environmental variables <envvariables.html>`_ (``os.environ``). 206These variables are *always* set. So if you use a PostgreSQL database 207you would look at these: 208 209``os.environ['CONFIG_PG_DBNAME']``: the name of the database you 210should connect to. 211 212``os.environ['CONFIG_PG_HOST']``: the host of the database (empty 213string means localhost). 214 215``os.environ['CONFIG_PG_PORT']``: the port of the database (empty 216string means default port). 217 218``os.environ['CONFIG_PG_USER']``: the user to connect as. 219 220``os.environ['CONFIG_PG_PASSWORD']``: the password to use (empty string 221means no password). 222 223``os.environ['CONFIG_PG_SQLALCHEMY']``: creating the SQLAlchemy 224connection string from all these pieces is somewhat annoying, so this 225does it for you. 226 227These variables will be set both during local development and when 228deployed, so you can always rely on them -- you don't need any special 229conditionals. 230 231``bin/`` 232-------- 233 234Yet to be written (it's kind of a pain currently). 235 236Setting Up Your Database 237------------------------ 238 239Now you have something that will *run*, but with a blank database. 240That's probably not what you want. 241 242You can tell Silver Lining to fetch a URL or call a script every time 243an application is updated. It is up to this script to do the right 244thing -- to alter tables, to notice tables that already exists, etc. 245All Silver Lining handles is fetching this program. 246 247To do this, add a setting to ``app.ini``:: 248 249 [production] 250 ... 251 update_fetch = /update-database 252 253That gets a URL, to call a script:: 254 255 update_fetch = script:bin/django-admin.py syncdb 256 257You can have multiple entries, one per line (indent subsequent lines). 258 259If you have to poke at a database remotely there's also a way to do 260that, with ``silver run``. 261 262These URLs or scripts will also be called everytime you start up the 263application locally using ``silver serve``. 264 265Static Files 266------------ 267 268If you have static files in your application, they go in ``static/``. 269They can (and should) be symlinks. Note that ``static`` doesn't 270appear in the URL; so if for example you have a convention that all 271your static files go in ``http://example.com/static/`` then you 272actually have to get the files into ``static/static/``. 273 274So do something like:: 275 276 $ cd $VIRTUAL_ENV/static 277 $ ln -s ../src/mything/mything/media media 278 279For Django you might also symlink things from the admin media into a 280location here, like:: 281 282 $ ln -s ../lib/python/django/contrib/admin/media admin-media 283 284and in ``settings.py``:: 285 286 ADMIN_MEDIA_PREFIX = '/admin-media/' 287 288Non-Python Libraries 289-------------------- 290 291Any non-Python libraries have to be installed in production as deb 292packages. For instance if you need lxml you'll need the 293``python-lxml`` package installed. You can get this to happen with 294this addition to app.ini:: 295 296 packages = 297 python-lxml 298 299You can have multiple packages, one on each line. This doesn't 300include databases though! (Well, SQLite is fine, as it runs 301in-process.) 302 303Testing It Out 304-------------- 305 306You can run this application locally now, using:: 307 308 $ silver serve ~/src/mything-app/ 309 310This will run any URLs or scripts in ``update_fetch``, and serve up 311the application at ``http://localhost:8080``. If you edit any files 312the server will restart to get the refreshed files. 313 314Note that when running locally you have to do things like create 315databases on your own. You can also `customize the local 316configuration <services.html#local-development-configuration>`_ (this 317is done *outside* the application). 318 319To test an actual deployment you need a server to deploy to. Silver 320Lining can deploy to any dedicated Ubuntu Karmic machine that has been 321configured by Silver Lining (using ``silver setup-node``), but the 322cloud makes getting a testing server very easy and quite cheap (when 323testing, you can allocate a server just for a few hours for testing 324before getting rid of it), so you might consider opening an account. 325 326 327Problems 328-------- 329 330Here are some problems you might encounter, with ideas for solutions: 331 332I can't see errors 333~~~~~~~~~~~~~~~~~~ 334 335Silver Lining doesn't do anything very useful with exceptions on its 336own. You should use whatever facilities your web framework has. 337Projects like `weberror <http://bitbucket.org/bbangert/weberror/>`_ 338and `flickzeug <http://dev.pocoo.org/projects/flickzeug/>`_ are a 339generic way of wrapping your application, and could be applied in the 340runner. 341 342To detect if you are in a production environment, you can use:: 343 344 from silversupport.env import is_production 345 346 application = normal instantiation 347 348 if not is_production(): 349 from flickzeug.debug import DebuggedApplication 350 application = DebuggedApplication(application, evalex=True) 351 352I need custom database setup 353~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 354 355If you want to, you can do things more manually than just using 356``update_fetch``. One way is to use ``silver run``. For instance, 357instead of using ``script:bin/django-admin.py syncdb`` you can invoke 358this manually on a server like:: 359 360 $ silver run mything-host.com django-admin.py syncdb 361 362This command can be interactive. There isn't good access directly to 363commands like ``psql`` though. 364 365I need a service that isn't supported 366~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ 367 368Then write the code! You can fork `the project 369<http://bitbucket.org/ianb/silverlining>`_ and add code in 370``silversupport/services/``. It's not very complicated, or at least 371not more complicated than using a database properly anyway.