PageRenderTime 122ms CodeModel.GetById 40ms app.highlight 1ms RepoModel.GetById 79ms app.codeStats 1ms

/docs/multiserver.txt

https://bitbucket.org/ianb/silverlining/
Plain Text | 138 lines | 112 code | 26 blank | 0 comment | 0 complexity | 32f726a029754fd0e2b22ee98fac1806 MD5 | raw file
  1Design Notes On Multiple Server Support
  2=======================================
  3
  4What should it look like to support multiple servers?
  5
  6Multiple servers will make up a "set".  In particular, a "nodeset".  A
  7configuration file will describe this::
  8
  9    [general]
 10    # Just naming the general setup:
 11    name = my-server-setup
 12    # This is used for filling in node names if they aren't explicitly given:
 13    base_domain = foobar.com
 14    # If you set this to false, then /etc/hosts won't be updated:
 15    set_etc_hosts = false
 16
 17    [provider]
 18    # You can just look up a provider in ~/.silverlining.conf:
 19    name = whatever
 20    # Or specify all the values:
 21    provider = rackspace
 22    username = foobar
 23    secret = XXX
 24    # or:
 25    secret_file = ${HOME}/.rackspace-secret.txt
 26
 27    [appserver]
 28    # This describes the app server
 29    # You can have multiple things setup:
 30    locations =
 31        http://foobar.com/ APP_NAME1
 32
 33    # You can setup multiple nodes:
 34    nodes = 10
 35    # default being 1, of course.
 36
 37    # You can indicate the size of these nodes too:
 38    size = id 1
 39    # You can also give the size in MB of ram (since that generally
 40    # increments):
 41    size = ram 256MB
 42
 43    # This is used to name the nodes:
 44    node_name = app{{n}}.foobar.com
 45
 46    # If not provided, this will be automatically determined by the
 47    # number of nodes (1=no balancing), and the existence of a
 48    # [balancer] section.
 49    load_balancer = true
 50
 51    [service:mysql]
 52    # How many nodes to assign:
 53    nodes = 1
 54    # Without special support, generally just 1 node is supported
 55    # also node_name/size
 56
 57    [balancer]
 58    # Describes the load balancer
 59    node_name = balancer.foobar.com
 60    hostnames = foobar.com
 61        baz.com
 62
 63This describes a set of nodes/servers, imagine this file is named
 64``foobar.conf``.  The commands for creating and setting up nodes
 65will take nodeset configurations, like::
 66
 67    silver create-node foobar.conf
 68    silver setup-node foobar.conf
 69
 70The command will specifically look for ``.conf`` in the name, then
 71inferring it is not a domain name.
 72
 73Then this configuration file will also be a target just like a
 74"location".  E.g., you can do::
 75
 76    silver update myapp/ foobar.conf
 77
 78This will match the name against the locations in the configuration
 79file to determine where exactly to upload the application to.
 80
 81Interpolation will be available anywhere in the configuration file,
 82using `this pattern
 83<http://blog.ianbicking.org/2009/04/20/treating-configuration-values-as-templates/>`_.
 84This way you can extend another file (using buildout-style directives;
 85these are already built into `INITools
 86<http://pythonpaste.org/initools/>`_) and abstract out aspects through
 87various variables (including environmental variables).
 88
 89When running ``create-node`` and ``setup-node`` all the servers needed
 90will be enumerated and named.  ``create-node`` will create any new
 91nodes necessary (and can be run to scale up -- not sure how to scale
 92down except manually deleting nodes, ``delete-node foobar.conf`` won't
 93really work).
 94
 95``setup-node`` will need to support multiple kinds of servers.  There
 96will first be a basic setup (install normal libraries, users,
 97authentication).  Then another layer of setup on top: one for an
 98appserver (what we are doing now), one for a service (not much more
 99than the basic setup), and one for the load balancer.  All servers
100will be updated with configuration pointing to all the other servers,
101including ``/etc/hosts`` entries for all the domain names.  This
102configuration will probably look roughly like::
103
104    appserver = app1.foobar.com 10.4.3.2
105    appserver = app2.foobar.com 10.4.3.3
106    appserver = app3.foobar.com 10.4.3.4
107    service.mysql = mysql.foobar.com 10.4.3.5
108    balancer = balancer.foobar.com 10.4.3.6
109
110The balancer will use this to find the appservers, and the appservers
111will use this to configure their services.  I guess ssh keys will be
112setup for automated ssh'ing between these machines?  Not sure if it is
113necessary.
114
115The update will be a bit more sophisticated.  Imagine you are updating
116an application with this ``app.ini``::
117
118    [production]
119    app_name = blog
120    service.mysql =
121    update_fetch = /setup-stuff
122
123On update first ``silver`` will connect to ``mysql.foobar.com`` and
124setup the service for this application.  This mostly just creates the
125database if necessary.  Then it will half-deploy this application to
126all app servers.  The application will be deployed but not active.
127Then it will activate the first application and do ``update_fetch``,
128then activate the rest of the applications.  (Maybe running
129``update_fetch`` everywhere, but with a special flag to note it's not
130the primary update.)
131
132The service ``app_setup()`` code will run on each app server, but it
133will use the configuration to actually fill in some of those values,
134now pointing to another server instead of a local server.  Also the
135``install()`` function will be run on both the appservers and the
136service servers (with a flag to indicate the different environments).
137Client libraries still get loaded in the appserver, servers in the
138service servers.