PageRenderTime 11ms CodeModel.GetById 3ms app.highlight 2ms RepoModel.GetById 3ms app.codeStats 0ms


Relevant Search: With Applications for Solr and Elasticsearch

For more in depth reading about search, ranking and generally everything you could ever want to know about how lucene, elasticsearch or solr work under the hood I highly suggest this book. Easily one of the most interesting technical books I have read in a long time. If you are tasked with solving search relevance problems even if not in Solr or Elasticsearch it should be your first reference. Amazon Affiliate Link
Plain Text | 246 lines | 194 code | 52 blank | 0 comment | 0 complexity | 0bd50305fa91ab24578671bd5e882b75 MD5 | raw file
  1Comparisons To Other Cloud Tools
  4There are a bunch of tools or hosted applications for managing cloud
  5applications.  As I come upon these, I'm trying to add at least a
  6casual comparison...
  8.. contents::
 10Differences From App Engine
 13App Engine is in many ways the basis for Silver Lining; at least, all
 14the choices I made when starting Silver Lining were in reaction to App
 15Engine: either deliberately copying App Engine, or deliberately *not*
 16copying it.
 18So, some specific differences:
 20* This doesn't do security isolation between apps on a server.
 21  Individual servers aren't that expensive, so untrusted applications
 22  just shouldn't comingle with Silver Lining.
 24* This doesn't have a scalable backend.  It would be possible to set
 25  up a scalable backend service, and do the associated configuration,
 26  but I don't know much about those tools anyway.  Scalability isn't
 27  the first goal; ease and reliability of deployment is more the goal.
 28  Not that this should *exclude* scalability, but it's not automatic.
 29  With work and when using scalable backends, this might become easy
 30  with Silver Lining in the future.  But it's going to be a lot of
 31  steps, and in total a great deal of work.
 33* You can write files.
 35* App Engine allocates server time dynamically, and this does not in
 36  any way, though at least it makes your deployments scriptable so you
 37  might be able to deploy new servers in an automated fashion.
 39* There's no authentication built in, and there won't be; this can be
 40  handled by per-application libraries.
 42* There's no limits (like, for example, the 1000-now-2000 file limit).
 43  As a result support for skipping files on upload is not such a big
 44  deal.  Cloud disk space is fairly cheap, so optimizing tihs isn't a
 45  big deal.  
 47* Dynamic support for multiple runners or static file locations isn't
 48  supported (like in ``app.yaml``), because it doesn't seem very
 49  important to me.  If you don't want process isolation, something
 50  like `Paste's urlmap <>`_
 51  can handle dispatch (implemented in your application's runner).  At
 52  some point it should be possible to upload entirely separate
 53  applications into different paths, but that will be more isolated
 54  than what App Engine provides through ``app.yaml``.
 56* Of course, you can run any library or service.  For me `lxml
 57  <>`_ and `PostGIS
 58  <>`_ are important, and the lack of
 59  those libraries really kept and keeps me from using App Engine.
 61* No pure-Python libraries are included, because they aren't needed
 62  (including, for example, Django and WebOb).  Silver Lining keeps the
 63  runtime quite clean (except for libraries installed system-wide,
 64  which would be the case for something like lxml).
 66Differences From Heroku
 69`Heroku <>`_ is an App Engine-like service for Ruby
 70applications.  But it works differently from App Engine, and has some
 71novel ideas:
 73* Heroku uses git to deploy.  I don't see any particular reason to
 74  integrate with a version control system.  This system uses rsync.
 76* Configuration seems to be done somewhat magically with Heroku, and
 77  just with environmental variables in this system.  Magic is
 78  annoying.  You need to glue together the environmental variables
 79  with your configuration manually, though you only need to do it
 80  *once*.  (If you aren't using a framework that includes
 81  "configuration" then you can probably just hardcode the
 82  environmental variables in your code.)
 84* Stuff like migration is built-in.  I want to do that stuff too.
 86* Apps seem to be more chroot'd than in this system.  I'm not that
 87  concerned about this, as I expect people to just do that isolation
 88  by using multiple servers.
 90* There's a bunch of nice add-ons for Heroku.  These will map to
 91  Silver Lining services.  While not yet implemented, in the future
 92  enabling Solr will be a matter of writing ``service.solr =`` in your
 93  ``app.ini``.
 95* Heroku uses Ruby.  I believe Ruby is very supportable by this
 96  system, but it isn't supported now, and I'm unlikely to add it
 97  myself (mostly because I don't know quite how it should work).
 99* Heroku has more support for dynamically scaling up your application.
100  You can give Silver Lining apps a bigger server (up to the limits of
101  your service provider), and if you deploy an application on multiple
102  servers/nodes you can be reasonably assured they are identical, but
103  scaling of the persistence is not handled, and so adding new nodes
104  isn't really handled.
106Generally Heroku seems like a faster-moving target than App Engine,
107with lots of interesting ideas; I hope to get more ideas from it going
110Of course, the primary difference of Silver Lining is: it's open
111source, you can adapt the system as you find necessary, and it's a
112**lot** cheaper (since you are paying only for the actual cloud
113service, which is reasonably priced).
115Differences From Media Temple
120looks similar, but I haven't looked at it yet.
122Differences from App Scale
125`AppScale <>`_ is an open source
126implementation of the Google App Engine API.  It features a bunch of
127database backends, and can do a lot of the management of scaling
130While Silver Lining takes ideas from App Engine, the priorities are
131really quite different.  Silver Lining is focused on simple, reliable,
132repeatable deployment of applications.  The deployments really aren't
133that different from how software might traditionally be deployed, it
134just formalizes those pieces and factors the problem in a way that
135makes the actual deployment as simple and reliable as possible.
137AppScale (at least from what I can tell) is more focused on scaling,
138and App Engine-like infrastructure.  It provides the App Engine API;
139Silver Lining was written because I liked *parts* of the App Engine
140toolset, but I couldn't embrace the constraints of the platform.
142Silver Lining also I believe scales *down* further than AppScale.  You
143can run as many sites on one server/node as you care to.  So it is
144quite suitable for deploying several small applications cheaply.  I
145believe Silver Lining will make the life of the average web developer
146easier and simpler very quickly.
148Silver Lining is meant to be usable with existing applications.  You
149can use pretty much any database (some support code needs to be
150written to manage each kind of database, but it's quite simple).  You
151can write files.  Traditional "configuration" is actually kind of
152pushed *into* the application, but it just means you should do things
153like (in Django)::
155    DATABASE_ENGINE = 'postgresql_psycopg2'
156    DATABASE_NAME = os.environ['CONFIG_PG_DBNAME']
157    DATABASE_USER = os.environ['CONFIG_PG_DBUSER']
159    DATABASE_HOST = os.environ['CONFIG_PG_DBHOST']
160    DATABASE_PORT = os.environ['CONFIG_PG_DBPORT']
162Everything you can do in a traditional deployment system, you can do
163with Silver Lining (at least, potentially; extra support code may be
164necessary).  I don't think AppScale provides this nor do I think this
165is a goal of AppScale.
167Differences from Typhoonae
170`Typhoonae <>`_, like AppScale, is
171an open source adaptation of the App Engine API.
173Typhoonae uses `supervisord <>`_ for monitoring
174the processes, while Silver Lining uses mod_wsgi.  They both have a
175similar function, but because mod_wsgi is integrated with the web
176server it is not just aware of the process dying, but also of the
177process becoming unresponsive.  Supervisor, because it is a
178general-purpose process monitor, cannot determine when a process is
179serving its function.
181People have asked about how Silver Lining can be used with something
182more closely related to App Engine.  I'm not that optimistic about
183AppScale, because I think it overlaps in the wrong ways with Silver
184Lining -- they are more like direct competitors than complementary
185tools.  Typhoonae on the other hand might work better.
187The pieces of Typhoonae that would be useful would be its
188implementations of the App Engine APIs, backed with MongoDB (data
189store), RabbitMQ (well... not quite sure what this is for), and
190ejabberd (for chat integration), and possibly other APIs that they've
191implemented with a better backend than what the App Engine SDK itself
192contains.  These would each be services.  Possibly something like the
193Data Store API would just be a library you install, plus
194``service.mongodb``, but they could be presented as first-class
195services (e.g. ``service.typhoonae_datastore``).  I'd probably prefer
196they not be first-class services, because that means Typhoonae
197releases would have to be tied one-to-one to Silver Lining releases.
198I'd rather see something like a ``silverlining_typhoonae``
199distribution that you could install into your app.
201Differences from Moonshine
204So far, `Moonshine <>`_ sounds
205like the closest thing to Silver Lining in functionality, with even
206many of the same choices, except Ruby instead of Python.  It uses
207Ubuntu, Apache, and Passenger (where Passenger is the closest thing to
208a Ruby equivalent of mod_wsgi).  It also uses Ruby Enterprise Edition
209(?) and MySQL, where Silver Lining does not have a single persistence
212``config/moonshine.yml`` is a lot like ``app.ini``, ``cap
213deploy:setup`` is like ``silver setup-node`` and ``cap deploy`` is
214like ``silver update``.
216It's built on Capistrano, not using its own native constructs.  In
217this sense it is glue, but it's kind of a matter of perspective.  If
218Capistrano worked a little differently then it might look like
219Moonshine was the application and Capistrano the library.
221It has a notion of "shared config", which is configuration not in your
222source tree, but which is needed for deployment.  Silver Lining needs
223something like this.
225Differences from Rubber
228`Rubber <>`_ is a tool based on
229Ruby/Rails and Capistrano.
231I'll have to look into it some.  It, like Moonshine, is based on
232Capistrano.  It is a more general configuration management tool.  It
233also has more features for scaling.
238`Bitnamo <>`_ is kind of a package for pre-built
239open source applications.  For instance, you grab a Joomla image and
240you get Joomla.  From what I can tell, Bitnami itself is actually
241proprietary.  The supported applications seem to be PHP.
243Bitnami also creates virtual images you can use to develop.  It has
244inspired me to at least add a `todo item