/doc/tutorials/manual/follows.rst

  1.. include:: ../../global.inc
  2.. include:: chapter_numbers.inc
  3
  4.. _manual.follows:
  5
  6#################################################################################################
  7|manual.follows.chapter_num| : `Arranging tasks into a pipeline with` **@follows**
  8#################################################################################################
  9
 10    * :ref:`Manual overview <manual>` 
 11    * :ref:`@follows syntax in detail <decorators.follows>`
 12    
 13    .. index:: 
 14        pair: @follows; Manual
 15
 16
 17***************************************
 18**@follows**
 19***************************************
 20
 21    The order in which stages or :term:`task`\ s of a pipeline are arranged are set
 22    explicitly by the :ref:`@follows(...) <decorators.follows>` python decorator:
 23    
 24        ::
 25    
 26            from ruffus import *
 27            import sys
 28            
 29            def first_task():
 30                print "First task"
 31        
 32            @follows(first_task)
 33            def second_task():
 34                print "Second task"
 35        
 36            @follows(second_task)
 37            def final_task():
 38                print "Final task"
 39
 40    
 41    the ``@follows`` decorator indicate that the ``first_task`` function precedes ``second_task`` in 
 42    the pipeline.
 43
 44
 45.. note::
 46
 47    We shall see in :ref:`Chapter 2 <manual.tasks_as_input>` that the order of pipeline :term:`task`\ s can also be inferred implicitly 
 48    for the following decorators
 49
 50        * :ref:`@split(...) <manual.split>`
 51        * :ref:`@transform(...) <manual.transform>`
 52        * :ref:`@merge(...) <manual.merge>`
 53        * :ref:`@collate(...) <manual.collate>`
 54
 55.. index:: 
 56    pair: pipeline_run; Manual
 57
 58=====================
 59Running
 60=====================
 61
 62    Now we can run the pipeline by:
 63        ::
 64            
 65            pipeline_run([final_task])
 66    
 67    
 68    Because ``final_task`` depends on ``second_task`` which depends on ``first_task`` , all 
 69    three functions will be executed in order.
 70    
 71.. index:: 
 72    pair: pipeline_printout_graph; Manual
 73    pair: pipeline_printout; Manual
 74
 75    
 76=====================
 77Displaying
 78=====================
 79
 80    We can see a flowchart of our fledgling pipeline by executing:
 81        ::
 82        
 83            pipeline_printout_graph ( "manual_follows1.png",
 84                                     "png",
 85                                     [final_task], 
 86                                     no_key_legend=True)
 87        
 88    producing the following flowchart
 89    
 90        .. image:: ../../images/manual_follows1.png
 91        
 92
 93    or in text format with:
 94        ::
 95        
 96            pipeline_printout(sys.stdout, [final_task])
 97        
 98    which produces the following:
 99        ::
100        
101            Task = first_task
102            Task = second_task
103            Task = final_task
104
105    
106
107.. index:: 
108    pair: @follows; referring to functions before they are defined
109    pair: @follows; out of order
110.. _manual.follows.out_of_order:
111
112***************************************
113Defining pipeline tasks out of order
114***************************************
115
116    All this assumes that all your pipelined tasks are defined in order.
117    (``first_task`` before ``second_task`` before ``final_task``)
118    
119    | This is usually the most sensible way to arrange your code.
120
121    If you wish to refer to tasks which are not yet defined, you can do so by quoting the function name as a string:
122
123        ::
124        
125            @follows("second_task")
126            def final_task():
127                print "Final task"
128
129    You can refer to tasks (functions) in other modules, in which case the full 
130    qualified name must be used:
131
132        ::
133        
134            @follows("other_module.second_task")
135            def final_task():
136                print "Final task"
137    
138.. index:: 
139    pair: @follows; multiple dependencies
140    
141.. _manual.follows.multiple_dependencies:
142
143***************************************
144Multiple dependencies
145***************************************
146            
147    Each task can depend on more than one antecedent task.
148    
149    This can be indicated either by stacking ``@follows``:
150        ::
151        
152            @follows(first_task)
153            @follows("second_task")
154            def final_task():
155                ""
156    
157    
158    or in a more concise way:
159        ::
160        
161            @follows(first_task, "second_task")
162            def final_task():
163                ""
164    
165.. _manual.follows.mkdir:
166
167.. index:: 
168    single: @follows; mkdir (Manual)
169    single: mkdir; @follows (Manual)
170
171
172******************************************************************************
173Making directories automatically with :ref:`mkdir <decorators.mkdir>`
174******************************************************************************
175
176    A common prerequisite for any computational task, is making sure that the destination
177    directories exist. 
178
179    **Ruffus** provides special syntax to support this, using the special 
180    :ref:`mkdir <decorators.mkdir>` dependency. For example:
181
182        ::
183    
184            @follows(first_task, mkdir("output/results/here"))
185            def second_task():
186                print "Second task"
187            
188    will make sure that ``output/results/here`` exists before `second_task` is run.
189    
190    In other words, it will make the ``output/results/here`` directory if it does not exist.
191
192
193