/doc/decorators/transform.rst

https://code.google.com/p/ruffus/ · ReStructuredText · 166 lines · 109 code · 57 blank · 0 comment · 0 complexity · 5f9cf1e94456dbdaa30dd034030525dc MD5 · raw file

  1. .. include:: ../global.inc
  2. .. _decorators.transform:
  3. .. index::
  4. pair: @transform; Syntax
  5. See :ref:`Decorators <decorators>` for more decorators
  6. ########################
  7. @transform
  8. ########################
  9. .. |tasks_or_file_names| replace:: `tasks_or_file_names`
  10. .. _tasks_or_file_names: `decorators.transform.tasks_or_file_names`_
  11. .. |extra_parameters| replace:: `extra_parameters`
  12. .. _extra_parameters: `decorators.transform.extra_parameters`_
  13. .. |output_pattern| replace:: `output_pattern`
  14. .. _output_pattern: `decorators.transform.output_pattern`_
  15. .. |matching_regex| replace:: `matching_regex`
  16. .. _matching_regex: `decorators.transform.matching_regex`_
  17. .. |suffix_string| replace:: `suffix_string`
  18. .. _suffix_string: `decorators.transform.suffix_string`_
  19. *********************************************************************************************************************************************************************************************************************
  20. *@transform* ( |tasks_or_file_names|_, :ref:`suffix<decorators.suffix>`\ *(*\ |suffix_string|_\ *)*\ | :ref:`regex<decorators.regex`\ *(*\ |matching_regex|_\ *)*\ , |output_pattern|_, [|extra_parameters|_,...] )
  21. *********************************************************************************************************************************************************************************************************************
  22. **Purpose:**
  23. Applies the task function to transform data from input to output files.
  24. Output file names are determined from |tasks_or_file_names|_, i.e. from the output
  25. of specified tasks, or a list of file names.
  26. This can be either via matches to the end of the file name (suffix matches) or, more
  27. flexibly, using regular expression pattern substitutions.
  28. Only out of date tasks (comparing input and output files) will be run
  29. **Simple Example**
  30. Transforms ``*.c`` to ``*.o``::
  31. @transform(["1.c", "2.c"], suffix(".c"), ".o")
  32. def compile(infile, outfile):
  33. pass
  34. Same example with a regular expression::
  35. @transform(["1.c", "2.c"], regex(r".c$"), ".o")
  36. def compile(infile, outfile):
  37. pass
  38. Both result in the following function calls:
  39. ::
  40. # 1.c -> 1.o
  41. # 2.c -> 2.o
  42. compile("1.c", "1.o")
  43. compile("2.c", "2.o")
  44. **Escaping regular expression patterns**
  45. A string like ``universal.h`` in ``add_inputs`` will added *as is*.
  46. ``r"\1.h"``, however, performs suffix substitution, with the special form ``r"\1"`` matching everything up to the suffix.
  47. Remember to 'escape' ``r"\1"`` otherwise Ruffus will complain and throw an Exception to remind you.
  48. The most convenient way is to use a python "raw" string.
  49. **Parameters:**
  50. .. _decorators.transform.tasks_or_file_names:
  51. * *tasks_or_file_names*
  52. can be a:
  53. #. Task / list of tasks (as in the example above).
  54. File names are taken from the output of the specified task(s)
  55. #. (Nested) list of file name strings.
  56. File names containing ``*[]?`` will be expanded as a |glob|_.
  57. E.g.:``"a.*" => "a.1", "a.2"``
  58. .. _decorators.transform.suffix_string:
  59. * *suffix_string*
  60. must be wrapped in a :ref:`suffix<decorators.suffix>` indicator object.
  61. The end of each input file name which matches ``suffix_string`` will be replaced by ``output_pattern``.
  62. Input file names which do not match suffix_string will be ignored
  63. The non-suffix part of the match can be referred to using the ``"\1"`` pattern. This
  64. can be useful for putting the output in different directory, for example::
  65. @transform(["1.c", "2.c"], suffix(".c"), r"my_path/\1.o")
  66. def compile(infile, outfile):
  67. pass
  68. This results in the following function calls:
  69. ::
  70. # 1.c -> my_path/1.o
  71. # 2.c -> my_path/2.o
  72. compile("1.c", "my_path/1.o")
  73. compile("2.c", "my_path/2.o")
  74. For convenience and visual clarity, the ``"\1"`` can be omitted from the output parameter.
  75. However, the ``"\1"`` is mandatory for string substitutions in additional parameters, ::
  76. @transform(["1.c", "2.c"], suffix(".c"), [r"\1.o", ".o"], "Compiling \1", "verbatim")
  77. def compile(infile, outfile):
  78. pass
  79. Results in the following function calls:
  80. ::
  81. compile("1.c", ["1.o", "1.o"], "Compiling 1", "verbatim")
  82. compile("2.c", ["2.o", "2.o"], "Compiling 2", "verbatim")
  83. Since r"\1" is optional for the output parameter, ``"\1.o"`` and ``".o"`` are equivalent.
  84. However, strings in other parameters which do not contain r"\1" will be included verbatim, much
  85. like the string ``"verbatim"`` in the above example.
  86. .. _decorators.transform.matching_regex:
  87. * *matching_regex*
  88. is a python regular expression string, which must be wrapped in
  89. a :ref:`regex<decorators.regex>`\ indicator object
  90. See python `regular expression (re) <http://docs.python.org/library/re.html>`_
  91. documentation for details of regular expression syntax
  92. Each output file name is created using regular expression substitution with ``output_pattern``
  93. .. _decorators.transform.output_pattern:
  94. * *output_pattern*
  95. Specifies the resulting output file name(s).
  96. .. _decorators.transform.extra_parameters:
  97. * [*extra_parameters, ...*]
  98. Any extra parameters are passed to the task function.
  99. If ``regex(matching_regex)`` parameter is used, then regular expression substitution
  100. is first applied to (even nested) string parameters. Other data types are passed
  101. verbatim.
  102. For example::
  103. @transform(["a.c", "b.c"], regex(r"(.*).c"), r"\1.o", r"\1")
  104. def compile(infile, outfile):
  105. pass
  106. will result in the following function calls::
  107. compile("a.c", "a.o", "a")
  108. compile("b.c", "b.o", "b")
  109. See :ref:`here <decorators.transform_ex>` for more advanced uses of transform.