/kai/templates/home/robots.mako

https://bitbucket.org/bbangert/kai/ · Mako · 15 lines · 15 code · 0 blank · 0 comment · 0 complexity · 3fa171c8ff5df03bcda771c6ff250296 MD5 · raw file 

    

  1. User-agent: *
    2. 

  2. Crawl-delay: 2
    3. 

  3. Request-rate: 2/3
    4. 

  4. Visit-time: 0800-1600
    5. 

  5. Disallow: /project/pylonshq*rev=*
    6. 

  6. Disallow: /project/pylonshq*format=*
    7. 

  7. Disallow: /project/pylonshq/query
    8. 

  8. Disallow: /project/pylonshq/search
    9. 

  9. Disallow: /project/pylonshq/wiki
    10. 

  10. Disallow: /project/pylonshq/newticket
    11. 

  11. Disallow: /project/pylonshq/changeset
    12. 

  12. Disallow: /project/pylonshq/log
    13. 

  13. Disallow: /project/pylonshq/milestone
    14. 

  14. Disallow: /project/pylonshq/timeline
    15. 

  15. Disallow: /project/pylonshq/roadmap
    16.