2011-16-050.html | searchcode

/projecte eclipse/TI/data/2011-documentos/16/2011-16-050.html

https://gitlab.com/bernagg/TI
HTML | 268 lines | 211 code | 41 blank | 16 comment | 0 complexity | 70f7113f27c2a2bb7406c71f2a55c582 MD5 | raw file

<!DOCTYPE HTML>

<html>

<head profile="http://gmpg.org/xfn/11">



  <title>Crowdsourcing Code | Boris Smus</title>



  <meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />

  <meta name="verify-v1" content="XA13dvk8en7VKjWDrr3p+DjdF8OXxaPCqJfH7MWQtfc=" />

<meta name="google-site-verification" content="OatoM34IExkzYBHBEo0-VQDVE7FuJ-VBHA7xOXxJWTI" />



  <link rel="stylesheet" href="http://www.borismus.com/wp-content/themes/borismus/style.css" 

    type="text/css" media="screen, print" />

  <link rel="stylesheet" type="text/css" media="print" 

    href="http://www.borismus.com/wp-content/themes/borismus/print.css" />

  <link rel="alternate" type="application/rss+xml" title="Boris Smus RSS Feed" href="http://borismus.com/feed/" />

  <link rel="pingback" href="http://borismus.com/xmlrpc.php" />

  

    				
	<script type="text/javascript">//<![CDATA[
	// Google Analytics for WordPress by Yoast v4.06 | http://yoast.com/wordpress/google-analytics/
	var _gaq = _gaq || [];
	_gaq.push(['_setAccount','UA-17930798-1']);
	_gaq.push(['_trackPageview']);
	(function() {
		var ga = document.createElement('script'); ga.type = 'text/javascript'; ga.async = true;
		ga.src = ('https:' == document.location.protocol ? 'https://ssl' : 'http://www') + '.google-analytics.com/ga.js';
		var s = document.getElementsByTagName('script')[0]; s.parentNode.insertBefore(ga, s);
	})();
	// End of Google Analytics for WordPress by Yoast v4.0
	//]]></script>
<link rel="alternate" type="application/rss+xml" title="Boris Smus &raquo; Crowdsourcing Code Comments Feed" href="http://borismus.com/crowdsourcing-code/feed/" />
<script type='text/javascript' src='http://www.borismus.com/wp-includes/js/l10n.js?ver=20101110'></script>
<script type='text/javascript' src='http://www.borismus.com/wp-includes/js/jquery/jquery.js?ver=1.4.4'></script>
<link rel="EditURI" type="application/rsd+xml" title="RSD" href="http://borismus.com/xmlrpc.php?rsd" />
<link rel="wlwmanifest" type="application/wlwmanifest+xml" href="http://www.borismus.com/wp-includes/wlwmanifest.xml" /> 
<link rel='index' title='Boris Smus' href='http://borismus.com/' />
<link rel='start' title='Top 5 Sessions of Web 2.0 Expo NY' href='http://borismus.com/top-5-sessions-of-web-2-expo-ny/' />
<link rel='prev' title='Crowdsourcing Articles with Mechanical Turk' href='http://borismus.com/crowdsourcing-articles-with-mechanical-turk/' />
<link rel='next' title='Visualizing MTurk Requesters' href='http://borismus.com/visualizing-mturk-requester/' />
<meta name="generator" content="WordPress 3.1" />
<link rel='shortlink' href='http://borismus.com/?p=568' />

<!-- All in One SEO Pack 1.6.13 by Michael Torbert of Semper Fi Web Design[70,114] -->
<meta name="description" content="As a follow up to my last post, I posted a HIT on Mechanical Turk asking 20 turkers if they know Java. I paid them 5 cents to answer the question. Surprisingly," />
<link rel="canonical" href="http://borismus.com/crowdsourcing-code/" />
<!-- /all in one seo pack -->
<meta name="tweetmeme-title" content="Crowdsourcing Code" /><meta id="syntaxhighlighteranchor" name="syntaxhighlighter-version" content="3.1.1" />
  <script type="text/javascript"

   	src="http://www.borismus.com/wp-content/themes/borismus/js/enhance.js"></script>

</head>

<body>

<div class="wrapper">

  <div class="header">

    <ul class="links">

      <li class="">

        <a href="/">Blog</a>

      </li>

      <li class="page_item page-item-2"><a href="http://borismus.com/about/" title="About">About</a></li>
<li class="page_item page-item-15"><a href="http://borismus.com/projects/" title="Projects">Projects</a></li>
    </ul>

    <div class="info">

      <h1>

        <a href="/">Boris Smus</a>

      </h1>

      <h2>

        <a href="/">Software Designer</a>

      </h2>

    </div>

  </div>

<div class="main">

  <!-- iterate through all posts -->

    <div class="post">

    

    <!-- only show title and comments if it's a blog post -->

        <h3 class="title">

      <a href="http://borismus.com/crowdsourcing-code/">Crowdsourcing Code</a> 

    </h3>

    

    <div class="post_meta">

      <a href="http://borismus.com/crowdsourcing-code/#comments" class="count" title="click to add a new comment">

        2 responses      </a>

    </div>

        

    <!-- only show date if it's a blog post -->

        <div class="post_date" title="Jan 16, 2010">

      <div class="month">Jan</div>

      <div class="day">16</div>

      <!-- only display the post year if it's not this year's -->

              <div class="year">2010</div>

          </div>

        

    <div class="body">

      <p>As a follow up to my <a href="http://borismus.com/crowdsourcing-articles-with-mechanical-turk/">last post</a>, I posted a HIT on Mechanical Turk asking 20 turkers if they know Java. I paid them 5 cents to answer the question. Surprisingly, 9 of 20 claimed to know. I was amazed at how strong selection bias was in this case, since surely not 50% of turkers know how to program! </p>
<p>I then asked those turkers who know Java to complete the following trivial Java method. If they wrote it correctly, I paid them a 45 cent bonus.<span id="more-568"></span><br />
<code><br />
public static String reverse(String source) {<br />
&nbsp;&nbsp;&nbsp;&nbsp;// your code here<br />
}<br />
</code></p>
<p>Here are the results:</p>
<ul>
<li>4 turkers used <code>StringBuffer.reverse</code></li>
<li>3 turkers created a new string by iterating through the original string in reverse</li>
<li>1 used recursion</li>
<li>1 used <code>Collections.sort(l)</code>. I&#8217;m not sure what was intended</li>
</ul>
<p>I was hoping that people would fill in the empty reverse method with their code, but many of them implemented their own methods and helpers. One person implemented their own class with extensive comments. This data as a nice existence proof, indicating that turkers can be harnessed for programming-related crowdsourcing. </p>
<p>I&#8217;d like to turn Mechanical Turkers into Mechanical Coders. Given a set of unit tests and a method to implement, their work could be automatically verified based on passing the unit tests. Furthermore, some turkers could be tasked to write additional unit tests for this method. Through this technique, it&#8217;s conceivable to harness the power of The Turk to implement whole classes. Code quality aside, what sort of software quality could be achieved by following this approach?
<div class="twitter_button" style="float: right; margin-left: 10px;"><iframe src="http://platform.twitter.com/widgets/tweet_button.html?url=http%3A%2F%2Fborismus.com%2Fcrowdsourcing-code%2F&amp;count=vertical&amp;related=borismus:&amp;via=borismus&amp;lang=en" height="62" width="55" frameborder="0" scrolling="no" allowtransparency="true"></iframe></div>
      

      Tagged with: <a href="http://borismus.com/tag/mechanical-turk/" rel="tag">mechanical turk</a>, <a href="http://borismus.com/tag/research/" rel="tag">research</a>      <div style="clear: both"></div>

    </div>

  </div>

  

  

  <!-- Necessary for password-protected posts -->


<!-- COMMENTS -->
<div class="comments" id="comments">
  
  <h2 class="comment_number">
    2 comments
    <span style="float: right">
    <a href="#comment_form">leave comment</a>
    </span>
  </h2>
  
      
  <div class="comment" id="comment-277">
    
    <div class="comment_avatar">
      <img alt='' src='http://1.gravatar.com/avatar/b3d67baaf0725554335bf127cf53e49b?s=36&amp;d=identicon&amp;r=G' class='avatar avatar-36 photo' height='36' width='36' />    </div>
    
    <div class="comment_body">
      <span class="comment_author">
        <a href='http://www.antipode.ca/' rel='external nofollow' class='url'>Allen Pike</a>      </span>
      <span class="comment_main">
        <p>The trick would be writing unit tests that were thorough enough in less time than it would take you to code it yourself. With that, you could add in Turks for QA and hey, maybe you could even get Turks to write the unit tests. I&#8217;m sure you could produce non-UI apps that function in basic tasks using this approach, but I doubt it could profitably extend to interesting or profitable types of apps.</p>
      </span>
      <div class="comment_date">
        on January 16th, 2010 at 6:28 pm      </div>
    </div>
    
  </div>
      
  <div class="comment" id="comment-278">
    
    <div class="comment_avatar">
      <img alt='' src='http://1.gravatar.com/avatar/765d68e3619b045b528129bfbddf6150?s=36&amp;d=identicon&amp;r=G' class='avatar avatar-36 photo' height='36' width='36' />    </div>
    
    <div class="comment_body">
      <span class="comment_author">
        <a href='http://www.borismus.com' rel='external nofollow' class='url'>Boris Smus</a>      </span>
      <span class="comment_main">
        <p>It&#8217;s pretty darn difficult to write any sort of reasonable unit tests for user interfaces, so I definitely agree with you there. </p>
<p>But the interestingness of an application is determined by the designer, not a code monkeying turker, so I&#8217;m not sure what you mean.</p>
<p>As far as profitability goes, perhaps this approach would allow a small number of software architects to roll out working (and unit tested) software very quickly, since turkers are easily parallelized.</p>
      </span>
      <div class="comment_date">
        on January 17th, 2010 at 9:49 am      </div>
    </div>
    
  </div>
    

</div>

<form action="http://borismus.com/wp-comments-post.php" method="post" id="comment_form">

  <!-- leave a comment -->
  <h2>leave comment</h2>

  <textarea name="comment" id="comment" cols="80" tabindex="4" placeholder="Your comment please"></textarea>

  <input type="text" name="author" id="author" value="" size="40" tabindex="1" placeholder="Name (required)" />

  <input type="text" name="email" id="email" value="" size="40" tabindex="2" placeholder="Email (required)" />

  <p><input type="text" name="url" id="url" value="" size="40" tabindex="3" placeholder="Website" /></p>

  <p><input name="submit" type="submit" id="submit" tabindex="5" value="Submit" />
  <input type="hidden" name="comment_post_ID" value="568" />
  </p>

  <p style="display: none;"><input type="hidden" id="akismet_comment_nonce" name="akismet_comment_nonce" value="b07be63b8c" /></p>
</form>

<!-- LINKBACKS -->
<div class="comments" id="pings">
</div>

<script>
/* Copyright (c) 2009 Jon Rohan (http://dinnermint.org)
 * Dual licensed under the MIT (http://www.opensource.org/licenses/mit-license.php) 
 * and GPL (http://www.opensource.org/licenses/gpl-license.php) licenses.
 *
 * Version: 1.0.0
 * Written with jQuery 1.3.2
 */
(function($){$.fn.ghostText = function() {
  return this.each(function(){
    var text = $(this).attr("placeholder");
    if(text!=""&&($(this).val()==""||$(this).val()==text)) {
      $(this).addClass("disabled");
      $(this).val(text);
      $(this).focus(function(){
        $(this).removeClass("disabled");
        if($(this).val()==text) {
          $(this).val("");
        }
      });
      $(this).blur(function(){
        if($(this).val()=="") {
          $(this).val(text);
          $(this).addClass("disabled");
        }
      });
    }
  });
};})(jQuery);

$(function() {
  // enable ghost text on the inputs
  $('input[type="text"], textarea').ghostText();
});
</script>  

      

  <div class="navigation">

    <div class="older">

          </div>

    <div class="newer">

          </div>

  </div>

</div>



</div>

<div class="footer">

  <div class="content">

    <ul class="bar1">

      <li>			<div class="textwidget"><a style="border-bottom: none;" href="http://www.twitter.com/borismus"><img src="http://twitter-badges.s3.amazonaws.com/follow_me-a.png" alt="Follow borismus on Twitter"/></a>
<br/>
And <a href="">subscribe</a> to this blog.
<br/>
<br/></div>
		</li><li><form method="get" id="searchform" action="http://borismus.com/">
<div><input type="text" value="" name="s" id="s" />
<input type="submit" id="searchsubmit" value="Search" />
</div>
</form>
</li>    </ul>

    <ul class="bar2">

      <li>			<div class="textwidget">Disclaimer: Content found on this site reflects my personal thoughts and opinions <strong>only</strong>, and is to no extent endorsed by Google or any other party.</div>
		</li><li><h3> </h3><a href='http://borismus.com/tag/web/' class='tag-link-7' title='19 topics' style='font-size: 16px;'>web</a>, <a href='http://borismus.com/tag/dev/' class='tag-link-20' title='11 topics' style='font-size: 16px;'>development</a>, <a href='http://borismus.com/tag/javascript/' class='tag-link-23' title='9 topics' style='font-size: 16px;'>javascript</a>, <a href='http://borismus.com/tag/music/' class='tag-link-19' title='9 topics' style='font-size: 16px;'>music</a>, <a href='http://borismus.com/tag/python/' class='tag-link-16' title='9 topics' style='font-size: 16px;'>python</a>, <a href='http://borismus.com/tag/research/' class='tag-link-13' title='9 topics' style='font-size: 16px;'>research</a>, <a href='http://borismus.com/tag/design/' class='tag-link-11' title='8 topics' style='font-size: 16px;'>design</a>, <a href='http://borismus.com/tag/google/' class='tag-link-37' title='7 topics' style='font-size: 16px;'>google</a>, <a href='http://borismus.com/tag/android/' class='tag-link-102' title='5 topics' style='font-size: 16px;'>android</a>, <a href='http://borismus.com/tag/mobile/' class='tag-link-59' title='5 topics' style='font-size: 16px;'>mobile</a>, <a href='http://borismus.com/tag/mechanical-turk/' class='tag-link-71' title='4 topics' style='font-size: 16px;'>mechanical turk</a>, <a href='http://borismus.com/tag/lego/' class='tag-link-14' title='4 topics' style='font-size: 16px;'>lego</a>, <a href='http://borismus.com/tag/webkit/' class='tag-link-35' title='3 topics' style='font-size: 16px;'>webkit</a>, <a href='http://borismus.com/tag/mindstorms/' class='tag-link-15' title='3 topics' style='font-size: 16px;'>mindstorms</a></li>      <a href="http://borismus.com/feed/" class="rss" title="Click to subscribe to my RSS feed."></a>

      <a href="http://www.twitter.com/borismus" class="twitter" title="Click to follow me on twitter."></a>

    </ul>

  </div>  

  <div class="end"/>

</div>



<!-- Not using javascript anymore

<script type="text/javascript"

   src="http://www.borismus.com/wp-content/themes/borismus/js/enhance.js"></script>

-->

<script src="http://www.borismus.com/wp-content/plugins/tweetmeme/button.js" type="text/javascript"></script></body>

</html>
<!-- Dynamic page generated in 0.310 seconds. -->
<!-- Cached page generated by WP-Super-Cache on 2011-04-03 09:41:36 -->