An experiment attempts to understand if its possible to produce quality work using crowdsourced labor that otherwise requires expertise.
A team of scientists and journalists are conducting an experiment called “My Boss is a Robot” that involves creating an automated system for producing journalism using unskilled workers from Amazon’s Mechanical Turk.
On a higher level, the experiment tries to explore whether it is possible to use untrained, crowdsourced labor to create something which requires skills and experience. The process aims to produce a science article of average length based on a new research paper. By breaking this process into small modules and having software overseeing all the tasks done by workers, the system hopes to be completely automated, with this human-powered machine churning out journalism pieces one after the other.
The project’s official site explains the process in detail:
A regular journalist would start by reading the paper. They might then call up the authors, and follow up that conversation by contacting other researchers who work on the same topic. The notes from those chats form the basis for the story.
That’s the process we’re trying to automate. To do so, we need to break it down into simple tasks, each of which is suitable for the workers on Mechanical Turk. One task might be: “use the references from the paper to identify researchers who could comment on the results”. Another: “read the abstract and identify the most interesting aspect of this paper”.
We also need software to manage these jobs. For example, we might want to ask five workers to read the abstract of the paper and say what they find interesting about it. We’re basically asking the workers what the story should be about. Thanks to an interface built by the CMU team, the workers’ answers will be fed back to the software that controls the process, aka the robot boss. The robot boss might then combine the five answers and ask workers to vote on which they find the most interesting. When it’s done, the workers, overseen by the software, will have selected the angle that the story will take.
The rest of the process — writing, editing, fact-checking — will work in a similar way. So if it does actually work, the system will be totally automated. Meaning that we will feed a scientific paper into this human-powered machine and, a few days later, out will pop a piece of journalism.
[via Technology Review]