menu

Machine Code Challenges Amazon’s Mechanical Turkers

Machine Code Challenges Amazon’s Mechanical Turkers
technology

A recent experiment asks: who's better - "the crowd," or the computer?

Chris Arkenberg
  • 22 december 2010

Researchers at the review site Yelp wanted to see if Amazon’s Mechanical Turk was indeed better at cataloging than a machine algorithm. They tasked 4660 Turkers to identify the proper category for a business based on a multiple choice test. Of the sample, only 79 were successful. The 79 successful Turkers were then split into groups of three and each group given a second task of categorizing businesses. The 79 Turkers scored around 62% accuracy. They then sent the same set of tasks to a supervised learning algorithm – a Naive Bayes classifier, to be precise. The results showed that the algorithm succeeded about 80% of the time, handily beating the Turkers by a wide margin. The results suggest one of two things (or possibly both): that the Mechanical Turk process needs to be refined to encourage better work, or that perhaps crowdsourced cataloging is steadily being threatened by the growing sophistication of machine algorithms. In either case, the future of work is likely to be influenced by more such challenges pitting humans against their machine counterparts.

Mechanical Turk

[via Technology Review]

Trending

Genetics Startup Is Working To Create A Completely Personalized, DNA-Based Range Of Products

Health
Travel Today

United Airlines Launches An Updated Business Class Program

The new Polaris product prioritizes customer service and updated modern amenities

Design & Architecture Today

Video Explores Complex Museum Architecture

A mini video gives a quick overview of the most beautiful cultural buildings built

Trending

Get PSFK's Related Report: Future of Automotive

See All
Sustainability Today

Smog Vacuum Turns Pollution Particulates Into Unique Jewelry

A large device was installed in Beijing that sucks up smog and compresses it into small centerpieces atop jewelry

Experiential Marketing Today

PSA Fashion Line Shines A Light On Victim Blaming

YWCA Canada is using fashion and sexist tweets to highlight how often we as a society blame the victim in cases of abuse

Related Expert

Brian W. Jones

Collabrative Art & Design

Syndicated Today

What PSX 2016 Tells Us About The Modern Games Industry

Nostalgia and big brands are defining how the console market is being shaped

Op-Ed Today

VP: Why Messaging Apps Are Issuing In A New Era Of Commerce

Matt Johnson discusses how mobile messaging commerce is creating a different modality for interaction between retailers and consumers

Technology Today

The PSFK Holiday Gift Guide 2016

Based on a year of research by PSFK Labs, we curated a list of innovative and unique holiday gifts

PSFK LABS REPORT

Future Of Retail 2017
Transformation Strategies For Customer-First Business
NEW

PSFK Op-Ed december 1, 2016

Communications Officer: What This Holiday Season Means For Millennial Shoppers

Dallas Lawrence of the Rubicon Project shares why holiday cheer is all about likes, views and retweets

PSFK Labs december 1, 2016

Retail Spotlight: Home Depot Reimagines How Employees Conduct Tasks

The home improvement retailer puts the customer first by initiating local fulfillment centers and simplifying freight-to-shelf inventory management

Food Today

Minute Maid Opens A Store That Sells Nothing

The beverage company opened a pop-up shop that encourages customers to write letters to their parents instead of buying a gift

Entertainment Today

Samsung And Viceland Partner For A Virtual Reality Documentary On The Syrian War

The White Helmets film uses VR to immerse viewers in the everyday conditions of the war-torn country

Design & Architecture Today

These Designs Bring Modern Architecture To The Humble Birdhouse

Artist Douglas Barnhard has imagined a series of designs emulating the work of architects such as Frank Lloyd Wright and Joseph Eichler

AI Today

Nine Technologies To Invest In When It Comes To Retail [Future Of Retail]

Retailers are leveraging assistive technology to help employees in their daily tasks and customers on the sales floor

Travel Today

Acura Cockpit Envisions The Future Of Autonomous Travel

The car brand has built a proof that offers a sneak peek of the self-driving vehicle experience

PSFK EVENT

FUTURE OF RETAIL 2017
Conference Built Around Our Report Launch
BUY TICKETS

Retail Today

How Artificial Intelligence Is Enabling Anytime Shopping & On-Demand Support

Retailers are leveraging advanced technology to relieve associates from mundane tasks, while better assisting customers

Cities Today

NYC Map Calculates Economic Benefits Of Every Tree

TreesCount! maps all the greenery in the five boroughs and even figures out how much money it saves the city

Design Today

Create A Perfectly Tailored Shirt Using Just Your Phone

UKYS has created an application to get accurate measurements of buyers through a few simple pictures for custom shirts

No search results found.