Pushing the Limits of the Netflix Prize

Since October 2006, Netflix has held a competition to improve their movie recommendation system.  The Netflix Prize is $1 million for anyone who can beat their existing system with a 10% accuracy improvement.  So far, around 30,000 teams have registered for the challenge and the top ten teams boast improvements around 9% over the incumbent recommendation engine, Cinematch.  Each team of competitors is hard at work generating an algorithm that can predict human behavior through movie predictions.  However, many believe they’ve reached the limits of computer prediction. Applying mathematical formulas to the data set provided by Netflix has yielded impressive improvements and the New York Times Magazine article does a great job in breaking down the equations:

Singular value decomposition works by uncovering “factors” that Netflix customers like or don’t like. Say, for example, that “Sleepless in Seattle” has been rated by 200,000 Netflix users. In one sense, this is just a huge list of numbers — user No. 452 gave it two stars; No. 985 gave it five stars; and so on. But you could also think of those ratings as individual reactions to various aspects of the movie. “Sleepless in Seattle” is a “chick flick,” a comedy, a star vehicle for Tom Hanks; each customer is reacting to how much — or how little — he or she likes “chick flicks,” comedies and Tom Hanks. Singular value decomposition takes the mass of Netflix data — 17,770 movies, ratings by 480,189 users — and automatically sorts the films. The programmers do not actively tell the computer what to look for; they just run the algorithm until it groups together movies that share qualities with predictive value.

Initially the problem was simple coding improvements, but the farther the competition got, the more psychological each breakthrough became.  Cult classics and unclassifiable movies like “Lost in Translation” or “I Heart Huckabees” throw a wrench into the system and make accurate predictions almost impossible.  Can computers accurately predict taste when movie rentals are often based on nuances in human behavior?  This problem has affectionately been dubbed the “Napoleon Dynamite Problem.”  Human taste evolves and changes, making cultural predictions based solely on existing data difficult though not impossible, however some computer analysts are suggesting that the competitors have reached the limits of predictive algorithm models.  If the computer accounted for these idiosyncrasies by suggesting something more extreme, Netflix could risk getting the customers tastes wrong and lose their business.  Has the Netflix Prize has brought us to the edge of computational accuracy or will programmers be able to adjust for individualistic and quirky human behavior?  What do you think?

[via NYTimes Magazine]