The Netflix prize, about to be won

Apparently at least one entrant has qualified to win the $1 million Netflix grand prize. One can follow the progress of the teams on the Netflix Leaderboard: http://www.netflixprize.com/leaderboard.

Netflix is company that lends movie DVDs by postal mail or streams directly to a computing device, for an monthly subscription fee. According to the company, the “Netflix Prize seeks to substantially improve the accuracy of predictions about how much someone is going to love a movie based on their movie preferences. Improve it enough and you win one (or more) Prizes. Winning the Netflix Prize improves our ability to connect people to the movies they love.” The contest began in October 2006, and features annual “progress” prizes of $50,000 (for the best result achieved), and a grand prize, now apparently about to be won, of $1 million, for improving the Netfix predictions of movies by 10 percent.

An interesting account of the competition, from some members of the first team to qualify for the grand prize, was published in the May 2009 IEEE Spectrum.*

The detailed rules that are available here, include this summary.

Terms and Conditions in a Nutshell

  • Contest begins October 2, 2006 and continues through at least October 2, 2011.
  • Contest is open to anyone, anywhere (except certain countries listed below).
  • You have to register to enter.
  • Once you register and agree to these Rules, you’ll have access to the Contest training data and qualifying test sets.
  • To qualify for the $1,000,000 Grand Prize, the accuracy of your submitted predictions on the qualifying set must be at least 10% better than the accuracy Cinematch can achieve on the same training data set at the start of the Contest.
  • To qualify for a year’s $50,000 Progress Prize the accuracy of any of your submitted predictions that year must be less than or equal to the accuracy value established by the judges the preceding year.
  • To win and take home either prize, your qualifying submissions must have the largest accuracy improvement verified by the Contest judges, you must share your method with (and non-exclusively license it to) Netflix, and you must describe to the world how you did it and why it works.

The intellectual property rights for the “Winning Algorithm” are handled as follows:

After qualifying for either the Grand or Progress Prize and being verified by the Contest judges, as a condition to receiving either Prize, the winning individual and/or team must grant to Netflix (including its affiliates and subsidiaries, employees, agents, and contractors), an irrevocable, royalty free, fully paid up, worldwide non-exclusive license under the Participants’ copyrights, patents or other intellectual property rights in the winning algorithm (”Winning Algorithm”) to reproduce, distribute, display, and create derivative works from the Winning Algorithm and also to make, have made, use, sell, offer for sale, and import products that would otherwise infringe the Winning Algorithm. Except as encompassed in the concept of “have made”, this license will not include the right to grant further licenses or sublicenses.

The Netflix prize stimulate a large academic literature of data mining techniques [such as these articles in Google Scholar], and reportedly has attracted so far 49,251 contestants on 40,467 teams from 184 different countries.

In recent blog, Melody Hildebrandt offered this comment regarding the incentives in the prize contest to collaborate:

This is one of the best examples of crowdsourced innovation and problem-solving out there. The winning team was initially 4 disparate pairs or individuals who they realized that they had complementary skills — machine learning, computer science, engineering — and decided to collaborate. Throughout the competition, as the market leaderboard tracked the top performers, teams would routinely share lessons learned. Even with $1M at stake, the market can indeed be collaborative and come to a better solution than a single company alone.


*Bell, Robert M., Jim Bennett, Yehuda Koren, and Chris Volinsky. “The Million Dollar Programming Prize: Netflix’s bounty for improving its movie-recommendation software is almost in the bag. Here is one team’s account.” IEEE Spectrum, May 2009. http://www.spectrum.ieee.org/computing/software/the-million-dollar-programming-prize/0.

See also:

Buskirk, Eliot Van. “$1 Million Netflix Prize So Close, They Can Taste It.” Wired.Com, June 17, 2009. http://www.wired.com/epicenter/2009/06/1-million-netflix-prize-so-close-they-can-taste-it/.

Hildebrandt, Melody. “Netflix prize is (nearly) awarded! A model in crowdsourcing.” TransCapitalist, June 26, 2009. http://www.transcapitalist.com/transcapitalist/2009/6/26/netflix-prize-is-nearly-awarded-a-model-in-crowdsourcing.html.

Linden, Greg. “Netflix Prize at KDD 2008.” Geeking with Greg, October 8, 2008. http://glinden.blogspot.com/2008/10/netflix-prize-at-kdd-2008.html.

Lohr, Steve. “And the Winner of the $1 Million Netflix Prize (Probably) Is ….” New York Times, June 26, 2009. http://bits.blogs.nytimes.com/2009/06/26/and-the-winner-of-the-1-million-netflix-prize-probably-is/.

Thompson, Clive. “If You Liked This, You’re Sure to Love That.” The New York Times, November 23, 2008, sec. Magazine. http://www.nytimes.com/2008/11/23/magazine/23Netflix-t.html.

For more background on innovation inducement prizes, see:

March 7, 2008. Selected Innovation Prizes and Reward Programs, KEI Research Note 2008:1

An Annotated Bibliography of Scholarly and Technical Articles and Books on Innovation Prizes, KEI Research Note 2008:2

http://www.zotero.org/groups/innovation_inducement_prizes

Prizes to stimulate innovation

Uncategorized