A $1 Million Research Bargain for Netflix, and Maybe a Model for Others - NYTimes.com
A $1 Million Research Bargain for Netflix, and Maybe a Model for Others Even the near-miss losers in the Netflix million-dollar-prize competition seemed to have few regrets.
Skip to next paragraph
Jason Kempin/Getty Images, for Sunshine Sachs“You’re getting Ph.D.’s for a dollar an hour,” Reed Hastings, chief of Netflix, said of the people competing for the prize.
Jason Kempin/Getty Images, for Sunshine SachsThe medals for the Netflix $1 million prize contest.
Netflix, the movie rental company, announced on Monday that a seven-man team was the winner of its closely watched three-year contest to improve its Web site’s movie recommendation system. That was expected, but the surprise was in the nail-biter finish.
The losing team, as it turned out, precisely matched the performance of the winner, but submitted its entry 20 minutes later, just before the final deadline expired.
Under contest rules, in the event of a tie, the first team past the post was the winner. “That 20 minutes was worth a million dollars,” Reed Hastings, chief executive of Netflix, said at a news conference in New York.
Yet the scientists and engineers on the second-place team, and the employers who gave many of them the time and freedom to compete in the contest, were hardly despairing.
Arnab Gupta, chief executive of Opera Solutions, a consulting company that specializes in data analytics, based in New York, took a small group of his leading researchers off other work for two years. “We’ve already had a $10 million payoff internally from what we’ve learned,” Mr. Gupta said.
Working on the contest helped the researchers come up with improved statistical analysis and predictive modeling techniques that his firm has used with clients in fields like marketing, retailing and finance, he said. “So for us, the $1 million prize was secondary, almost trivial.”
Indeed, since it began in October 2006, the Netflix contest was significant less for the prize money than as a test case for new ideas about how to efficiently foster innovation in the Internet era — notably, offering prizes as an incentive and encouraging online collaboration to tap minds worldwide.
The lessons of the Netflix contest could extend well beyond improving movie picks. The researchers from around the world were grappling with a huge data set — 100 million movie ratings — and the challenges of large-scale modeling, which can be applied across the fields of science, commerce and politics.
The prize model is increasingly being tried on work like new science and freelance projects in design and advertising. The X Prize Foundation, for example, is offering multimillion-dollar prizes for path-breaking advances in genomics, alternative energy cars and private space exploration.
InnoCentive is a marketplace for business projects, where companies post challenges — often in areas like product development or applied science — and workers or teams compete for cash payments or prizes offered by the companies. A start-up, Genius Rocket, runs a similar online marketplace mainly for marketing, advertising and design projects.
“The great advantage of the prize model is that it moves work away from the realm of the beauty contest to being performance-oriented,” said Michael Schrage, research fellow at the Center for Digital Business at the Sloan School of Management at the Massachusetts Institute of Technology. “It’s the results produced that matters.”
The emerging prize economy, according to some labor market analysts, does carry the danger of being a further shift in the balance of power toward the buyers — corporations — and away from most workers.
Thousands of teams from more than 100 nations competed in the Netflix prize contest. And it was a good deal for Netflix. “You look at the cumulative hours and you’re getting Ph.D.’s for a dollar an hour,” Mr. Hastings said in an interview.
Netflix, Mr. Hastings said, did not do a crisp cost-benefit analysis of its investment in the contest. But several crucial techniques garnered from the contest have been folded into the company’s in-house movie recommendation software, Cinematch, and customer retention rates have improved slightly. Better recommendations, Netflix says, enhance customer satisfaction.
“We strongly believe this has been a big winner for Netflix,” Mr. Hastings said.
The prize winner was a team of statisticians, machine-learning experts and computer engineers from the United States, Austria, Canada and Israel, calling itself BellKor’s Pragmatic Chaos. The group was actually a merger of teams that came together late in the contest.
In late June, the team finally surpassed the threshold to qualify for the prize by doing at least 10 percent better than Cinematch in accurately predicting the movies customers would like, as measured against actual ratings. Under the contest rules, that set off a 30-day period in which other teams could try to beat them.
Post a Comment on Bits