This post is in Estzter territory, and probably just reflects ignorance on my part, but I’d be grateful for the information from those in the know, anyway. Following “one of Eszter’s posts recently”:https://crookedtimber.org/2005/02/07/networks-and-tastes/ , I signed up to “Movielens”:http://movielens.umn.edu/ and have been dutifully entering my ratings in various spare moments. Like Amazon, “Movielens”:http://movielens.umn.edu/ tells me that based on the movies I like I should check out various other ones. Presumably, the program checks the database to see which movies I haven’t seen are highly rated by other people who like the same films that I liked (ditto Amazon for books, dvds etc).
Now here’s my problem. When we all come to such systems “cold” (as it were), the links between our choices provide genuinely informative data. But once we start acting on the recommendations, even chance correlations can get magnified. So, for example, suppose we have three movies A, B and C. Perhaps if we showed these films to a randomly chosen audience there wouldn’t be any reason to suppose that people who like A prefer B to C or vice versa. But if the first N people to go to the expert system happen to like both A and B, then the program will spew out a recommendation to subsquent A or B lovers to follow up their viewing with B or A. And those people in turn, having viewed the recommended movie, will feed their approval back into the system and thereby strengthen the association. Poor old movie C, excluded by chance from this self-reinforcing loop, will not get recommended nearly so often.
I guess the people who design these systems must have considered these effects and how to counteract them. Any answers?
{ 5 comments }
lth 03.17.05 at 6:43 am
To be honest, I don’t think the people who invent these systems have thought about these effects, and they work exactly as you describe.
Tom T. 03.17.05 at 8:04 am
All else being equal, there shouldn’t be any reason to think that there will be significantly more people who start out by discovering A + B rather than A + C or B + C.
Also, even assuming that C did get excluded by chance from these moviegoers’ initial discovery, people don’t stop at two movies. Those who have seen A + B will now have to go out and discover a third movie on their own. Presumably, they’re just as likely to discover C by chance as they were to have initially discovered B by chance after having seen A.
Michael Mouse 03.17.05 at 11:25 am
I’ve designed similar systems. I’m still surprised that they work so well. What if people have strange and idiosyncratic habits? It turns out that most people don’t, or at least, that useful information can come from the aggregate of lots of strange and idiosyncratic habits. You can come up with any number of theoretical objections, but empirically it seems to work well enough to keep the punters happy.
On the particular issue mentioned, I’m convinced that the reason they continue to work is that they are not the only thing governing people’s choices. So, ‘frinstance, Chris will presumably still be getting ideas for movies to watch from friends, advertising, blogs, old-fashioned news media, etc, and will continue to do so as well as Movielens.
anonymous 03.17.05 at 1:39 pm
People have most certainly thought about these problems. The animating principle is to recommend the movie you have the highest probability of liking, NOT the movie that the greatest raw number of people have seen and liked, with a similar background to you. Otherwise, the most popular movies would get recommended to everyone, with slight changes. If this wasn’t built in, the system wouldn’t work at all.
Sumana 03.17.05 at 2:28 pm
One way to avoid such problems is to make sure you institute “you may not have thought about this”/out-of-left-field recommendations, and instituting an “indie rock” preference that DOESN’T recommend things that you have probably already heard about and seen or otherwise considered. A recommendation engine for web pages implements these:
http://www.crummy.com/software/UltraGleeper/
Comments on this entry are closed.