Also – for those who say “no one trusts recommendation systems anyway” – there is an influential set of people who would claim otherwise and (here is my real gripe) are pushing creative and idealistic artists towards trusting outlets like Amazon and Netflix to directly connect to their audience without any of those elitist publishers in the way. See Clay Shirky’s talk at http://web2expo.blip.tv/file/1277460/ and, of course, The Long Tail for examples.

But if there is any one scenario I have in mind it is Amazon. I don’t see it being a diversifying force at all when it comes to books and literature.

OK, but if we’re talking about the recommendation system alone I don’t see how it’s anything but a trivial result. I mean, the whole basis of a recommendation system is “people who like this, also tend to like that”. That’s obviously not a driver for diversification but reinforcement of existing relationships. Indeed, this is the big challenge for creating useful recommendation systems, as the Netflix prize has shown. They can’t account for off-beat combinations of tastes or “Marmite” films.

Oh, I think it is clear when you consider the vast pool of historical artifacts that were previously buried and inaccessible but now are available to all (e.g. public domain book scans, historical youtube musical performances).

But if you want to worry about something, worry that this explosion of availability of historical cultural artifacts is necessarily competing with new cultural production—because past culture is now so readily available, it’s likely that the incentives for production of new culture is impaired to some degree.

Well, it has expanded the choices available to any individual. But whether it has expanded the amount of options in total is a lot less clear. I could imagine Tom Slee’s phenomenon even without recommendation systems.

My own experience is that the internet has made my reading material more English/American-oriented, and this is probably true for people all over the world. Even the offline recommendations around me have become more American, simply because everyone has a similar exposure.

It might be true that this effect is more than compensated for by non-geographic niching, but that’s far from certain.

But remixing and ‘geographical subcultures’ are not the only alternatives. Interest-based subcultures can be as self-contained as geographical subcultures ever were. Which is to say—somewhat, but not very, self-contained. When you read Kwame Appiah, for example, you understand that the containment of geographical sub-cultures has long been quite porous.

2. Gini is a very very very bad measure for concentration when you have long tails (lots of products with very few buyers). If you add a million fictitious products with zero sales to the actual sales distribution of a vendor the Gini will shoot up. This isn’t a problem if your choice set is invariant and compact but it’s a big problem if for the long tail of products the membership to the choice set is ambiguous (e.g. sold only by third party vendors, out of print and only available used, erroneous double listings, etc.). A better way of looking at concentration in online vs. offline sales is the contribution to total sales of the N best-selling products.

The internet has obviously expended the universe of choices by an enormous factor—that’s not in question. Is the power of recommendation systems such that despite the new richness of options, people are consistent steered by recommendation algorithms into a lower level of cultural diversity? I don’t believe it. Certainly not based on those simulations (and on no data).

And I agree with others here that the offline-equivalent of the recommendation system might not be friend-to-friend contact. If you take bestseller lists, or music top-100s as the offline equivalent, the recommendation system looks a lot better.

Music tops shw by the way a clear example of a system where diversity reduction is a feature, since people actively prefer to know the music other know too. Subcultures may scorn those lists, but they do tend to have their own, less explicit, lists to have everyone on the same page.

Maybe, maybe not.

Ok, but again I don’t understand why you think the friend to friend to friend model is close to what happens with brick and mortar specialty stores. A much better model would be 2-4 key employees doing the recommendations for everyone. And that doesn’t sound like it would lead to much more diversity.

@Bruce & Sebastian: it may also turn out that letting new subcultures develop in a relatively protected environment may do good things. For all the talk about remixing, for me most of the more interesting musical trends come out of particular geographic scenes.

@Ginger Yellow: it’s just a posting, not a paper. But if there is any one scenario I have in mind it is Amazon. I don’t see it being a diversifying force at all when it comes to books and literature.

@Nina: I label the points as consumers and products, but that’s just convenience really. Still, I agree that treating culture as a matching problem has some pretty strong limitations. I could claim it models “believers” and “religions” because that’s also a matter of taste, but I’d not want to.