It’s a measure of a term achieving zeitgeist status when people apply it liberally, even in circumstances where it doesn’t really fit — as with the managers who sought to label their initiatives as Total Quality Management or Business Process Re-engineering in the nineties, even if they only half-grasped the original intention behind these terms. In the 21 months since Chris Anderson published his article on the Long Tail in Wired, this new term has come close to achieving similar status. To some extent that’s a sign of its attractiveness and strength, but will it end up (forgive the almost unavoidable pun) tailing off, as TQM and BPR have done?

The Substance of the Long Tail

A quarter of the way into his book The Long Tail: How Endless Choice Is Creating Unlimited Demand, Anderson explains, “The theory of the Long Tail can be boiled down to this: Our culture and economy are increasingly shifting away from a focus on a relatively small number of hits (mainstream products and markets) at the head of the demand curve, and moving toward a huge number of niches in the tail.”

What drives this Long Tail and the niches that populate it? Anderson identifies three ‘forces’.

1. More stuff is being produced. Technology and the internet make it cheaper and easier to record and distribute your own songs, publish your own writings and so on. This lengthens the tail.
2. There is better access to niches, again thanks to the reach and economies on the net. This fattens the tail.
3. Search and recommendations connect supply and demand. This drives business from hits to niches.

All this is good stuff. Anderson suggests that we all have niche interests; it’s just the constraints of mass media that have focused us on the common denominators of mainstream hits. Clearly this resonates with a lot of people (and I’m among them), who have been quick to adopt the term and apply to it to their own domains.

Anderson explains that the Long Tail is an example of a power law, and that “powerlaws come about when you have three conditions:

1. Variety (there are many different sorts of things)
2. Inequality (some have more of some quality than others)
3. Network effects such as word of mouth and reputation, which tend to amplify differences in quality.”

These conditions are important, for reasons I’ll come back to. Why do they breed Long Tails? “The characteristic steep falloff shape of a popularity powerlaw comes from the effect of powerful word-of-mouth feedback loops that amplify consumer preference, making the reputation-rich even richer and the reputation-poor relatively poorer. Success breeds success,” writes Anderson. Which brings me to the first area in the book that I’d like to see developed further.

Word-of-mouth, filters and network effects

If word-of-mouth feedback is what creates the steep fall-off shape, should the internet’s amplification and speeding up of word-of-mouth make the fall-off steeper? Why shouldn’t the amplification mean there is worse access to, or worse awareness of, the reputation-poor niches? Does the amplification apply unevenly, or does the third force (filters and recommendations to connect supply and demand) counteract it?

Anderson seems to imply that maybe both of these apply. He says, “filtering is often most effective at the genre level rather than across the entire market… between genres their [filters’] effect is more muted.” I struggled to follow the evidence to support this argument. How could filters and recommendations work best at genre/category scale, when genres and categories are abstract constructs that people impose over the data, and the genres I use to divide a catalogue may not be the same ones you use?

I wonder if perhaps what Anderson is grappling towards is the increased independence that the internet provides for communities of fans to organise themselves and communicate, separate from the mainstream. This scope for word-of-mouth amplification within small pockets of the market could help fatten the tail and boost awareness of niche products within niche communities. But if that is what Anderson meant, I couldn’t find it clearly articulated or explained.

Independence and the wisdom of crowds

In general, the book seems muddled in its treatment of the role that individual, independent opinions play in creating filters, and in the circumstances under which it’s helpful to aggregate or average these opinions. Anderson refers several times to the term The Wisdom of Crowds, named after James Surowiecki’s book of the same name. In charting how ‘the many can be smarter than the few’, Surowiecki specifies clear conditions on which the wisdom of the crowds depends (summarised in Wikipedia), and gives many examples of circumstances when crowds are the opposite of wise.

Anderson’s liberal, broad-brush references to the wisdom of crowds ignore these conditions and imply that it can be taken for granted across many situations: “Yahoo music ratings, Google PageRank, MySpace friends, Netflix user reviews — these are all manifestations of the wisdom of the crowd”. No, they’re not. Many of them fail Surowiecki’s conditions, and fall prey to what he calls ‘information cascades‘ (a lack of independence between the people making the ratings or reviews). Anderson even contradicts himself in this area. Referring to Top 10 lists, he asserts, “There’s nothing wrong with ranking by popularity — after all, that’s just another example of a ‘wisdom of the crowds’ filter.” But then he says things that don’t sell well may be just as good as those that do, and argues a few pages later that, “The movies don’t get worse at rank 100 (some would argue they actually get better)”. Apparently he puts little store in the ‘wisdom’ of the rankings.

Long Tails everywhere?

It’s one thing to be a little cavalier with someone else’s terminology and theory. I wonder if Anderson is also sometimes a little lax with his own, which may encourage others to see Long Tails where they don’t exist. I guess this is one of the hazards of riding the zeitgeist, but he cites with approval someone’s reference to the Long Tail of National Security (involving niche producers of violence), even though it’s hard to see how this example meets the three conditions for power laws and the three forces for growing the tail, as given above.

Some of his case studies also stretch the application of the Long Tail terminology to breaking point. I found myself struggling with the analysis of the rise of Chicago House music, which explains that, “Clubbing is really about surfing the Long Tail of dance music” (my emphasis) and that “clubs and warehouse parties offered… democratized distribution”. Maybe such analysis will become widely accepted, and the resistance I feel will be similar to that which Freud’s first readers experienced when informed that certain behaviours were really about killing your father and sleeping with your mother. Time will tell.

But no Long Tails in the Long Tail business?

I asked myself, “What’s the lesson of this book, and who stands to profit from it?” Anderson provides a succinct answer to the first part of the question, in his last chapter: “The secret to creating a thriving Long Tail business can be summarized in two imperatives: 1. Make everything available; 2. Help me find it.”

Not every reader is in a position to make everything available, even within their own domain or niche. The organisations that can are those that Anderson cites throughout the book: Ebay, Amazon, Rhapsody, iTunes, Google, Netflix. So does the future belongs just to the big aggregators?

A year ago, in this blog post, Anderson acknowledged that most aggregators are ‘hits’ — big corporations with mass markets — but added that there are other business opportunities for niche suppliers and filters. Why doesn’t the book address these opportunities at any length and provide advice for those niche suppliers and filters?

It seems almost as if there is a loss faith in the viability — at least in pure economic terms — of small scale Long Tail businesses. “Consumers want one-stop shopping… Stores that give consumers the most confidence that everything they want is there are going to be the ones that succeed.” But isn’t the web the one stop we need for shopping, the platform that aggregates a billion stores, all within a few convenient clicks of each other?

Having criticised Anderson for over-generalising in his use of Long Tail thinking and terminology, I wonder why he holds back from applying it to the very domain that he is mapping out. Stressing the need for aggregators to include both the head and the tail of the market, Anderson provides the case study of defunct MP3.com: “The problem with MP3.com was that it was only Long Tail… there was no familiar point of entry for consumers, no known quantity from which further exploring could begin” (emphasis in original). Does not the same point apply to the successful independent retailer and rights aggregator CD Baby, which Anderson refers to positively in both the book and his blog. Surely the Long Tail conditions (above) do apply to filters and recommendations — rather better than to national security.

What’s missing

So if I were a niche supplier or a builder of recommendation systems, what could I learn from this book? Lacking Amazon’s budget, I can’t ‘make everything available’. Therefore, to thrive, I have to help people find stuff. Otherwise the business case is going to be challenging, to say the least: I may have to accept that I’m going to be in the part of the Long Tail that runs on commitment, reputation and spin-off benefits rather than by paying its way financially.

If Yahoo, Google, Amazon and Netflix are the ‘head’ of recommendation systems, then blogs and fan sites are the Long Tail. “Blogs,” writes Anderson, “are shaping up to be a powerful source of influential recommendations… What they may lack in polish and scope, they more than make up in credibility” (emphasis in original). But, although he makes frequent references to blogs, Anderson doesn’t address the impact of blog recommendations, and how to be an influential blogger, at any greater depth than this. His advice is aimed at the head.

A richer engagement with the little guys in the recommendation and filtering business might lead to a better modelling of how word-of-mouth works, how market niches develop, and the dynamics of independence and influence between them.

This would help us get a deeper understanding of Anderson’s third force.

It might also help understand the processes that underpin another Long Tail phenomenon that his book is curiously silent on. Some cultural products lie dormant in the lower reaches of the Long Tail for years or even decades. Then suddenly word-of-mouth grows, apparently out of nowhere, and the products move from the tail much closer to the head. There are many examples of such re-discovery in the music world, but Anderson himself started off the Long Tail bandwagon with just such an example from the book world, in the story of how Touching The Void became a hit when it was on the verge of going out of print. Anderson puts this example down to Amazon’s recommendation systems, but they surely cannot be the whole of the story. If more books, films and music go unappreciated for long periods and then suddenly find favour, this will have a significant impact on the development, health and economics of our culture. It’s worth trying to understand how this happens.

Buzzword marketing

Over the last week, questions have been raised about the robustness of the data underpinning the Long Tail thesis. These seem to be questions of degree rather than the core substance of the thesis, and I’m not in a position to judge them (though I noticed that Anderson drew global conclusions from US-only data about radio listening and music sales, even though some of the the trends are relatively short-term and are not clearly reproduced in other markets).

Andrew Orlowski gives a fairly loaded run-down of the criticisms, both before and since Chris Anderson’s initial response. Orlowski is no stranger to stirring up controversy (some would say ‘trolling’), so take him with a pinch of salt when he writes, “The Long Tail, then, isn’t snake oil. Rather its restorative properties have been exaggerated. Who wants to let facts get in the way of a buzzword bestseller good thesis?”

Nevertheless my concerns about Anderson’s loose use of concepts and terminology are consistent with Orlowski’s suggestion that the Long Tail has been sexed up a bit to maximise its buzzword profile. If the Long Tail plays on being a faddish term, then its shelf-life may be limited. As cited in Wikipedia, fashionable management terms (like Quality Circles, Total Quality Management and Business Process Re-engineering) tend to follow a life-cycle in the form of a bell curve. And a bell curve, unlike a power law, has quite a short tail.