Screen-scraping from Wikipedia to Google Books

Most people will have had this experience: you go to a search engine and type in a search term and up come a host of results with what looks like encyclopedic entries. But you click on them and they all have exactly the same text.

This practice is known generally as data scraping. For online data, it's web scraping or web harvesting. When you're reading webpages and taking the text from them, that's screen scraping, but it's also possible to use RSS feeds, databases, and other sources to get text. Why? Well, once you have text, you can get people reading it, and once you have readers, you can get money from advertising.

Thanks to Google and other less reputable ad brokers, it's easy to stick ads on your webpage and make some money. For this reason, many people thought Google would never remove screen-scraping sites from their search, but recently they seem to have taken action and such sites have fallen down the rankings.

answers.com is perhaps the leading example, reproducing Wikipedia pages, though other sites do the same - wapedia reformats Wikipedia for mobiles; astrology site astrotheme.com combines star signs with Wikipedia biographies. Sites such as fullbooks.com display uncopyrighted books - their edition of Emma serves Google ads asking "Looking For Rich Women" and offering "Inside A Boyfriends Mind". Anyone can display an out-of-copyright text, and because of the licensing of Wikipedia content, it can be reproduced as long as you credit or link back to the source. Wikipedia does not show advertising but many of the sites reproducing its content do.

Some companies attempt to increase their value by adding extra functionality. The now-defunct LJ Find, for instance, scraped LiveJournal (based on RSS feeds) and offered a search facility - something the site itself didn't offer - as well as displaying the entire contents of people's journals with ads alongside. Other sites offer fully-searchable novels, or attempt to package content from multiple sites onto one page.

It's in this context that I come to Google Books. While Google have removed many of the web-scraping sites from their search results, the world's leading ad broker has another way of making money from other people's content. They have digitised a huge number of books and now serve them up online with appropriate adverts displayed both when you search for a book, and in a side panel when you read the book. I'll be interested to see if anyone can screen-scrape the books that Google has scraped from the world's libraries, and put up their own advertisements - or even better, ads supplied by Google.

Annoy a liberal: work hard. Annoy a conservative: tweet?

The new Freakonomics book from Levitt and Dubner has caused controversy. Critics have accused them of wilful contradictarianism, and moved on to question why their controversialism is anti-liberal, and thence to ask why it seems to be so much more fun to contradict liberals than conservatives.

Andrew Gelman asks:

why is it that "pissing off liberals" is delightfully transgressive and oh-so-fun, whereas "pissing off conservatives" is boring and earnest?

He mentions Nate Silver and Michael Moore as people who successfully piss off conservatives, but they're in the minority of all the counterintuitive argumentation in the world.

Kieran Healy complains that it's impossible to get a liberal equivalent of the rightwing bumper sticker "Annoy a Liberal. Work hard. Succeed. Be happy." Commenter Anthony (referencing a remark of LBJ's) relates this to the apologetic nature of liberals:

American liberals have a problem because it's altogether too easy to force them to deny that they're pig-fuckers, and they have no clue why.

Another commenter, Bad Jim, may get closer to the problem

It's a waste of time to piss off conservatives. They're already as angry as can be.

John Quiggin continues the debate by questioning the idea of whether it's really so wonderful to be contrarian (even though it does sell books.

Contrarianism is a cheap way of allowing ideological hacks to think of themselves as fearless, independent thinkers, while never thinking (in fact reinforcing) the status quo. [...] To sum up my current view: "contrarianism" is mostly contrary to reality, the "conventional wisdom" is probably wiser than the typical unconventional alternative, and "politically incorrect" views are almost always incorrect in every way: literally, scientifically and morally.

Mark Liberman, in a blog post called "Freakonomics: the intellectual's Glenn Beck?" is suspicious of the contrarian economists, referencing Barbara Ehrenreich's recent critique of their claim that statistics show women are more miserable now than pre-feminism. He points out that contrarianism of whatever stripe sells books. And J Bradford Delong offers advice on how Levitt and Dubner could rewrite their own book.

Disputes about global warming are hardly novel, and nor is worrying over the lack of left-wing polemic, particularly in an America where Rush Limbaugh and Glenn Beck call the shots. In Britain the more common complaint is that all satirists are left-wing (except Ian Hislop). But the right still have Jeremy Clarkson, Jeremy Kyle, possibly James Whale. The left may have jokes and sandals, but they're not ranting, with even civil liberties being as likely to be defended by crazy hate-filled right-libertarians as by bleeding heart liberals.

Despite this, Twitter's British membership (which sometimes seems to be the entire Guardian readership and their electronic devices) seems quite adept at conducting nice witch-hunts with liberal ideals, with the last few days seeing agitated action against Carter-Ruck and Trafigura injuncting newspapers and Daily Mail columnist Jan Moir being homophobic following Stephen Gately's death, following the more benign welovetheNHS Twitter campaign.

But is it the left shouting here? Much of the antipathy to Carter-Ruck came from right-wing bloggers: Iain Dale, Guido Fawkes, and the Spectator. It's fun having a go at judges and shady multinationals, playing the game of figuring out what's being injuncted; your politics isn't important. Likewise, based on most people's uninterest in the Mail's past transgressions it seems to be mainly Boyzone fans who're really upset with the coverage of Gately's death. Sometimes you need to join a crowd and shout against conservatives and capitalists, but that doesn't make you a socialist. Though maybe Gately did more for gay rights than Peter Tatchell.

Sic ASA Parrot

The first edition of what will hopefully be a regular roundup of adjudications by the Advertising Standards Authority, the British ad industry watchdog.

Danone not so good for you? The yogurt manufacturer failed to show Actimel is "scientifically proven to help support your kids' defences". Referenced studies were on hospitalised Indian kids or children under 3 years, neither of which featured in the TV ad.

Peta falsely imply that eating meat causes swine flu. Rearing pigs might get you, but fortunately most bacon-eaters never see a real live oinker.

Male drivers ARE dangerous. A government campaign warned women not to get into a car with a man if she didn't trust his driving. A viewer felt this was offensive to male drivers, not all of whom are crazed careless killers on wheels (allegedly). The ASA felt the ad "highlighted the statistical risks to encourage prudent behaviour" - who could ask for anything more?

Abtronic? Abysmal. Sellers of an electronic muscle-stimulation belt were picked up by ASA monitors. Oh dear: it's not the easy path to a sculpted six-pack, no substitute for going to the gym, and the ASA doubted claims that it was relaxing, suggesting there's no pain no gain: "We acknowledged that the 'relaxation' setting on the Abtronic X2 was likely to feel relatively comfortable, but considered that to achieve the claimed muscle-toning, the Abtronic X2 would need to be set to the exercise programme, which was likely to cause some pain or discomfort." Ouch!

We Buy Any Car vindicated. They'll buy any car, even a limo, but not your pick-up. Because it's not a car. Sorry, complainer!

Tesco and Asda are at war. The casus belli: who has the cheaper prices. Asda tries to prove it's better with an independent price comparison website checking a wide selection of products. Tesco has a different plan. They select random people's shopping baskets (I assume they store details of everything each person buys, rather than grabbing a random shopper as he heads for the tills with two fillet steaks, a bottle of Cava, and a twelve-pack of condoms), and they check the price of this randomly selected basket against what Asda would sell it for. Someone complained, objecting that this is biased as it doesn't take account of the fact that 95% of Tesco products could be overpriced, but if people walk in the door, see the prices, refuse to buy the bad-value stuff and just take the buy-one-get-one-free spinach bags and the half-price wine from a country you've never heard of, then Tesco will look really cheap (assuming ASDA doesn't have deals on spinach and Bolivian Merlot). The response: "Tesco said it was generally accepted that advertising sought to present companies in the most favourable light. Price comparison ads were, by their very nature, highly biased in favour of the advertiser." The ASA said: victory to Tesco. Also cheaper than Boots, by slightly better reasoning.

Park's miracle mattress won't help you concentrate or relieve your pain.

Coke in trouble over Vitaminwater. Viewers may confuse Jean-Claude van Damme and Brussels sprouts, and that's just the beginning of their problems, as the ASA dispute their health benefits and suggest 4.6g sugar/100ml is a lot of calories for a drink that claims to be water.

Something about frying-related franchises. Purifry Ltd challenge FiltaFry's uniqueness. Purifry offers franchised deep-fat fryer care, cleaning, and oil disposal. FiltaFry do much the same but their website has exciting videos! There are business opportunities here! It's war of the fryers! This could get messy!

talkSPORT offer pre-pay Maestro cards? And don't tell you that there's a fee for topping them up? Naughty.

Penis pump maker deflated.

Chinese buffet ok. Anyone can call themselves "world famous" even if nobody's heard of you - it's a meaningless subjective phrase not a statement of fact.

Mercedes not so environmentally sound.

Leona Lewis in trouble: apparently if you call something your "new album" it shouldn't just be a bunch of old tracks in a new cover.

Guthy-Renker beauty products - this web page offers ASA highlights, but it seems that most of their adjudications are against utility companies, made-up religions, and French cosmetics with dubious scientific claims.

Universal Church of the Kingdom of God's blessed oil probably won't save your life, even with the disclaimer "The UCKG does not claim to heal people but believes that God can through the power of faith."

The Queen is in trouble: truly the ASA fears nobody. The Duchy of Lancaster making unjustified claims in an attempt to build new houses in the ancient village of Cloughton.

Book of historical failures

An amusing old conspiracy theory has recently come my way. In 1981 many right-wing Labour party members quit to form a new party, the SDP. The year before, the Labour party had elected the very left-wing and pro-disarmament Michael Foot as its leader. Some of those who formed the SDP voted for Foot despite their political differences. Why? Nevil Sandelson, an MP for Labour and later the SDP, claimed that he was one of six right-wing Labour members who conspired to elect Foot and thus destroy the Labour party. Sandelson went even further: as an SDP MP in 1987 he called on voters to vote for the Tories rather than his own party, in order to destroy Labour more thoroughly. Could any MPs today have similar plans? (The Telegraph's obituary of Sandelson)

In a garret or Waitrose - on the earnings of poets

Earlier this month, New Zealand poet Leigh Davis died, aged just 54 (Charles Bernstein; National Business Review). As well as a poet and publisher, Davis was a successful businessman and merchant banker.

Despite the popular impression of starving poets in garrets, many poets actually made or make a decent living - just not from poetry. While some doubtless get by in menial jobs waiting tables or cleaning floors, most successful poets seem to find some form of middle-class employment. Perhaps the starving poets can't afford the stamps to submit their work or like Weldon Kees and Hart Crane commit suicide, or maybe poetry does bring rewards. Clearly there is more research to be done on whether the jobs precede or follow the poetry. What follows is just a sketch of how poets get their money.

Insurance executive Wallace Stevens is one of the most famous examples, working his way to vice-president of the Hartford Accident and Indemnity Company. The few poets to be seriously rich have inherited their money, such as Frederick Seidel, who is acclaimed as one of the greatest living American poets for verse about wasted lives and motorbikes, and Elizabeth Bishop who was independently wealthy until late in her life. Richer than both was James Merrill, the son of Charles Merrill of Merrill Lynch. In an earlier age, Elizabeth Barrett Browning was the child of a wealthy sugar plantation owner, although later they fell on hard times and she married Robert Browning.

While few poets were as rich as those, poetry is perhaps the most middle-class of all the arts. William Carlos Williams was a doctor, though not wealthy. TS Eliot worked for Lloyd's Bank until 1925. Robert Bridges, who had little success as a poet during his lifetime, was a doctor until lung disease forced his retirement. RS Thomas was a clergyman who lived a very simple, austere life; in an earlier age, Gerard Manley Hopkins was a priest though he never published during his lifetime. For women poets, a husband is a common source of income - these days perhaps it works the other way around. Edna St. Vincent Millay was supported by her husband in her verse and unconventional life. HD (Hilda Doolittle) had wealthy husbands and many admirers. Sylvia Plath, little-known during her life, had a pushy mother and a husband with connections.

The best or luckiest poets, particularly the Americans, supplemented their income with grants and awards such as the Fulbright program, Guggenheim fellowship, and MacArthur fellowship, which can provide large sums.

---

There is a welfare system for poets with fewer commercially-valuable skills and no ancestors in trade: this funding scheme is the university system, active on both sides of the Atlantic. Some poets, such as AE Housman and William Empson, were at the top of their academic specialisms, others were inspirational teachers. Some probably got their jobs through their fame as a poet; others took a conventional route into academia. Here follows a brief list of poets in academia:

• Maya Angelou: Wake Forest University (American studies)
• John Ashbery: Brooklyn College, Bards College (languages and literature)
• WH Auden: U of Michigan, Bennington, Smith, Oxford (poetry) and others
• Charles Bernstein: Columbia University, Brown University, and Princeton University (poetry and creative writing)
• John Berryman: U of Iowa
• Elizabeth Bishop: U of Washington, Harvard, NYU, MIT
• Carol Ann Duffy: Manchester Metropolitan University (English and creative writing)
• William Empson: Peking University, Sheffield (English), and others
• Robert Frost: Middlebury College, and others
• Seamus Heaney: Berkeley, Queens Belfast, and Carysfort College (a teacher-training college)
• AE Housman: UCL and Cambridge (Latin)
• Randall Jarrell: University of Texas at Austin, Sarah Lawrence, University of North Carolina
• Louis MacNeice: Birmingham (UK) (classics), Bedford (London) (Greek)
• Edwin Morgan: University of Glasgow (literature)
• Paul Muldoon: Oxford (poetry), St Andrews, Princeton, and others.
• Sean O'Brien: Sheffield Hallam (poetry), Newcastle (creative writing)
• Charles Olson: Black Mountain College (literature)
• Don Paterson: St Andrews (English)
• Bob Perelman: Penn (English)
• JH Prynne: Cambridge (English literature and poetry)
• Henry Reed: U of Washington
• Adrienne Rich: Rutgers, Stanford, and others
• Theodore Roethke: Michigan State, U of Washington, and others
• Delmore Schwartz: Princeton and others (creative writing)
• Stephen Spender: U of Cincinnati, Gresham College, UCL (English), and others
• George Szirtes: UEA, Norwich School of Art and Design (creative writing)
• Allen Tate: Princeton (creative writing) and others
• Derek Walcott: Boston U (poetry and drama), University of Alberta (creative writing)

And Philip Larkin was a librarian at Hull University (by most accounts a good one and an efficient organiser).

---

Many poets made a living from a career in writing - not writing poetry, which does not make many people rich, but in other forms of wordsmithery, such as fiction. Cecil Day Lewis, the son of a clergyman, wrote detective novels as Nicholas Blake. John Masefield wrote numerous novels, including children's classic The Box of Delights, and military history. Thomas Hardy had become a famous novelist by the time he turned to poetry. Robert Graves wrote the very popular WWI memoir Good-bye to All That, the novel I, Claudius, and the philosophical book The White Goddess. Siegfried Sassoon wrote most of his poetry while in the army, but had his principal successes with prose such as Memoirs of a Fox-Hunting Man. Maya Angelou is also known for her autobiographical fiction and lectures. Robert Penn Warren is best known for the novel All The King's Men, which has been filmed twice. Carl Sandburg wrote children's stories and much other prose. Langston Hughes wrote in almost every form: fiction, a screenplay, books for children, autobiography, plays, opera libretti, essays, and translations. Although poor for much of his life, Charles Bukowski wrote cult novels, some of which were eventually filmed. Hugh MacDiarmid wrote prose, translations from Scottish Gaelic, and journalism. George Szirtes works as a translator and publisher as well as teaching and writing.

Ian Hamilton worked as a journalist and critic and wrote books on football and poetry. John Betjeman, self-styled "poet and hack" wrote journalism, often on architecture or other aspects of England, and presented programs for the BBC. Christopher Logue worked as a journalist, playwright, screenwriter, and actor. Alfred Austin was a journalist and editor, and wrote prose and plays. John Ashbery worked as journalist, art critic, and translator. Hugo Williams has worked as a film, theatre, and music critic and poetry editor for various magazines. Henry Reed, a man known for one great poem, The Naming of Parts, wrote a lot of radio drama before getting a teaching job. Louis MacNeice worked for the BBC as a reporter and dramatist. Stephen Spender and Randall Jarrell also worked extensively in literary journalism, editing, and criticism.

In addition to those working for magazines or small presses, poets including Cecil Day Lewis, TS Eliot, and Lawrence Ferlinghetti have worked more seriously in publishing. At Faber and Faber, Eliot published many of the finest poets of the 20th century. Stevie Smith worked as a secretary at the firm of magazine publisher George Newnes. Michael Longley worked for the Arts Council of Northern Ireland for many years. Other poets such as WB Yeats and Derek Walcott have been active at running theatres.

Ted Hughes seems to have made a living at least partly from poetry, although he wrote classic children's book The Iron Man and non-fiction. Allen Ginsberg lived largely off his poetry, readings, etc, although later in his life he taught English at Brooklyn College. Gregory Corso, a petty criminal in his youth, later lived off the legend of the Beats with readings and occasional teaching jobs. The Australian poet Les Murray retired to write full-time in his 30s.

Of course there are other ways you can get money. TS Eliot's estate receives a lot of money from the musical Cats. Ian Hamilton Finlay was a successful artist, working with numerous collaborators on many commissions. Robert Frost had a farm, though he also taught English and creative writing for most of his life. Langston Hughes worked for a while as a hotel busboy to have time to write, while Charles Bukowski worked for the US postal service. Dylan Thomas worked for the BBC but his main life skill seems to have been a talent for borrowing money, and he's not the only poet to have lived from hand to mouth.

And spare a thought for those like Edward Thomas, Wilfred Owen, Rupert Brooke, Alun Lewis, Keith Douglas, and Sylvia Plath who didn't live long enough to have a chance as professional poets.

---

For a concrete proof of the central place of academia in poetry, look at winners of the TS Eliot Prize, reckoned Britain's most prestigious poetry award:

• 2008 - Jen Hadfield, Nigh-No-Place (main income unknown, but sells books and visual artworks, lives in Shetland)
• 2007 - Sean O'Brien, The Drowned Book (worked Newcastle University)
• 2006 - Seamus Heaney, District and Circle (formerly Carysfort College etc)
• 2005 - Carol Ann Duffy, Rapture (worked Manchester Metropolitan U)
• 2004 - George Szirtes, Reel (worked Norwich School of Art and Design and UEA)
• 2003 - Don Paterson, Landing Light (worked St Andrews)
• 2002 - Alice Oswald, Dart (unknown - lives with husband and children in Devon)
• 2001 - Anne Carson, The Beauty of the Husband (worked McGill - reticent about life)
• 2000 - Michael Longley, The Weather in Japan (Combined Arts Director, Arts Council of Northern Ireland)
• 1999 - Hugo Williams, Billy's Rain (critic and editor)
• 1998 - Ted Hughes, Birthday Letters (poetry and prose)
• 1997 - Don Paterson, God's Gift to Women (worked St Andrews, as above)
• 1996 - Les Murray, Subhuman Redneck Poems (writes full-time)
• 1995 - Mark Doty, My Alexandria (worked Rutgers)
• 1994 - Paul Muldoon, The Annals of Chile (worked Princeton)
• 1993 - Ciaran Carson, First Language: Poems (worked Queen's Belfast)

Of 15 winners, 9 teach or have retired from teaching, 1 worked in arts administration, 1 works in journalism, 2 are of unknown income, and 2 may have lived largely from poetry (Hughes and Murray).

This is not the place to delineate the tightly-knit web of modern poetry (Private Eye has chronicled the too-close relations between poetry judges and poetry prize-winners, for instance). Nor will I speculate on whether this narrowness and inwardness is a weakness. This is a matter for further inquiry, which should probably focus on both financial questions and the comparison of career paths in poetry and academia.

---

I won't list individual sources here, but references included Ian Hamilton's Against Oblivion: some lives of the 20th century poets; Wikipedia; various university websites; www.poetryarchive.org; www.kirjasto.sci.fi; The Guardian and other newspapers. I wasn't able to come up with a clear picture on the earnings of some poets, including Robert Lowell, Alfred Lord Tennyson, Robert Browning, Marianne Moore, Andrew Motion, EE Cummings. As I said, work in progress.

Featured Artist Wars

The last few days have seen another battle in the war over filesharing and the future of the music industry. One one side is a group of recording artists including Lily Allen, Elton John and James Blunt, and on the other side a group of recording artists including members of Radiohead and Pink Floyd as well as Allen's rival in estuary pop Kate Nash and long-time activist Billy Bragg (Guardian). Trade body UK Music and business secretary Peter Mandelson want a three-strikes-and-you-lose-your-internet policy for people breaching copyright online (Guardian; The Register). The Featured Artists Collection (FAC), whose directors include Bragg, Nash, Nick Mason (Pink Floyd), and Ed O'Brien (Radiohead), believe this is too severe a punishment, although they oppose copyright violation.

Lily Allen set up a blog called It's Not Alright (which some critics reckoned was a pun on her first album Alright, Still) to promote UK Music's point of view, which received support from people such as rapper Tinchy Stryder to criticise the Featured Artists Coalition. "They do sell-out arena tours and have the biggest Ferrari collections in the world", Allen complained (alluding to Nick Mason's hobby) while Stryder lambasted the FAC: "These guys have probably never even been on a computer/iPod/mobile phone LET ALONE the internet LOL, so I don't know how they're so sure!!" (BBC) Bragg had already led a campaign to change MySpace's terms of service which he felt were ripping off artists (New York Times) but Allen and others were newer to this kind of thing, and she shut her blog down a few days later. She announced by Twitter that she was giving up the fight amid accusations she had illegally distributed copyright material online (Guardian; TechDirt). The next stage will be a summit between UK Music and the FAC to try and find a common position.

---

The FAC is an organisation of British "featured artists", those people whose names appear on record covers (as distinct from session players, etc). As well as high-profile members of Blur, Radiohead, Travis, and the Futureheads, its committee includes members of smaller bands like The Young Punx and the Fire Escapes and rapper Master Shortie. The organisation was founded with the belief that "music artists should control their destiny" (FAC website), a theme raised in the past by the likes of Prince and George Michael. They campaign for artists to have greater rights over record companies, including giving artists ownership of recordings and improving the contracts recording artists sign. They also recognise the disruptive nature of the internet, MP3s, filesharing, etc, on music revenues, and seem to be pursuing a policy of enlightened self-interest to maximise their income in the face of technological changes.

CD sales have been falling in recent years; album sales including downloads dropped 3.2% from 2007 to 2008 (Guardian). On the other hand, due to downloads singles sales are booming (Guardian) - though a 79p single download bring less profit than a £10 album.

There are many factors involved - music is now available for free (with advertisements) from Spotify and YouTube. There are far more radio stations than 20 years ago, although not always with a corresponding increase in variety. Illegal downloading may play a part in falling album sales, but some studies have found it has no effect or even increases sales (Washington Post; Industry Canada); they allow people to preview music which they may still purchase. Legal downloads mean that people can buy their favourite tracks rather than purchase the whole album as well as allowing them to try a couple of tracks before they buy the whole album. More power for consumers often means lower profits for business - although this isn't necessarily the case if consumer power gives them more confidence to buy.

Some musicians have tackled downloading by making CDs more desirable, with de luxe packaging, booklets, DVDs, MP3s, and other gimmicks. Both Pink Floyd and Radiohead have tried that in the past, but the most famous example is Josh Freeze who offered a range of deals for buyers of his album; for $50 they got a thank-you call, and other packages included miniature golf, having a song written about you, and hanging out with him for a week (only $20,000). This is probably not something every musician should do.

Recent events such as the takeover of EMI by private equity group Terra Firma (BBC) have placed the industry under financial pressures from investors at the same time as revenues are under threat from the internet. This is already leading to cost-cutting at EMI. It may be that we will see a lasting change in recording industry practices, which traditionally worked on a slightly bizarre model where new artists get large advances on signing but must pay for their own recording fees, recouped by the label retaining royalties (see Wikipedia; in contrast, film directors or actors are typically paid a fee plus residual payments and sometimes a share of profits, and do not have to pay the cost of film production, but benefit less if the film is an unexpected success). The FAC wants a better deal for artists; record labels would doubtless like to reduce spending; Lily Allen argues that record company investment is necessary for developing talents.

The result of all this is emotive language: while home taping may not have "killed" music in the 80s, downloaders are called "pirates" (ironically in view of the rise in real piracy on the ocean) and "thieves", although downloading is legally different to stealing. Even the term "illegal downloading" may be an overstatement: small-scale downloading or copying for private use is not a criminal offence; only if it is commercial or on a large enough scale to affect the owner of the copyright is it a criminal offence under the Copyright, Designs and Patents Act 1988, which applies in Scotland and England. Otherwise it is a civil matter, like other infringements of property rights, and the most the copier will face under English law is to surrender the material, pay compensation for quantifiable financial losses, and pay the costs of the suit.

---

How artists and record labels should be compensated in the event of declining music sales is a complex issue. In the past, musicians have had to deal with the introduction of recording and the early recording industry was unnerved by the growth of radio, both of which threatened business models and earnings; more recently we have seen singles become by the 1990s loss-making advertising for an album rather than sources of profit, and the internet offering a way of distributing music at almost no cost. One way out is for artists to not rely on major record labels but to self-distribute or partner with independent labels; this is probably more viable if you are already famous or willing to spend years building a fanbase. The internet circumvents the necessity for shops and distributing physical objects; you can sell worldwide with minimal overheads - other than the cost of recording the music in the first place.

Only a small fraction of artists actually make a living from record sales, and the changes proposed by Lily Allen will not affect that. Most musicians must rely multiple sources of income. Ticket sales are reckoned a panacea by some, but not all artists choose to perform or are able to, due e.g. to their reliance on studio techniques or large numbers of musicians, or family commitments, and smaller bands may make little money. Other sources of income are fees for radio plays, songwriting royalties, revenue from services such as Spotify which provide small fees for each play of a track, and licencing music to advertising and films. Many musicians will object to advertising (even if they are asked) and other methods bring in small sums of money: while a musician may get 10-15% of a 79p iTunes download (Guardian), comparable to the percentage for a CD royalty, payments from a Spotify are tiny.

Some people have questioned Spotify's business plan: in August 2009 with half a million British users it had UK "advertising income of £82,000 and just 17,000 UK users signing up to pay £120 a year for Spotify Premium" (The Guardian); The Register suggested in June that it receives income of 14p per user per month. Nobody wants to pay for the premium service, and the advertising that non-premium users hear brings in negligible income (advertisers are limited to a few record labels - who as they part-own Spotify may get cheap rates - and some public-sector bodies).

In the past a number of governments imposed a tax on blank tapes to pay recording artists and record labels for revenue allegedly lost due to home taping. In the digital age there have been similar plans suggested. These range from a tax on MP3 players or the internet, to the currently popular ideas of licencing file-sharing so that for a monthly payment people can share all files - this mirrors the current licences for public performance of music. ISP PlayLouder offered this facility without a large takeup; more recently Virgin Media has reportedly considered and rejected it (Guardian).

Also open to musicians are the traditional methods of income for practitioners of other arts: teaching, working second jobs, relying on parental or spousal wealth, etc. In some art forms such as poetry and literary fiction there is little money to make; most well-known poets are simultaneously employed as academics, many young artists teach, while many authors and playwrights make ends meet with reviewing, journalism, teaching creative writing, and other sources of income. Such secondary jobs have advantages in keeping the artist in contact with the world, but can pose a significant drain on time and energy, and for musicians would interfere with touring.

---

Encouraging people to buy music is another tactic to increase sales. In the 1980s the music industry claimed "home taping is killing music". Recently the motion picture industry has tried various advertising strategies to encourage people into cinemas and away from pirated and illegally downloaded movies. These advertisements ranged from the "Knock-off Nigel" theme in which buyers of pirated movies are ridiculed as skinflints and meanies, to a series based on blockbusters like The Lion The Witch And The Wardrobe and Fantastic Four which compared the experience of cinema-going with big screens and booming sound systems to the experience of watching bad-quality pirate movies, to the most recent series in which celebrities such as actor Martin Freeman thank audiences for supporting the British Film Industry.

Billy Bragg has suggested that the music industry wants ISPs to take action against infringers rather than pursuing them through court (which is the remedy mandated under existing copyright law) because the music industry does not want to seem the bad guys - when the RIAA in the USA started pursuing filesharers through the courts there was a massive reaction, much of it negative. The British film industry has been trying to show itself in a good light. The record industry may have a similar goal, but seems to have no idea about PR.

Already many people in cult or up-and-coming bands have to work to support their music. It's not as bad as in poetry where (except for the independently-wealthy Frederick Seidel who writes verse about expensive motorcycles - much as many wealthy rock stars do) nobody is rich, or in classical music where state funding and corporate sponsorship are the norm. Whether we want state funding of pop music is another question (arguably Britain had it in the 1970s and 1980s thanks to generous unemployment benefits and subsidised student unions).

Most people enjoy music, and most people seem willing to pay for it in some way or another, whether by CD purchase, download, live, listening to commercial radio, some other even more esoteric delivery mechanism, or just by spending hours hunting for illegal downloads. While the FAC are seriously asking how recording artists can maximise their incomes, the one question nobody seems to be asking is what method is best for the consumer.

Impartial opinion

As Britain approaches a general election in early 2010, thoughts turn to opinion polls. Prominent market research organisation YouGov was founded by Stephan Shakespeare and Nadhim Zahawi. Shakespeare, now their Chief Innovation Officer, also owns the ConservativeHome website (formerly 18 Doughty Street), was Conservative parliamentary candidate for Colchester in the 1997 general election, and worked as Jeffrey Archer's spokesman in the late 1990s. Nadhim Zahawi, the CEO, was a Conservative councillor in Wandsworth, a Conservative parliamentary candidate, and campaign manager for Archer's bid to be mayor of London.

YouGov is not the only polling organisation with political connections. ICM chairman Nick Sparrow was private pollster to the Conservative Party Central Office from 1995 to 2004. Ipsos Mori life president Robert Worcester was knighted by Tony Blair in 2004. This does not mean that opinion poll results of any of these organisations are biased, and it doesn't look like being a hard election to call, but it will be interesting to track differences between pollsters as the election approaches.

EDIT: In April 2010, YouGov were caught out doing a biased poll on one of the election leaders' debate, in which they timed the survey so people would have to give their opinion having watched David Cameron but not Nick Clegg. This was seen as part of a wider conspiracy by the Tories and right-wing media against Clegg (see e.g. Guardian; Iain Dale).