Wendy Harmer recently had an encounter in a cocktail bar with a big swinging dick executive in Sydney mainstream media that to her summed up the times. Since July 2011, Harmer has been stewarding a website, The Hoopla, targeted at mature women. Wry and warm, it has won a loyal following; run mainly on zeal, it has made little money.

“The Hoopla?” said the executive languidly. “Yes, I’ve heard of it. I suppose I should make you an offer to buy it.” Harmer assured him she wasn’t a seller. The next day, the executive was fired.

All of a sudden, the news media is its own breaking story. Once-great institutions are under threat; what might supplant them is unclear; reputations afford no protection. Think of it as the fate of many a protected industry. Journalists like to think of themselves as rugged individualists, but they have always been cosseted by high barriers to entry, fat margins and privileged access.

The effect of the internet has been as profound as it would have been on any oligopoly suddenly confronted by overpowering quantities of high quality, real-time content priced at zero. It has shivered markets to fragments, atomised and energised audiences, and gnawed business models away at each end, impacting newspapers’ abilities both to attract advertising and to charge for news, winnowing television audiences away and eroding the value of free-to-air broadcast spectrum. The proportion of Australians who read a daily newspaper, and who watch an evening news, are roughly half those of 30 years ago.

Social media is revolutionising traditional media’s forms and practices: Twitter allows users to curate their own news streams; Facebook “likes” drive a growing proportion of online traffic; commentary on blogs, usually written for free, is regularly at least as expert if not more than what appears in newspapers. Locked in an awkwardly intimate embrace with an audience they have been apt to take for granted, journalists face a degree of scrutiny they used to reserve for others, and of competition they used to think was a great idea for everyone else. The government is set in four weeks to respond to the fruits of two substantial inquiries that presented findings in the past six months: Justice Ray Finkelstein’s report of the inquiry into media and media regulation and Glen Boreham’s Convergence Review.

This survey by Crikey, however — a multi-part weekly series beginning today — will be more in the nature of a preview: an exploration of the range of possible futures, based on discussions in the present with participants and observers from an industry in the grip of radical change.

Worldwide change, too. In some ways, we may not have seen the worst of it. English newspaper circulations have declined nearly a third over the past decade; American newsroom populations have fallen a third from their peak, and cities as big as New Orleans left without a daily newspaper. Not even The Newsroom, Aaron Sorkin’s long-awaited American cable news melodrama, is rating. Only in regions where internet penetration is low, such as Africa and India, are news institutions stable or growing. Otherwise, news media has an unprecedented and unwanted distinction, observes The Australian’s sagacious economics columnist George Megalogenis: “First agriculture then manufacturing both declined as a share of GDP in every Western country, but services always grew. Media is now the first services business to go backwards.”

Australian media organisations have done better than most in courting the online eyeball. Fairfax, News Corporation, the Nine Network (through ninemsn) and the ABC still own the sites most visited. But the business models of the two biggest newspaper groups have been devastated. There is every chance that Fairfax’s Sydney Morning Herald and The Age will cease weekday print publication in the next two years; News Corporation is set to hive off its newspapers, including The Australian and top-selling tabloids the Herald Sun and The Daily Telegraph, into a separate listed vehicle to placate restive investors. Even communications minister Stephen Conroy has joined the deathwatch: “In terms of the future of print, I’m pessimistic. In terms of the future of news reporting, I’m very pessimistic.”

“In a time when media can no longer rely on the big budgets of the past, we should be doing really well. And the last barrier has been removed: in the digital world, we’re all equal.”

The Media, Entertainment & Arts Alliance estimates that more than 1000 jobs have been lost in journalism in the past three years; by this time next year, head counts will probably have been reduced by a further 3000, although some investors insist that that’s not enough. “The place is run by journalists who are good at some things but are not always very commercial,” complains Simon Marais, whose funds management group Allan Gray is the second-largest shareholder in Fairfax. “We have struggled to get the idea across that you can have editorial independence but you must also make money. If you don’t make money you’re gone. Not only must you make money you must also maximise your profits. That’s what you’re there for. You’re not there to provide great quality journalism.”

Marais becomes particularly animated when he discusses The Australian Financial Review: “The Financial Review is twice as thick as the Financial Times. How can that be? We [Australia] have a smaller readership and a much smaller market. I’m the target audience and I don’t read it; I don’t have time. Which suggests to me that there’s twice as many journalists as they need. They should cut the number in half. There’s not enough news to go round. Cut your costs and cut your paper in half, you’ll sell just as many copies and make more money.”

For journalists, it’s a daunting time. Industry jargon used to extend little further than “splash”, “spill”, “em” and “en”; nowadays, it’s all CPMS, CTRS, page views and platforms. Once there were only suppositions and superstitions about consumer preferences, now there are growingly precise analytical tools to measure traffic, from where they are coming, for how long they are staying, what they are dwelling on, and to where they head next. In the newsroom of AFL.com.au — the brand-new news service of the Australian Football League at Docklands that has quietly become Australian sport’s digital market leader and already employs more journalists than Channel Ten — an electronic scoreboard refreshed every 15 minutes charts unique browser numbers over the preceding 24 hours for individual stories. “It’s very new school,” agrees head of content Matthew Pinkney. “As a journalist, I’m guided by my intuition; I’m wary of statistics. But the management here, who are very smart business people, are very stats-driven and want to make editorial decisions based on metrics. That’s a reality today.”

THE RACE FOR ONLINE READERS

Not surprisingly, speed is in fashion. “The way you respond to a story has huge implications,” says Hal Crawford, head of news at ninemsn. “If smh.com.au beats us to a breaking news story by two minutes, that can have a huge impact on traffic figures for that month. If you’re always slower, the audience will go elsewhere. We don’t have a metric yet to measure response times accurately but the difference is palpable. A lot of people hear about stuff first of all from the radio. They then go online to verify the news, which they’ve only half-heard. So if ABC Radio is running something about a guy who got bitten in half by a shark in Perth, you’d better have it on your site. Doesn’t matter how small it is; it has to be there somewhere.”

Twitter has emerged as a tremendous source — and also, in its way, a potent rival. “I was sitting in my living room on Racecourse Road the other night and nine fire engines went past,” says Margaret Simons, director of Melbourne University’s Centre for Advanced Journalism. “I went on all the mainstream media sites but couldn’t find anything; the first accurate news report by someone living near the Racecourse Hotel, which was on fire, was posted on Twitter with a twitpic 20 minutes before the Herald Sun had anything. I checked the man’s Twitter feed, and he clearly was not a journalist. But it was an act of journalism.” Examples are no longer so prosaic either: events from the death of Osama bin Laden to the suspension of Collingwood’s Dane Swan have been heralded first on social media; Twitter users in May 2011 revealed an extramarital affair that footballer Ryan Giggs had tried to keep secret by simply flouting a court gagging order that restrained traditional media. “The business model for breaking news,” says Conroy succinctly, “is f-cked.”Boundaries between print, radio, television and online have never been more porous, with the ABC a fount of written online opinion at The Drum, The Australian producing Australian Agenda for Sky News, and smh.com.au screening last night’s MasterChef, so versatility is also de rigueur. “In days gone by, you would have been a TV journalist, a radio journalist or a newspaper journalist,” says Leigh Sales, host of 7.30. “In the future you’re simply going to be a journalist. Instead of being sent to a court story to do one version of the news, you’ll be told to go down there, ring the office and find out who you’re filing to. You’ll be told: ‘That’s not good enough for TV, but give us three radio stories, and 10 pars for online.’ If you’re a great writer but not a great oral communicator you’re going to have to learn, and vice versa.” That, says Sales, will further intensify rivalries: “At the ABC we don’t have much competition in that pointy-headed end of TV current affairs journalism. But once news websites begin offering video and doing interviews we will get more competition there. In the same way as the question will be asked whether I can write 900 words of political analysis, the question will be whether Michelle Grattan can interview a politician in a studio?”

In some ways, skilful practitioners might in the long run be better off. “We’ve actually come a long way from what publishers used to be happy with, which was drunks who could write and talk to people,” observes Brant Houston, a Pulitzer Prize-winning reporter at the Kansas City Star and Hartford Courant before becoming a professor at the University of Missouri School of Journalism. “The good news is that nobody has a stranglehold on the news or the investigations you can do now, or the medium in which you have to do them. You still have people in the US, particularly in the academy, talking about print and broadcast. But if you hold up your iPhone, you iPad or your ‘droid [Android device], you realise that our audience and our young journalists don’t see the difference that you’re talking about. They use the best medium possible for the story — text, audio, video, they’re all one. They’re the facts of life.”

The commercial imperatives of digital media might even foster a renaissance in excellent, in-depth writing at length. “One thing I’m encouraged by is that there had been an expectation that good writing would be less valuable, but it hasn’t turned out to be the case,” says Alan Kohler, former editor of The Age and The AFR whose Australian Independent Business Media has for the past five years pioneered online business journalism at the Eureka Report. “If anything, good writing that draws you through is more valuable because you can now measure time on site. Because it is measured, it is valued, and can be understood by advertisers. The way you get people to stay with a story is by writing well, by sprinkling nuggets of gold throughout a piece to keep readers going. Space is getting hard to find in newspapers, but online there’s as much space as you want. If you can get people to read a long piece to the end, you’re extremely valuable. You are worth hanging onto.”

That may even preserve a niche for print. While it is being surpassed as a deliverer of news, it still, to paraphrase the old beer ad, refreshes parts others do not. “If we didn’t have print, we would have to invent it,” argues Wilson da Silva, editor of the successful science magazine Cosmos. “Requires no batteries, only the application of light. Three hundred dpi resolution? You just can’t get that in the digital world. There’s still no better way to present journalism in longer form.” Although pessimistic about newspapers in general, Kohler has unsentimental regard for print: “The idea of finding out what happened in the world in a newspaper is ridiculous. But the act of reading on paper has a lot to recommend it. It’s good for the eyes. It’s pleasing to the hands. It’s not entirely replaceable by anything.”

Nonetheless, if you wanted to condense the future of news media to a single word it would probably be: poorer. Quite simply, digital readers and digital advertisers are not as lucrative as their print counterparts, and won’t be for some time, if at all. “The future for the mass media is certainly smaller and less profitable — fewer journalists doing more and being paid less, by and large,” says Simons. “Quite a lot of journalism will be done for not very much as well, and already people are doing a great deal for free, especially opinion. The upside is that it will be owned by people who care for media for its own sake; the downside is that you will never again have that accretion of economic power which allows journalists to take on a government.”

Now that paywalls have been introduced at News Corporation’s titles and foreshadowed at Fairfax’s, Simons adds, the issue is shifting from the hardy perennial of what people want to read to what they can be persuaded to pay for: “We’ve acculturated people to expect mass media news for free and I doubt if people will ever pay for that. It will need to be supported other ways. We do know that they will pay for business news, like the Eureka Report; we do know they will pay for gossip, Crikey suggests that; we do know they will pay for a better access to sports coverage, pay television proves that; I suspect that people will pay for high quality, intensely relevant content; but it will have to be special, and not readily obtainable elsewhere.”

WE DON’T TRUST THEM, THEY’RE UNHAPPY

Yet how well equipped is the traditional Australian media to provide content of high quality and intense relevance? There is room for doubt. Levels of public trust in mastheads and current affairs programs have never been lower; confidence in the trade itself is poor. Journalism at the Speed of Bytes, a recent MEAA survey of 100 senior editorial staff from Australia’s main metropolitan and national newspapers, presented a melancholy industry self-portrait. Sixty-two per cent of respondents believed the quality of newspaper journalism in Australia to be “average” or “poor”; 66% said the same of online journalism. The necessities to do more with less have depleted reporting ranks and cultures, entrenching the genres of “churnalism” and “newszak” — stories originating in press releases or provided by publicists then festooned with clichés and quotes.

Newsroom management is shambolic, and anything but meritocratic. “The backbone of newsrooms are overworked, underpaid people between 25 and 35 who are run ragged, and frankly burned out,” says Bill Birnbauer, a veteran newshound of The Age, now a senior lecturer in journalism at Monash University. “Then you have this layer of stars, many of whom started at the same time as their editors, are mates with their editors, and are looked after. The performance reviews we have are a joke. I can’t recall a journalist ever being told they needed to lift their game in a particular round. I sacked a guy who was working in my cluster at The Age when I covered the health round. I didn’t like what he was producing and I told him I didn’t want him. It was very hard. In the newsroom he had a lot of friends, and people thought I was an arsehole … People in other businesses have KPIs, quotas, incentives to produce. Journalism has a lot more free riders.”

Chris Masters, probably Australia’s most distinguished investigative journalist, is politely scathing about the standards he observed during a recent stint at Sydney’s Daily Telegraph, and scarcely less critical of the ABC where he spent much of the previous 40 years. “I admired the industry of the people at The Telegraph,” he says. “They were on the phone all day. What I didn’t admire was that they hardly ever left the office, and that they weren’t so much looking for stories as for headlines and for quotes. It’s an irony of having access to so much information online that you often do weaker, more superficial research.

“The Sydney Morning Herald is what they think people should read; smh.com.au is what they think people do read.”

“At the ABC, though, they don’t even seem to be on the phones. They’re more like event managers or producers — the stuff they get a pat on the back for is not breaking a good story but for getting a premier to say something silly … Newspapers are having a go at the ABC for not breaking stories and they’re pretty right about that. Reputable journalists at the ABC know that that’s an issue. Four Corners doesn’t get everyone else off the hook, and News 24 has done to the ABC what’s happened across the industry: it broadens the platform without deepening it. Good journalism isn’t practised in an office or a studio, but it’s getting to the point where young journalists think that is what the job is.”

Nor does the responsiveness and innovation that is regarded as mandatory at the moment come naturally to institutions with long traditions of top-down control. Historically, entrepreneurs such as Kohler, Harmer, Mia Freedman at Mamamia, Tim Burrowes at mUmBRELLA and Stephen Mayne and Eric Beecher of this parish have been atypical in the Australian media. One of the designers of Fairfax’s new digital strategy, Peter Gearin, agrees the challenge is as much cultural as economic. “When we changed the structure, it was basically to disperse the power, push it back to the journalists,” he says. “It shouldn’t be about a person sitting at the top of the pyramid trying to make all the decisions. It’s saying that there’s a whole lot of people who need to take responsibility for what they do, including the journalists. It’s about the journalist being more entrepreneurial, seeing an opportunity, and saying that you’ll pursue it because you think it will have an impact. We want journalists who are willing to have a go rather than always deferring to a greater power. That’s a huge part of the culture change ahead of us, which is going to take a fair bit of time to happen.”

Fairfax’s digital strategy, announced in June, involves the restructuring of its metropolitan dailies as local mastheads served by an essentially national newsroom. The company has a considerable digital heritage; its websites, theage.com.au and smh.com.au, date to the mid-1990s. But, run as adjuncts rather than as core businesses, these have created as many problems as they have addressed, developing identities quite separate to the newspapers they were designed to parallel. As veteran media analyst Peter Cox puts it: “The Sydney Morning Herald is what they think people should read; smh.com.au is what they think people do read.”

People have read them, to be sure: Fairfax claims a 30% growth in readership of its print and online editions in the past five years, augmented in the last year by an iPad app. But the sites have hardly fostered a deep identification with the company’s print properties. Fairfax’s sites are renowned for what is sometimes called “clickbait”: headlines written to beguile passing eyeballs that obscure nondescript or irrelevant stories. “Footballer r-pes woman in bar,” jokes Annabel Crabb, the ABC’s chief online political writer. “You think: goodness me, who’s that? It turns out to be someone playing fourth division soccer in the East Midlands. You confuse readers when your online offering is so different to your print offering, but once it works for you in terms of eyeballs, it’s a very difficult habit to kick.”

As for monetising it, now that Fairfax has mooted paywalls, she agrees: “You can’t.” With their chaotic mess of links, annoying autoplay videos (which you can now opt-out of after much criticism) and in-your-face advertising, being free may be about all the Fairfax websites now have going for them. And while traffic is useful to point to when it comes to touting for digital advertising, advertisers armed with analytics are increasingly discriminating. “[Clickbait] is rebounding on them [Fairfax] because the advertisers can measure the amount of time spent on each piece,” observes Kohler. “If people are spending five or 10 minutes, that’s valuable. But if it’s 15 seconds and then people bounce somewhere else, that’s no use to an advertiser. It’s been hugely counterproductive because … you cannot sell a 15-second view of an article. To get your five cents or whatever, you have to get the reader to stick around.” Conjecture that Fairfax intends making its metro dailies resemble its websites more closely strikes David Marr, recently departed from The SMH, as potentially suicidal: “Well, that would be f-cking idiotic wouldn’t it? Newspapers have to go upmarket or they will die.”

Not every day do you hear a sentiment from Marr that you could also imagine emanating from Chris Mitchell, editor-in-chief of The Australian — but this is one. News Corporation has taken a different route, including two months ago buying AIBM from Kohler and his equity partners (including Beecher, publisher of Crikey). Most obviously, it has identified strongly with centre-right values both in print and online, and campaigned vociferously on pet causes. “We have a pretty clear idea of where we sit on the political spectrum and why,” says Mitchell. “Some people would say that’s who Chris is; others would say that’s because he’s defined a territory and no one else has. I would argue very much the second, and I think it’s worked well for us.” Since its paywall was erected, The Oz has accumulated 65,000 digital subscribers, which includes a rolling number of one-month trialists but also 43,000 who have been paying for more than six months.

At The Australian, too, the website and app strongly reflect the newspaper, because the latter has always been in control. “One of the tricks is that the newspaper must control your online presence,” says Mitchell. “The big problem all along at Fairfax has been the separation of the two … We feel that ours is quite an unusual demographic — and one of the fascinating things looking at how the paywall works is that our average subscriber is spending more than two hours a week on the website and is happy to pay for it.

“The other thing we’ve found is that the new readers we’ve picked up weren’t newspaper readers. Two big demographic bubbles we’ve picked up, for example, are Southbank in Melbourne and Pyrmont in Sydney, which suggests younger, reasonably well-off professionals in high-rise buildings who haven’t had ready access to papers; in fact our print figures there are terrible. Once again I’m happy with that. We’re stretching into an area we haven’t gone to before.” Relative to its competition, then, The Australian may not be badly positioned. Some read it to be informed, some to be infuriated; but read it they do. “For a traditional media outlet to survive now, they have to appeal to their supporter base,” says Peter Cox. “It’s arguable that Fairfax has not done so. And that News has.”

Mind you, Mitchell senses the future is cloudier for his News Corporation tabloid stablemates, Melbourne’s Herald Sun, Brisbane’s Courier-Mail and Adelaide’s The Advertiser, working towards charging for content, but after their newsrooms have been stretched thin by austerity regimes and expanding digital obligations. “It’s hard to be in the middle,” Mitchell observes. “It is very hard to be a city-based newspaper trying to be all things to all people. The rise of things like AFL.com and NRL.com make sport less of the glue than it used to be. In Melbourne when the old Sun News-Pictorial was selling 700,000, a lot of it was based on AFL. Just as you see a lot of material in common between The SMH and The Age, you also see a lot in the Courier that is in the Herald Sun. I think that’s tricky: people will pay for something very deep and very tightly focused in their local community, but will they pay for a replicated national service that they could get on the ABC, at ninemsn or Yahoo News?” There remains, too, News Corporation’s abiding challenge: that such stability as it enjoys is invested mainly in the longevity of an octogenarian proprietor.

‘TOO MUCH IS MOVING TOO FAST’

Yet perhaps for the first time in half a century, the most energetic competition is not between the great media franchises themselves but between old and new, between the sophisticated all-you-can-eat banquets of news, current affairs, commentary and lifestyle to which we have grown accustomed, and what look more like cafes and takeaway outlets with house specialities. “There aren’t any rivers of gold to be had,” says Monica Attard, founding editor of one of the ritzier outlets, the long-form current affairs website The Global Mail. “There’s a whole lot of players doing different things, none of them making much money, most of them regularly changing identity. I don’t think there’s such a thing now as a website that stays the same for more than six months. Too much is moving too fast, and the audience is now so fragmented and fickle that people move on very quickly to the next big thing, and the fanciest tricks win out. Obviously there will be winners, but so far nobody has come close.” That includes The Global Mail, which Attard left in May.

Walk around the pristine offices of AFL.com.au and it’s hard to believe that news media faces any problems at all. The equipment is state-of-the-art, the newsroom hums with quiet industry, the traffic is through the roof. Matthew Pinkney’s last job was as digital editor at the Herald Sun, helping to set up its paywall — a strategy, ironically, from which his new employer has benefited richly. Yet the contrast between working for Rupert Murdoch and working for Andrew Demetriou was not as great as Pinkney expected. “There are big strong personalities in both organisations,” notes Pinkney. “Commercial outcomes are as important here as they were at News. Perhaps naively, I thought that would not be the case, but it very much is. At the heart of both jobs, though, the task is to create content that people will like — thinking up things that if I wasn’t a journalist I would love to look at.”

At the newer end of town, there is also a disarming good cheer. “I’m much more optimistic than newspaper people because I’m in the digital environment already,” says ninemsn‘s young boss Hal Crawford. “And one thing that’s amazing about being on digital media and using these analytical tools is that your respect for the audience grows. I came here [from The West Australian and Radio Netherlands] with probably the mainstream media view that the mass audience in Australia would be a bit ignorant, maybe a bit racist. After six years I can say that, on the contrary, the audience impresses me. How is it that they can work out when a story isn’t fresh? But they do. The collective intelligence sniffs out truth and relevance.”

Find a better niche, and Wilson da Silva of Cosmos is convinced the world will beat a path to your door. “In Australia, we’ve always taken pride in being able to produce good things with finite resources,” he says. “So guess what? In a time when media can no longer rely on the big budgets of the past, we should be doing really well. And the last barrier has been removed: in the digital world, we’re all equal. Assuming your content is good and your cost base is low, you should be thriving … All these people in newspapers are running around saying it’s the end of journalism. But journalism’s doing well. Audiences have grown dramatically. You have a revenue problem, but not a readership problem.”

Cosmos‘ readership grew more than a third last year, because, for one thing, the mainstream media has historically covered science so poorly — something that continues, The Oz and The SMH having parted with respected science writers in the latest round of redundancies. Da Silva finds it hilarious: “The world is undergoing a technological revolution and the only people covering it in newspapers are a handful of science writers, while editors go on thinking that everyone is fascinated by the byzantine manoeuvrings of Labor factional politics.”

Harmer radiates a similar passion about The Hoopla, which she founded to provide reading and writing opportunities for her own neglected demographic. Digital media, she argues, depends more than anything on the ability to create a sense of community, of belonging. “It’s about the personality, the sense that someone’s home, that there’s a person behind it,” she says. “There’s nothing to keep you on a screen the way you feel the need to flick through a newspaper. But there’s an idea that there’s someone home — in the way that people like to think that Tina Brown is home when they visit The Daily Beast, or Martha Stewart is home when you visit her site.” Providing a facility for making comments, as newspapers do, no more engenders community than putting up a wall that attracts graffiti. “I write back to readers,” she explains. “If someone puts up a comment, I’m there responding, thanking them, or replying or disagreeing. You can’t imagine Graham Perkin doing that, but that’s what people really dig online.”

Whether the individuals behind niche ventures have the capital and staying power to survive in the long-term is another matter. Harmer foresees a period of consolidation: “What’s going to happen is that there are players in the market who are consolidating across a number of sites. They will try to find like-minded sites and partner up. It’s going to be like an aquarium. Big fish gobbling up little fish. Taiwanese trawlers with drift nets. Sting rays. There will be areas fished out, areas that are farmed.” In the meantime, Harmer agrees, there is only one certainty — that “no one knows what the f-ck is going on”. Which is kind of stimulating: “Eventually something will crawl out of the primordial soup, grow legs and walk on land. We don’t know what it is. But isn’t it fantastic to be in this soup?”

Related links: