If you follow stories about the internet, right now it’s all bots and fake news and dark posts and Russian hackers. But the real story hiding underneath is that digital advertising is being disrupted. It’s flatlining and cannot sustain journalism.

Demands for greater transparency, scepticism by advertisers themselves, community resistance to tracking — digital advertising is going sideways and those media organisations planning on riding an ever-rising curve find themselves caught by the change.

Last week, Facebook announced significant changes to its advertising policy in what it called “a new standard for transparency”, ostensibly in response to the ongoing controversy over its role in the US election. The platform was responding to concerns over so-called dark posts — messages that are only seen by the people actually targeted.

For example, during the US 2016 election, digital messaging was directed at the Haitian community in Florida (a key swing state) alleging (falsely) that The Clinton Foundation had defrauded millions of dollars out of relief monies after the 2010 earthquake.

Last week’s response by Facebook came hard on the heels of a report, published the week before, by New York-based not-for-profit ProPublica, which stated the platform enabled ads that targeted anti-Semites. Using the self-service ad-buying program, ProPublica was able to place ads (at a cost of US$30) to about 2300 people who expressed interest in the topics of “Jew hater,” “How to burn jews,” or “History of ‘why jews ruin the world’”.

After being alerted by ProPublica, Facebook adjusted their algorithm so that these hate speech terms would not emerge. Both CEO Mark Zuckerberg and COO Sheryl Sandberg expressed their genuine shock. But it was a striking demonstration of the challenges of programmatic advertising from both the buy and sell side.

[Mass media chase ad dollars but the rivers of gold have run dry]

While the story was embarrassing, of greater financial concern is that major advertisers are questioning the value of any programmatic advertising while activist groups are targeting advertisers whose ads surface on particular sites.

In July, the world’s largest advertisers — Proctor and Gamble, and Unilever — revealed they were slashing their advertising budgets, particularly for digital advertising. P&G said they’d cut digital spend by 40% and Unilever by 60%. This affected both volume and the sites they agreed to allow ads to appear. When advertisers of this size roll over, the world shakes.

There’s a reflection in these cuts in Australia. According to the Standard Media Index, digital ad spend this calendar year to date has gone sideways, up by a mere 1.7% on last year at $1.2 billion. New Zealand is down about 4.9%.

In all countries, most of this money is funnelled through the big platforms. About 90% of new spend goes to Facebook and Google.

Proctor and Gamble have been explicit about their thinking: they are sick of paying for traffic that wasn’t real, being driven instead by bots programmed to generate clicks to produce higher numbers of “viewers” and by clickbait and other objectionable content.

At one level, the internet exists for one reason only: to sell you, the user, to advertisers. It’s a big brother monitoring machine — but for advertisers, not governments. Programmatic advertising is that tool that links you to advertisers: algorithms place ads by monitoring your searches, working out your demographics and the sort of person you are, even, until recently, by reading your emails.

So, if you’re a Haitian living in Florida, sent an email about post-earthquake developments, you could then be targeted with a post (often from an unidentified source) about fraud in aid relief.

The big platforms are trying to civilise advertising on the web. Demands for greater transparency from the US Congress and P&G (and threat of regulation from the EU) was reflected in Facebook’s announcement last week. Political advertising, at least, will now need to be effectively authorised, just like ads on television, radio and print.

Advertisers will have to disclose any dark posts. That means, you can visit any advertisers’ page and see what messages they’ve sent to which segment of Facebook’s audience.

[Playing in traffic: the dirty tricks publishers use to boost online views]

The other tech giants are also responding to the push back. Both Apple’s browser Safari (default on iPhones) and Google Chrome have increased anti-tracking and ad-blocking capacity, although when ads come through sites you visit often (that is, Facebook, Google) it’s less effective. Google has also said it will stop tracking through Gmail.

Complicating life for both advertisers and advertising platforms alike, has been community resistance to advertising. We know now: people hate it. Ad-blocking software, now at about 25% of the mobile market has skewed young to advertisers’ favourite market.

Activist groups are also targeting advertisers directly. The virtual group Sleeping Giants messages companies whose ads surface on Stephen Bannon’s Breitbart demanding they blacklist the right-wing site. According to The Washington Post, most comply.

However, there is some anecdotal evidence that advertisers respond by withdrawing from all controversial sites. In the fake news debate (Pretty much: “You’re fake news!” — “ No, you’re fake news!”) any news site can be too controversial for advertisers. So, as total digital advertising goes sideways, news-related advertising goes down.