When a series of national security reforms, including mandatory data retention, was referred to the Joint Committee on Intelligence and Security by the government last year, many in the online rights community suspected a stitch-up. Some (as always) self-described Anonymous members launched an “op” to attack the inquiry and anyone submitting to it. Others maintained the inquiry was a piece of theatre to give the impression of consultation before the inevitable imposition of data retention.

Twelve months later the inquiry is finished, it declined to recommend data retention, produced page after page of detail from submissions on why it was problematic from privacy and other perspectives, and urged a minimalist model in the event any government should decide in the future it wants data retention. The government, albeit one in its death throes, has put data retention on hold indefinitely pending “further advice from the departments and relevant agencies and comprehensive consultation”.

Some stitch-up.

And there are those do not emerge from it looking too flash.

The Attorney-General’s Department has been humiliated. A government-controlled committee (Labor MP Anthony Byrne is chair) has rarely belted a government department as hard as the committee did AGD in its report, putting into black and white the frustration that was evident throughout the hearings process last year.

And let’s be clear about this: AGD developed in secret a data retention scheme, and almost certainly drafted legislation for it after an extensive, multi-year secret consultation process with industry and other departments. But when time came for AGD to explain and justify the scheme in public, it failed abjectly, barely even mentioning the proposal in the discussion paper designed to kickstart the inquiry. And since then, it has chopped and changed its story on what it did (and when) on data retention, claiming a middle manager merely had a few chats with industry and nothing formal was ever drawn up. Quite what AGD did on data retention and why it later chose to try to hide its activities may never be completely clear, but we’ll see what the Freedom of Information process brings.

“… in that face of what appears to be an unstoppable wave of surveillance aimed at us all, there are ways to respond effectively.”

Either way, if you want a reverse playbook on how to develop and pass a controversial reform, AGD has just spent the last year writing it. Department Secretary Roger Wilkins has some serious questions to answer about his department’s performance.

There’s unlikely to be any pressure on Wilkins to explain himself, however, because of the lack of mainstream media interest in the inquiry or the issues it has raised. Dylan Welch did an excellent job covering the inquiry for Fairfax before heading to Afghanistan last year. Other than that, and Greg Sheridan’s froth-mouthed demands for a surveillance state at The Australian, it has only been independent media, and particularly the IT industry media, that have shown interest. Even News Limited’s usual antipathy to Labor couldn’t be relied on to motivate any coverage: while Kim “OMG not self-regulation!” Williams confected nonsensical claims about a Labor attack on the media via its media reform package, the far more plausible threat to a free press from Labor’s data retention scheme was being ignored beyond one Daily Telegraph editorial hesitantly expressing concern.

The use of information retained under the EU data retention directive by security agencies to track down journalists’ sources in Poland has been the subject of legal action there; the EU directive is the exact model put forward by AGD. As Greens MP Scott Ludlam has pointed out, journalist shield laws aren’t much help if obtaining information on where a hack’s mobile phone was, and whom she called and who called her, is so straightforward.

That all said, look at the US example. There was no public debate or inquiry into data retention there; the Obama administrations used extraordinarily broad legislation passed after 9/11, like the Patriot Act, to develop, in secret, vast surveillance regimes in which not just telecommunications data but content itself was monitored and stored (the Bush administration simply broke the law and did it, then Congress retrospectively legalised it). Obama’s assurances to Americans in the aftermath of the Edward Snowden revelations that “nobody is listening to your phone calls” have been shown to be patently false, while internet traffic has been captured on a global scale by US and UK agencies. Not merely was the scheme itself secret, but it was buttressed by secret legal opinions, a secret court, secret hearings and secret judicial decisions.

The result is that the only way accountability and transparency could be provided and debate instigated was via a whistleblower, not through normal channels.

And many in the US media haven’t settled for ignoring the issue, but have actually taken the government line that secret population-level surveillance is acceptable and that Snowden is a spy or a criminal. Indeed, even the journalists who reported his revelations have been targeted by other “journalists” demanding to know why they shouldn’t be jailed.

The Attorney-General’s Department, with or without the connivance of Labor attorneys-general, wanted the same secrecy here about mass surveillance as in the US. They didn’t get it, and as a result their surveillance plans have been defeated. The system worked, kinda. It shows that, in that face of what appears to be an unstoppable wave of surveillance aimed at us all, there are ways to respond effectively.