If we step back from day-to-day politics, it’s noteworthy that our major political parties are in markedly different positions than ones they’ve occupied in recent years.

Labor is now into its second term with the same leader, something not seen since last century. It also has a stronger and clearer agenda. For a party that was criticised while in government for lacking a “narrative” (while passing a large volume of important legislation) and being uncertain as to what it stood for, it has led the policy debate in crucial areas for two years. And while rickety, its overall economic policy is internally consistent: mildly protectionist on trade, investment and jobs; big spending on health and education but committed to additional taxation and greater taxation integrity; strongly neo-protectionist on manufacturing. It’s a set of policies that appeals to its base, the union movement, and to much of the electorate. And it is structured around a theme of inclusive prosperity that is suddenly gospel right across the political spectrum. Even Prime Minister Malcolm Turnbull and big business are suddenly talking about the need for economic growth to bring everyone along. The ideological ground has shifted in Labor’s favour.

Bill Shorten also has luck, an important trait for a politician. In particular, his mistakes and misjudgements — like the spectacular error of insisting on the elevation of the appalling union hack Kimberley Kitching to the Senate as Stephen Conroy’s replacement — tend to be obscured by the government’s own bungling and stuff-ups.

Malcolm Turnbull, however, finds himself in a difficult position ideologically — and while the entire commentariat has been on his back about his failings as Prime Minister, it’s actually hard to think of another way for him to handle the divisions he finds himself confronting.

First, the “broad tent” nature of the Liberal Party is becoming more problematic. Whereas Labor was once the party of hardened ideology-based factions, now the parliamentary Liberal Party is more noticeably segmented into moderate and conservative groupings. And so trigger-happy are the right within his ranks, that the barest hint of shift to centre — for example, on an issue like climate policy, as we’ve seen recently — elicits a tantrum that quickly snaps Turnbull back into line. For any leader without the stature of, say, John Howard, it is difficult to keep those groupings in balance, especially when the party in a key state like NSW is riven with factional strife. After the 2009 putsch against Turnbull, moderates locked in behind Tony Abbott for the sake of electoral success, but the moment his fortunes faded they queued up to knife him. Now right-wing Liberal MPs again snipe at, and undermine, Turnbull.

[Whisper it softly: is this government any better than Abbott’s?]

And while Turnbull is perceived — correctly — as suffering from clinging to Tony Abbott’s policies, the Liberals continue to suffer from another problem that manifested itself during Abbott’s leadership: an incapacity to effectively develop and sell positive policies. Even with the resources of government, the number of positive policies developed and implemented by the Coalition since 2013 has been limited. Tony Abbott and Joe Hockey tried to implement a hardline liberal economic agenda but were so poor at selling it, it destroyed them. Where they tried to develop a positive agenda, such as on infrastructure, the results were frustrating and highly mixed.

Their failure made policy life much worse for the government, despite Turnbull promising to run a properly liberal government the night he became Prime Minister. The result is a self-contradictory economic agenda: free trade agreements, manufacturing protectionism and hostility to foreign investment; a proclaimed emphasis on fiscal discipline while running massive deficits and slashing taxes for major Liberal party donors. The ideological shift toward protectionism and “inclusive prosperity” has only made the Liberals’ ideological position more problematic.

So, too, has the shift to lower growth and poorer revenue growth. It’s likely the Liberals really did believe that all they had to do was get elected and the budget would whir back into surplus, that Labor really was somehow simply incapable of adding up. Governing has been a rude shock: the deficit has blown out again, and again, and again, and again, so persistently that there’s now a real chance Australia’s triple triple-A rating, so painstakingly obtained by Wayne Swan, will be lost by the Liberals — the ultimate humiliation for a party that has persistently claimed surpluses are “in its DNA”.

The deep problems of the Liberal agenda are the reason why Turnbull, like Abbott before him, increasingly relies on attacking Labor rather than selling his own policies, and returns to issues, like asylum seekers, where he believes he can inflict damage on Labor. The problem, as Abbott’s prime ministership showed, is that the electorate doesn’t rate you much as a leader if all you do is bag your opponent.

Meantime, the Liberals’ Coalition partners, the Nationals, have continued their slide into mediocrity. The leaders of the Howard years, Tim Fischer and John Anderson, now seem like giants compared to Barnaby Joyce. The return of One Nation has terrified the Nationals, but rather than creatively responding to the sense of exclusion that fuels the lunatic fringe like Fischer and Anderson did, they are resorting to aping Pauline Hanson. And some Liberals have done the same. That’s the basis for the attacks on Lebanese Australians by Queenslander Peter Dutton.

[Did Dutton’s anti-Muslim spray breach section 18C?]

As with One Nation, Dutton’s goal is not so much to single out one group for vilification — although that’s taken as read — but to delegitimise all immigration. The goal — similar to his goal in smearing refugees as illiterate job thieves during the election campaign — is to convince Australians that we’re not the world’s most successful multicultural society — something of which both sides of politics ought to be be proud, because both have contributed to that success — but that we’re a failure, and that immigration and immigrants can be blamed for perceived social and economic problems.

It’s a sentiment that appeals to the Liberal Party’s reactionary base, some Labor voters, and to voters tempted to shift to One Nation. Labor’s 457 visa campaign — which has a long history now — is aimed at similar ends, but Dutton’s words are more overtly racist, and plainly his leader doesn’t feel internally strong enough to rebuke him.

As the year draws to a close, Malcolm Turnbull has the more complex ideological challenge, mostly for reasons beyond his control. And the problems won’t be remedied any time soon — especially not if his party continues to deny him the kind of authority needed to bridge the ideological gap within his party.