PBL Media’s battle to get its debt recapitalisation package past its bankers is not the only thing troubling James Packer at the moment. He’s got 3.3 billion reasons to wish that 2008 would hurry up and finish.

After being unseated as Australia’s richest man by Andrew Forrest on BRW’s Rich 200 list in May, Packer’s year has gone from bad to worse as the Australian sharemarket has plunged to five-year lows.

Hardest hit has been the cornerstone of Packer’s fortune, a stake in international gaming giant Crown Limited. The value of the Packer stake has fallen by almost 59% or $2 billion since the start of the year, from $3.4 billion to $1.4 billion.

The value of Packer’s Consolidated Media stake has halved since January, falling from $1 billion to $500 million. No wonder Packer is refusing to pump any more cash into the media business.

Since the death of Kerry Packer in Boxing Day in 2005, Packer has made a series of strategic investments in listed companies. As Crikey reported back in 2007, Packer appeared to have inherited his father’s golden touch as equity markets soared. But in 2008, many of the company’s Packer invested in have been sold-off sharply after being caught with too much debt.

Queensland-based property developer Sunland Group was Packer’s most prominent play – as well as grabbing a 37% stake in the company in 2006, Packer agreed to sit on the board. Packer invested at around $1.50 and the stock soared as high as $4.50 at the start of 2008. But Sunland’s stock has since slumped to just 79c, taking Packer’s stake from $168 million to just $28 million.

The Packer family’s big investment in Challenger Financial Services Group has also fallen dramatically this year, from $608 million to $168 million as the company was belted by the credit crisis and short sellers.

A series of smaller speculative investments Packer has made in the last few years have also performed terribly. Shares in Energy World Corporation are down 75% this year, while Wildhorse Energy stock has plummeted 90.4%. A small investment in Magellan Flagship Fun has sunk 26.5%.

All up, the value of Packer’s investment portfolio has dropped from $5.5 billion to $2.2 billion, a fall of $3.3 billion or 60%. Over the same period, the benchmark ASX 200 has fallen by 45%.

The Packer family’s large portfolio of private assets makes it difficult to put an exact figure on the size of his diminished fortune. But even if the value of his private assets has remained steady since the start of the year (which is highly unlikely) Packer’s fortune would have had to sunk from around $6.5 billion to about $3.5 billion since the start of 2008.

There are rumours that Packer is taking his losses very hard and has become something of a recluse. Certainly he dodged his last official public company obligation, the Sunland annual general meeting on 10 November and did not give a reason for his non-appearance. A week before this, he refused to allow photographers into the Seek.com AGM and in October he reportedly snuck into the Crown AGM through the hotel kitchen.

If there’s one consolation for Packer, it might come from the fact that he hasn’t lost as much as Andrew Forrest, who has seen $4.3 billion wiped off the value of his stake in Fortescue Metals Group since the start of the year.

Both men have been replaced at the top of Australia’s wealth tree by Westfield boss Frank Lowy, who is still worth around $5.5 billion.