Andrew Forrest’s recent address to the National Press Club signalled another war of words with the Gillard government, this time over his plan for indigenous jobs. Forrest used the occasion to criticise the government’s approach to training indigenous job seekers, arguing that billions was being wasted on training programs that failed to lead to sustainable employment.

In making his case, Forrest relied on a key set of figures. He stated that while the government’s efforts at assisting indigenous job seekers to find work through its Job Services Australia scheme have resulted in a retention rate of only 45% over three months, his own indigenous jobs program — the Australian Employment Covenant — has effected 10,500 job placements and a retention rate of more than 70% after six months. According to Forrest, this is evidence that his model works where the government’s approach too regularly fails.

It is certainly true that the government’s management of indigenous employment services could be dramatically improved. However, those readers following Forrest’s media appearances could be forgiven for being confused.

On the same day as Forrest’s address to the National Press Club, a news report quoted Julie Collins — the Minister for Indigenous Employment and Economic Development — as saying that, based on figures given to the government by the AEC on February 23, it had made just over 7000 employment placements and only about 2100 people had stayed in those jobs to six months. This is far fewer placements — and seemingly a much lower retention rate — than Forrest has claimed.

So which set of figures should we believe? Marcia Langton — a member of the AEC’s steering committee — argued the government had it wrong. According to Langton, the minister’s representation of the evidence was “shifty” and “dodgy”. Somebody (perhaps the person who briefed the minister) had “massaged” the figures.

The reality, though, isn’t quite so simple. Because complexity isn’t amenable to short news grabs or provocative headlines, most news reports have glossed over any discrepancies in data, content to paint Forrest and the government into opposing camps. But the story gets much more interesting the deeper the analysis, so it’s worth taking a few moments to look at the evidence.

According to the Australian Employment Covenant, the latest figure is just under 10,700 job placements. This is significantly higher than the number quoted by Collins because it is based on the most up-to-date reports from employers, while Collins’ number refers to older data from the December 2011 quarter. As to retention, the AEC confirms its records show a 71% retention rate at six months. However, this corresponds to only about 3500 individuals. How the AEC translates this into a 71% retention rate requires some explanation.

First, the AEC excludes the number of people placed in employment within the past six months. This is appropriate because, by definition, they could not have reached six months retention in their jobs, even though they may well do so over time. AEC data show that there are about 2800 individuals in this situation, so this leaves about 7900 job placements for which retention data should be available.

However, the AEC also makes a second adjustment: to exclude employers who have not reported retention data to the AEC, apparently because they do not yet have the internal reporting systems to easily produce such information. Although the number of these employers is small, together they account for almost 3000 of the total job placements. The AEC subtracts these placements from 7900 so, effectively, it is only reporting on retention rates for about 4900 jobs.

While this should have been made clear in Forrest’s speech, there is no reason to believe that either the AEC, or the government, have deliberately “cooked the books”.

Interestingly, though, there are several additional statements made by Forrest in his Press Club appearance that warrant further investigation. The first is the assertion that the 10,500 job placements correspond to 10,500 indigenous people moving off welfare and into work. While it makes for a catchy slogan, there are no data to support this claim.

Many of the individuals employed in AEC jobs would have come from the pool of unemployed, but numerous others would have moved into these positions from other paid work. If they have found jobs, or a career path, that better suit their needs then this is still a positive result. But there is a clear danger here of careless — if catchy — rhetoric bending the truth.

Forrest’s argument that the Australian government should support a particular training model based on the Vocational Training and Employment Centre at his Pilbara mining operations also confused matters. Established in 2006, VTEC has so far trained about 1000 indigenous people to work with Fortescue Metals Group or its contractors. The link between the AEC and VTEC is fairly straightforward: Fortescue and participating contractors are also members of the AEC, so the 1000 or so employment outcomes achieved through VTEC can be counted towards the AEC’s overall success. However, the AEC also includes several hundred more participating employers. None of these have used the VTEC facility, so conflating the AEC jobs figures with arguments for expanding the number of VTECs seems rather clumsy.

Forrest also suggested that the VTEC and AEC schemes have cost only about $13,000 for each individual trained and placed into a job. This calculation refers to the 10,500 AEC job placements, for which participating employers have been allocated $132 million for training through the Australian government’s Indigenous Employment Program. The figure of approximately $13,000 simply divides this IEP allocation by the number of jobs filled. The problem with this approach is that it leaves out several funding streams outside of the IEP. These include, for example, investments from Fortescue and other employers involved in training indigenous job seekers who have committed funds in addition to their IEP contracts.

The full extent of these investments has not been made public, but a rigorous analysis would include them in any calculation of the costs and benefits of Forrest’s scheme.Perhaps most confusing, though, is Forrest’s claim in his address that FMG had “commissioned an independent study of our VTECs”. According to Forrest, “that independent research tells us that the VTEC model works”. Again, this statement needs careful explanation.

Earlier this year, GenerationOne — another organisation Forrest is associated with — published commissioned research on six employers deemed to have successful indigenous employment outcomes. The report attempted to distil key principles that could be the foundation for success elsewhere. While these principles have informed what Forrest is calling the “VTEC model”, none of the six employers have any association with FMG’s VTEC in the Pilbara, having relied instead on funding from the IEP.

The AEC says that while there is plenty of anecdotal evidence that FMG’s VTEC facility works, a formal study of its effectiveness is yet to be commissioned. In this context, the government is wise to insist that a “proper assessment” be conducted before committing to fund additional VTECs elsewhere.

Forrest’s enthusiasm for these issues is no doubt well-intentioned. He clearly believes that he is onto something that works. But where his rhetoric confuses the evidence it does nothing to inform public debate.

One might expect that media scrutiny would reveal any inconsistencies here, particularly because Forrest attracts so much public attention and is lobbying the government about the expenditure of taxpayer funds. However, much media reportage has simply served to heighten confusion about what the AEC and VTEC have actually achieved.

For a recent example, one need look no further than an April editorial in The Australian, which congratulated Forrest on “helping place more than 55,000 Aborigines in jobs since 2008”. A subsequent story on page one of The Oz repeated this claim. Based on Forrest’s own figures, though, he has done nothing of the sort. Press coverage like this may be careless, rather than deliberately misleading, but if there are any allegations of being “dodgy” in representing the figures then such examples should not escape attention.

Ongoing confusion about outcomes is a shame, because Forrest’s ideas are worth debating. His proposal relies on a combination of the AEC scheme (in which employers informally “guarantee” a job to an individual who successfully completes appropriate training) and a network of VTECs to co-ordinate the training and support services for the individual wanting to take on that role.

There is certainly some evidence that where employers guarantee jobs to those who successfully complete specific training this can contribute to positive employment outcomes. Incidentally, though, this has been the case in several instances in which the employers did not participate in the AEC or rely on training delivered through VTEC, so implying that VTEC is necessarily the best model or, as Forrest has, the “most effective instrument to change lives ever seen in Australian history” is probably taking the argument a bit too far.

Suggesting that the government should only fund training where there is a “guaranteed job” identified by an employer is also problematic, particularly if these jobs do not match the aspirations of some indigenous people — such as those choosing to live very remotely on their own lands. Economic opportunities in these locations might rely on more effective support for local governance and enterprise development, as well as training for any existing jobs.

Forrest’s approach may be one of a suite of measures that can help to improve the socio-economic position of indigenous Australians who have been unemployed or discouraged from looking for work. But with so many conflicting figures bandied about publicly, it is difficult to discern just how much merit there is in his proposal.

Suggestions that either the government or the AEC are ”dodgy” with their use of evidence are difficult to substantiate. But in an area as important as indigenous employment, we should look forward to some clearer evidence and, hopefully, more careful, critical analysis in the press.