Yesterday I wrote about the connections between some of the principal individuals who have been vocal in opposing wind farm development in Australia.
The various organisations at the forefront of the opposition recently published eight papers in a special edition of an allegedly peer-reviewed journal that you could be excused from never having heard of. The Bulletin of Science, Technology & Society is a journal that has appeared erratically over the past few years. It was indexed between 1981-1995 by the Web of Science, the international scientific indexing platform that “covers over 10,000 of the highest impact journals worldwide, including Open Access journals and over 110,000 conference proceedings”.
But after 1995 it was dropped from the list of journals being indexed, generally a sign that indexing services regard a journal as having fallen below an acceptable scientific standard.
In the 14 years it was indexed, a citation search conduced on October 10, 2011 showed that it published 961 papers that had been cited for a grand total of just 345 times — an average of 0.36 per paper — almost a homeopathic strength citation rate. Today, Web of Science shows it has published only seven papers that have been cited seven or more times, with the most-cited paper in its history having been cited just 15 times. PubMed, the indexing service of the US National Library of Medicine also does not index the journal.
Nonetheless, anti-wind farm websites have jubilantly described the journal as a “leading scientific peer-reviewed journal” and the issue as “groundbreaking”. In summary, this is a journal that cannot be described as low ranking in scientific research publishing. It is more accurately described as “unranking”.
All the papers in the issue are only accessible by pay-per-view, but to give an idea of the quality, here’s an example: Krogh CME. Industrial wind turbine development and loss of social justice. Bulletin of Science, Technology and Society 2011;31(4):321-333.
This paper contains no “methods” section, so it fails to conform to the most basic requirement of scientific reporting: that it contains details of how the research reported was undertaken. This is a fundamental requirement because without it, readers have no way of assessing the adequacy and rigor of any investigation, and whether any results report and conclusions drawn are justified or not. Instead, the author — a retired pharmacist who PubMed shows published one paper in 1985 — explains that she “began investigating reports of adverse health effects made by individuals living in the environs” of wind turbines in Ontario, Canada for “more than two years”. That’s it. And this is called peer reviewed “research”.
Instead of describing any research, the author has written a paper that mixes up statements somehow apparently made to her by de-identified informants about negative effects of exposure to turbines with similar examples from other parts of the world, from websites and submission to inquiries by wind farm opponents. We are told nothing about the process by which her informants were interviewed, the questions they were asked, how they were selected and whether her “study” was approved by any institutional research ethics committee. This is not a paper that would be make first base as an example of serious scientific investigation about wind farms and health.
Its findings contain not a single example of any informant reporting anything but adverse effects of exposure to wind farms, when it is widely acknowledged that a large majority of those so exposed report no adverse effects nor complain about the turbines and even (make the sign of the cross …) like them.
In an attempt to understand the process of peer review that had been followed, in August 2011, I wrote to the editor of the Bulletin, asking the following questions:
- Were you approached by those participating in [a 2010 anti-wind farm meeting held in Ontario] to publish these papers? Or did the initiative come from you?
- Did you personally edit this issue or were guest editors used? If so, can you please describe how they were selected?
- Was there a charge made to the authors to publish their papers together like this?
- It is plain that all the papers are openly negative about wind farms, which is curious given that there is a large body of research that demonstrates a very different picture. Did you put out a call for submissions or approach researchers working in this area to submit manuscripts?
- Did you approach any authors who did not have affiliations with the anti-wind farm movement?
- Were all the papers peer reviewed?
- Did the authors propose their own reviewers and were these the reviewers used?
- Can signed or de-indentified copies of these reviews be made available to others on request?
Over several testy email replies, the editor made the following comments:
“A third party mediated between the organizers of the symposium and myself. We are dealing with a very difficult situation in which there is no balanced approach to begin with. Deep pockets have controlled the research agenda and professional people with impeccable credentials did what they did in this case out of there (sic) own pocket. As far as refereeing is concerned, never has any issue been so over refereed by people with impeccable credentials in anticipation of the kinds of concerns you voice.
“I can assure you that this Bulletin is not a front for any special interest group and that I would not have dreamt of publishing this issue had it not been for the questionable conduct of the windfarm industry and government officials. The issue attempts to create a little bit of balance, and show that there are legitimate other voices coming from people with impeccable credentials who are not funded because of their views.”
No copies of reviews or reviewers names were provided. A researcher from the University of Adelaide has subsequently written directly to the authors and received no reviews back. The principle of open peer reviewing is widely discussed in research publishing and while requests by others to see reviews are unusual, refusal to be transparent can only promote suspicion about the process, particularly when the quality of the papers is considered.
Could it be that a retired pharmacist knows more about health issues than a professor of public health who is a sociologist, not a medical doctor? I have seen claims of the latter kind made by medical specialists in the letters pages of the Medical Journal of Australia.
While I am not a “principal individuals ……………….opposing wind farm development in Australia”, I am opposed to inapropriate placement of industrial complexes, toxic waste processors, high rise buildings and 160 m high propellers with engines attached; and to the the people of Australia subsidising such things and subsidising people to spend time bantering about such things. This article will no doubt attract as many comments as Mr Chapman’s “connections” article. It does not matter how professionally and persistently anyone’s integrety is questioned – the simple solution to it all is to request the government to impose a moratorium, fund proper research, produce a proper report and proceed to act on that report, very clearly holding the professionals involved responsible thereafter (in the architect lifelong responsibility model).
@MUZZA Posted Friday, 14 October 2011 at 1:51 pm
If you are going to attack the man like that you need to lose the anonymity, tell us who you are, what your qualifications are, and where you have published–because frankly your claim yesterday about publishing in scientific journals left me totally unconvinced from all your earlier comments. You do not seem to have a clue about the rigours of scientific publishing.
And incidentally, if you think citing a letters by “medical specialists” in the MJA will convince a lifelong research scientist (who, BTW, refuses to publish in Australian-only journals), again you don’t have a clue. A “letter” to a journal is not anything equivalent to peer-reviewed data-based paper. I am not saying I would dismiss it out of hand but I am not going to give it anything like the same weight either.
As I wrote in yesterday’s comments one of the first things a scientist does when looking a published paper, is a quality screen on the journal–exactly as Simon Chapman does in his article today. That journal and those papers will simply not carry much weight with serious researchers; indeed I have to admit the opposite, it induces immediate suspicion and doubt, and yes scepticism even before reading them. This is an astro-turfing strategy–set up quasi-bogus publishing channel so they can claim whatever they like from the “scientific literature”.
Come on Dave Burraston. Give us your vitriol.
Given that Simon Chapman’s piece today is entirely concerned with assessing the bona fides of journals claiming to be worthy of scientific journalism, I will reproduce my relevant comment from yesterday. Yes, it is a bit of a scientific publishing 101, but let’s face it, very few of Crikey’s (or any newspaper) readership have any idea of how it really works. The scientific publishing world is self-policing and it is perhaps surprisingly effective despite its imperfections. This is why we get pretty annoyed, either when it is abused (as in Chapman’s article today) or when media gives equal weight to the likes of Monckton/Bolt/Jones et al, in which it has taken a tortuous and expensive High Court case to reveal that Bolt is extremely “economical with the truth”.
[(crikey.com.au/2011/10/13/the-web-of-vested-interests-behind-the-anti-wind-farm-lobby/#comment-163741)
MICHAEL R JAMES Posted Thursday, 13 October 2011 at 9:55 pm | Permalink
Muzza, your examples are the opposite of what you want to say or show. Sometimes a discovery that is counter to the prevailing paradigm will be rejected, or considered improbable. However in the crystals story, he in fact published it in peer reviewed journals and other scientists still thought it was nonsense. But the point of such publication — indeed the top requirement for publication of conclusions based on experimental data is that you must provide enough detail to allow an equivalent scientist/research lab to reproduce the experiment and so either confirm or reject the results. This was what was done and eventually multiple other labs did reproduce his results and it slowly became accepted. That is how it is supposed to work. Another recent example is of the neutrinos travelling faster than light: the scientist have published their experiments so that others can attempt to either see the error (which is what everyone expects) or in fact reproduce it. Of course peer-review is also to check that the experimental approach seems valid and that the conclusions can be logically made from the data.
And you obviously have no idea of how peer review works. But look I really do not want to adopt a tone here. (sorry about the earlier sneer, it came from frustration) It is not easy for a non-scientist to understand; however please try. And do not make uninformed silly comments about it, as if it was like publishing a novel of fiction or a frontpage article in The Australian.]