Product search
Currency Settings
Shopping Basket
Your shopping basket is empty
A response to "Acupuncture: Does it alleviate pain and are there serious risks? A review of reviews" by E. Ernst, Myeong Soo Lee and Tae-Young Choi, PAIN®, Volume 152, Issue 4 (April 2011)
Edzard Ernst's latest paper on acupuncture continues his endeavours to demonstrate that acupuncture is both more harmful and less effective than is claimed.
Called, "Acupuncture: Does it alleviate pain and are there serious risks? A review of reviews" by E. Ernst, Myeong Soo Lee and Tae-Young Choi (DOI: 10.1016/j.pain.2010.11.004) appeared in PAIN®, Volume 152, Issue 4 (April 2011) with an accompanying commentary, "Acupuncture's claims punctured: Not proven effective for pain, not harmless" by Harriet Hall, MD.
Ernst et al's paper declares that "numerous systematic reviews have generated little truly convincing evidence that acupuncture is effective in reducing pain, and serious adverse effects continue to be reported, and "Many systematic reviews of acupuncture for pain management are available, yet they only support few indications, and contradictions abound," and "Acupuncture remains associated with serious adverse effects."
Hall's commentary states that "Importantly, when a treatment is truly effective, studies tend to produce more convincing results as time passes and the weight of evidence accumulates. When a treatment is extensively studied for decades and the evidence continues to be inconsistent, it becomes more and more likely that the treatment is not truly effective. This appears to be the case for acupuncture. In fact, taken as a whole, the published (and scientifically rigorous) evidence leads to the conclusion that acupuncture is no more effective than placebo."
This is a serious and damning conclusion for acupuncturists and clearly Ernst's paper requires a considered and informed response. For that reason we invited Mark Bovey, Research Co-Ordinator of The Acupuncture Research Resource Centre (funded by The British Acupuncture Council) to respond. He reports major problems with Ernst's paper ("inadequate methodology that at times crosses the border into misrepresentation") that clearly call its conclusions into question.
Comments on Ernst et al's 2011 overview of acupuncture-pain systematic review by Mark Bovey
They scored SRs as positive (+), negative (-) or equivocal (+/-), apparently by reading the authors' concluding statements and making a value judgement on their meaning. A section of relevant text is quoted as substantiation in each case. The method is not actually defined in the paper (which is very poor practice). I shall take the example of migraine to examine this process in more detail, as I have recently reviewed that literature.
Ernst et al found that 'conclusions were largely negative' for migraine, while for migraine prophylaxis 'reviews failed to show effectiveness'.
The oldest of the three SRs is Griggs (2006), which reached 'no solid conclusions or advice' because of the inadequacy of the trials to date. Ernst's score of +/- seems reasonable. However, 6 of the 13 RCTs included in this review were actually for tension headache not migraine, a fact unremarked by Ernst et al, which calls into question their methodological rigor (and is especially ironic given that they later criticize Sun and Gan's (2008) review on headache for being too broad in its diagnostic categories).
Next is Zhang et al (2008). This is scored (-) on the basis of the finding that there were no high quality trials on acupuncture for migraine prophylaxis in China, and hence no accurate evaluation was possible. However, of the 12 RCTs included only three were from China; the nine non-Chinese studies were of a sufficient quality to allow accurate evaluation, so this seems a perverse basis for the negative score. In fact Zhang's paper was essentially about criticising study quality in China and trying to get it improved, not about estimating effect sizes (which was done the same year for Chinese migraine trials by an Australian team, a SR not included by Ernst – see below).
This highlights a fundamental problem with this review method. Different reviewers may apply very different standards when drawing conclusions. For example some are non-committal if the data are largely from low-quality trials; others will note the low quality but still make positive recommendations if the data show significant superiorities. The methods used by Ernst et al cannot possibly make sense of these variations.
Worse, though, is when there is selective reporting and misrepresentation of the SR authors' conclusions, and there is a good example of this in the third migraine paper, Linde et al (2009). In Linde's main text conclusions there are two sub-sections, relating to practice and to research. The first says: 'Although the available results suggest that the selection of specific points is not as important as had been thought by providers, acupuncture should be considered as a treatment option for migraine patients needing prophylactic treatment due to frequent or insufficiently controlled migraine attacks, particularly in patients refusing prophylactic drug treatment or experiencing adverse effects from such treatment.'. The second says 'There is a clear need for further studies', and goes on to suggest various possibilities. Ernst et al have not used any of these conclusions for their scoring process; instead they have gone to the abstract's conclusion. Perhaps they did not read the full text, or perhaps they were looking for a negative statement but could not find one there.
The abstract shows four conclusions: i. consistent evidence that acupuncture provides additional benefits to treatment of acute migraine attacks only or to routine care; ii. no evidence for the effect of true acupuncture over sham, though this is difficult to interpret, as exact point location could be of limited importance; iii. acupuncture is at least as effective as, or possibly more effective than, prophylactic drug treatment, and has fewer adverse effects; iv. acupuncture should be considered a treatment option for patients willing to undergo this treatment.
It is hard to believe that a scientific reviewer could construe these conclusions as negative. Ernst et al extracted just one of them (the second – and that without the proviso) and used that to determine their verdict. This would be admissible only if their stipulated scoring criteria were based solely on verum-sham comparisons, but they were not. It can only cast huge doubt on the accuracy of the other SR scores, and on the validity of the whole exercise.
Continuing with migraine, it should be noted that Ernst et al did not seem to locate two Australian SRs that carried out meta-analysis. The first, Scott and Deare (2006) concluded that acupuncture with or without conventional treatment would be likely to reduce migraine frequency, with minimal side-effects. The second, Wang et al (2008), included a review of the Chinese trials, which has the advantage of not overlapping at all with Linde's Cochrane SR. Wang et al concluded that acupuncture might be an effective prophylactic treatment, used alone or together with drugs. Better quality trials would be needed to confirm this. Hence both have predominantly positive conclusions. Other recent overviews (Pickett and Blackwell, 2010; Drug and Therapeutic Bulletin, 2010; Sherman and Coeytaux, 2009) all recommend that acupuncture be used along with existing treatment options, i.e. they have positive findings and directly contrast with Ernst et al.
Returning to the general picture, another major difficulty with Ernst et al's approach is that all SRs get equal weight in the summing up. Despite the fact that the more recent ones will tend to include many of the component trials in the earlier ones. Hence for tension headache there are four SRs listed, three are deemed negative, dated 2005, 2005 and 2008. The last is a Cochrane review from 2009, which has the best methodology and the best data. In the previous version of that SR only 5 studies had been included, and no conclusions could be reached. The 6 more recent studies changed that, leading to positive results. However, this one positive review is trumped by the three earlier inferior ones, leading to a verdict of 'largely negative'. This gives a misleading picture of the evidence.
Other points made by Ernst et al
i. There has been an increase in the proportion of positive reviews over the years. Thus an overview in 2001 located 17 SRs, with the majority being negative (Linde et al, 2001). In 2006 Derry et al claimed that there was no robust evidence that acupuncture works for any indication. Now in 2011 Ernst et al report on 58 SRs (not 57 as stated), with 25 +, 13 +/- and 20 -, a distinctly positive trend even with Ernst's questionable grading standards. This is in direct contrast to those (e.g. Hall, in her accompanying commentary) who maintain that decades of research keep producing the same outcomes, so we should just accept that there will never be evidence supporting acupuncture.)
ii. There are many inconsistencies between multiple reviews on the same condition. Many conditions are covered by just a single review (and many of course by none). Hence we need more primary data and more rigorous studies.
iii. The majority of SRs are positive for low back pain and osteoarthritis
iv. There is a positive trend for myofascial pain
v. Reviews on rheumatoid arthritis are largely negative
vi. Those on dysmenorrhoea and TMJ are contradictory. This may be true, but for the reasons given above we should not accept it at face value.
Placebo
Ernst et al attempt to undermine the positive SR results by suggesting that they are largely due to placebo effects. This involves more selective reporting. They cite Cherkin et al (2009) as suggesting that placebo effects may explain acupuncture's benefits; but fail to tell us that this was just one of Cherkin's possible explanations, the other being, "superficial point stimulation directly stimulates physiological processes that ultimately lead to improved pain and function." Cherkin et al went on to say that acupuncture sham controls are controversial, pointing out that superficial , and non-invasive needling can be seen as effective and real acupuncture. They recommended acupuncture as a reasonable option for clinicians and patients, being relatively safe and effective (for low back pain).
Madsen et al (2009) is cited to support the contention that, "adequately controlling for non-specific effects in future is likely to demonstrate that acupuncture has no or few specific effects on pain." This is of course possible, but it is entirely conjecture at present. There is no agreement as to what are the specific effects of acupuncture and indeed the concept of specific and non-specific may not be very meaningful in this context (Hopton and MacPherson, 2010; Sherman and Coeytaux). Ernst et al fail to point out that Madsen's meta-analysis indicated statistical significance between sham and acupuncture across a range of pain conditions.
Most acupuncture researchers believe that there is no workable sham procedure for acupuncture (Lundeberg et al 2009) and that hence such studies may actually underestimate the acupuncture-placebo difference (Linde et al, 2007). There are many interesting scientific questions arising from the often paradoxical clinical data involving acupuncture, sham, no treatment and other interventions, with various possible answers, but no indication that we're anywhere near reaching definitive conclusions (Wayne et al, 2009). Moreover, most acupuncture researchers believe that sham studies, whilst of interest to researchers, are not very useful for informing clinical decisions, which require other kinds of evidence (Wayne et al 2009; Linde et al 2009; Hopton and MacPherson, 2010).
Adverse effects
One would not want to argue with the main points in the review which are that,
i. adverse effects do happen; ii. the number of serious ones is minute in relation to the numbers of treatments; iii. all acupuncturists should be adequately trained.
There is nothing new here. The trouble is that the mere mention of 95 serious adverse effects and five deaths is taken by some (Hall, 2011) to show that acupuncture is dangerous. Hall ludicrously ends by suggesting that drugs with this evidence record ( her interpretation being zero effectiveness allied to serious adverse effects) would not be prescribed, so why should acupuncture be treated any differently. There has been plenty of mud-slinging from both sides about the hazardous side-effects of the other's treatments and the low-science approach of Ernst et al merely provides more ammunition. If you wish to present high quality quantitative evidence on adverse effects, to accompany effectiveness evidence from systematic reviews, then you should use the available prospective survey data, not the randomly recorded retrospective cases gathered here. We know that, i. estimates of the risk of serious events are very low (Drug and Therapeutic Bulletin, 2010; Weidenhammer et al, 2007); ii. no deaths or permanent injuries have been reported from observational studies of hundreds of thousands of patients (Witt et al 2009; MacPherson et al, 2004).
Hall (2011) also gets horribly tied up in her arguments on risk/benefit ratio: acupuncture is no more effective than a placebo, therefore it has no benefit, therefore any risk is too great. The first premise has been discussed already; the second is obviously untrue, as placebo effects can be substantial; hence the third is irrelevant.
Summary
This paper is useful in that it assembles in one place the references and some basic details for acupuncture-pain SRs and for reported adverse effects. Unfortunately it adds nothing to the evidence base due to inadequate methodology that at times crosses the border into misrepresentation.
It is useful to compare it with another recent overview of acupuncture for painful conditions (Hopton and MacPherson, 2010). The latter selected only reviews with pooled meta-analysed data for the three most common pain conditions and found acupuncture superior to sham for all three (there was some inconsistency in the longer-term outcomes with back pain). Their approach appears to be better and more objective than Ernst's (it's certainly better reported) but it is still of course dependent on the size and quality of the primary studies. An international consortium of researchers is currently working on the sort of analysis that will give us much better evidence on the 'is acupuncture better than sham?' question (Vickers et al, 2010). By pooling individual data from the best RCTs of the last decade they will surmount many of the problems that have resulted in inconsistent and inconclusive SRs. When these results are published we can look forward to debating with more hard facts at our disposal.
Categories: Mark Bovey, Peter Deadman
Keine Kommentare:
Kommentar veröffentlichen