Quackwatch Home Page


In January 1993, an article in the New England Journal of Medicine concluded that one out of three Americans had used at least one unconventional therapy in the previous year. Although the report has been widely cited as evidence of public interest in "alternative methods," the study's design was extremely poor. The following analysis -- originally published in the Fall/Winter 1999 issue of The Scientific Review of Alternative Medicine -- provides the details.


The Eisenberg Data: Flawed and Deceptive

Timothy N. Gorski, MD, FACOG

". . . scientific integrity, [is] a principle of scientific thought that corresponds to a kind of utter honesty -- a kind of leaning over backwards. . . . Details that could throw doubt on your interpretation must be given, if you know them. You must do the best you can -- if you know anything at all wrong, or possibly wrong -- to explain it. . . . [you must] give all of the information to help others to judge the value of your contribution; not just the information that leads to judgment in one particular direction or another."

-- Richard Feynman (1918-1988) in his lecture "Cargo Cult Science," 1974

In January 1993, a paper entitled "Unconventional Medicine in the United States: Prevalence, Costs, and Patterns of Use" was published in The New England Journal of Medicine (NEJM.). The article contributed significantly to the recent explosion of interest in "alternative medicine" in the United States. The paper reports a telephone poll of 1539 people on their use of "unconventional" methods on the previous year. Until that time quackery and medical pseudoscience practices existed in a fringe subculture. This began to change around the time of this article's appearance and the report has become one of the most frequently cited articles in the medical literature. The lead author was David M Eisenberg, MD, whose chief publication up until this time had been a popular book on the wonders of traditional Chinese medicine entitled Encounters With Qi.

The article began by acknowledging the difficulty of defining "unconventional, alternative, or unorthodox therapies," referring to "medical practices that are not in conformity with the standards of the medical community." This is a fairly satisfactory definition of "alternative medicine" if "the standards of the medical community" are understood to include methods that are truly effective or which are under appropriate and well-designed scientific investigation. But the Eisenberg article settled on a definition of "unconventional medicine" being those "medical interventions not taught widely at U.S. medical schools or generally available at U.S. hospitals." It addressed the question of how widespread these methods are and how much money is spent on them in relation to medical care that is more or less in conformity with medical and scientific standards.

The 1539 people interviewed were a subset of 2295 respondents who agreed to participate and actually finished the interview. The 2295 people were, in turn, found by calling 5158 telephone numbers and excluding more than half of them because they were not working, were "not assigned to households," did not speak English, or were cognitively or physically impaired. But for the sake of argument and brevity, my comments leave aside the implications of these facts - that the survey was not representative of the population.

The telephone polling began by asking respondents about 24 medical conditions. Only 8% of respondents offered other "important conditions." The ten most frequently reported medical conditions listed by the Eisenberg survey are shown in Table 1 with, for comparison, figures from the Centers for Disease Control's Vital and Health Statistics, Prevalence of Selected Chronic Conditions 1990-92.

Table 1
Medical Condition

From Table 3 of
Eisenberg Article.

CDC Vital and Health Statistics, Prevalence of Selected Chronic Conditions 1990-92

"Back problems"

    Deformities/impairments of back
    Intervertebral disc disorders

20%

 

7.3%
2.0%

Allergies

Chronic sinusitis
Hayfever
Asthma

16%
 
 
13.5%
9.7%
4.6%
Arthritis

16%

12.8%
Insomnia

14%
 

Strains/sprains

    Deformities/impairments of Upper Extremities
    Shoulder
    Lower Extremities

13%

 

1.5
.2
5.0%

Headache

    Migraine
    Other Headache not tension

13%

 

4%
4.1%
High blood pressure

11%

11.1%

"Digestive problems"

Ulcer
Hernia
Indigestion/other functional disorders
  of stomach and digestive system
Constipation
Hemorrhoids

10%

 

1.7%
1.9%
2.6%

1.7%
3.8%
Anxiety

10%
 
Depression

8%
 
Deafness/Hearing Impairment  

9.3%

Skin disorders

    Dermatitis
    Dry skin
    Ingrown nails
    Corns/calluses
 

 


3.7%
2.1%
2.4%
1.9%
Heart Disease  

8.2%
Diabetes  

2.8%

Thyroid disorders

    Goiter
    Other thyroid
 

 


1.9%
1.3%
Chronic bronchitis  

5.2%

The Eisenberg survey also asked about dizziness, diabetes, and cancer, but the rates of these disorders and 11 others that the 1539 respondents were asked about were not given in the article. Presumably they were less than 8%, although, as is shown, hearing problems, skin conditions, and heart disease in the general population are in this range of prevalence according to the CDC. Deafness, of course, was the condition that D.D. Palmer "cured" when he "discovered" chiropractic more than a hundred years ago.

Notably absent from Eisenberg's data were common ailments such as colds, cough, and flu. Neither was there mention of obstetric or gynecologic disorders, which are probably the most common medical problems of reproductive aged women. Among women aged 18-44, for example, roughly the range in which Eisenberg reported the highest rate of use of "unconventional medicine," the CDC reported the prevalence of menstrual disorders as 2.8%, bladder infections 1.4%, kidney infections 1%, and other disorders of the female genital tract at 3%. Taken together as genitourinary problems, and including pregnancy and pregnancy-related complications, these would almost surely total 8% or more. Yet they appear to have been excluded.

Also absent was obesity, which is notable because commercial weight loss centers were included in "unconventional medicine" methods and were used by more respondents than the majority of other methods. As one of a variety of chronic medical conditions that has proven extraordinarily resistant to successful management by any means, obesity has been and continues to be a prime target for "alternative medicine" promoters. Its absence here is inexplicable.

When the survey respondents were asked about their use of "unconventional medicine" and whether or not they had seen a practitioner or "provider" of same, the answers broke down as shown in Table 2.

Table 2 (From Eisenberg et al, NEJM 1993;328:246-252.)
Type of Therapy

Used in Past 12 Months (%)

Saw a provider
(% of those who used)
Relaxation techniques

13

9
Chiropractic

10

70
Massage

7

41
Imagery

4

15
Spiritual healing

4

9
Commercial weight-loss

4

24
Lifestyle diets

4

13
Herbal medicine

3

10
Megavitamins

2

12
Self-help groups

2

38
Energy healing

1

32
Biofeedback

1

21
Hypnosis

1

52
Homeopathy

1

32
Acupuncture

<1

91
Folk remedies

<1

0
Exercise

26

-†
Prayer

25

-†
1 or more therapy (excluding exercise and prayer)

34

36

† "In the case of prayer, we thought [additional] questions would be inappropriate. As for exercise, we thought the term was too vague and the practice sufficiently ubiquitous to preclude the gathering of useful data." [from "The Interview" section of Eisenberg et al, 1993]

The survey did not include some common aberrant methods, and included ones that did not fit the report's own definition. Among those not mentioned were chelation therapy (for atherosclerosis), antineoplastons, iridology, reflexology, aromatherapy, crystal therapy, applied kinesiology, magnets, amino acids and other non-herb "nutritional supplements" consisting of biochemical precursors, co-factors, and intermediates, Maharishi Medicine (previously known as Ayurveda), naturopathy, urine-drinking, bee pollen, royal jelly, snake venom, colloidal silver, colonics, light therapy, and ozone therapy. All of these methods and others continue to receive credulous media attention, are widely promoted, and have attracted outspoken and devoted advocates. Their absence from the list raises questions of completeness and of representation of the "alternative" field.

Perhaps their use fell below the level of detection, but if so, that makes the "34%" use of "alternative methods" to be made up of other methods, perhaps not at all unconventional. Turning to the listed items, hypnosis, while only 1%, was included in "unconventional medicine." Yet hypnosis is an accepted adjunct to medical science in appropriate circumstances. It is taught at many U.S. medical schools and psychiatry residencies. Biofeedback, despite lack of firm effectiveness, retains a similar position. Although chiropractic was listed as the second most commonly used method, the study did not distinguish it from manipulative therapy, used by some osteopathic physicians and physical therapists. Some respondents may have experienced "chiropractic " in those forms. The study did not look at these important distinctions in order to distinguish, say, hypnosis used for pain control from use in past-life regression or to recall alien abduction experiences, or medical manipulation for joint mobilization from chiropractic treatment of childhood ear infections and learning disabilities.

The following statement explained about the methods reported in the survey: "We did not ask whether the respondents' providers of unconventional therapy were medical doctors." So, conceivably, some "chiropractic" was administered by physical therapists and D.O.'s. Some, or possibly all "biofeedback" took place at medical clinics specializing in pain management. And "megavitamin therapy" might include niacin for hypercholesterolemia and other standard pharmacologic applications of vitamins.

In the case of chiropractic there is another curious anomaly. Fully 30% of those who said they used chiropractic in the previous 12 months apparently did so without benefit of a chiropractor. The authors acknowledge that "Some forms of unconventional therapy typically involve a provider (for example, a chiropractor or acupuncturist)." But there is no attempt to explain or account either for the 30% of chiropractic use that people were practicing on themselves or the more curious finding that 9% of users of acupuncture had apparently been practicing on themselves. Or do these inconsistencies suggest another form of massage?

Again, "Some users of unconventional therapy may visit a provider less often than once a year but may continue to use the prescribed unconventional therapy." Was this the case only with imagery, "spiritual healing," and commercial weight loss programs, or also with a third of those who said they had used chiropractic and nearly a tenth of those who said they had used acupuncture?

The most-used "unconventional medicine" was "relaxation techniques." It was among the most-used methods for all of the ten top medical conditions considered besides "back problems" and allergies, as shown in Table 3.

Table 3 (From Table 3 Eisenberg et al, 1998; JAMA 280:1569-1575)
Of Those with

"Unconventional Medicine" use in the last 12 months
Saw Provider
(% of those who used)

Therapies Most Commonly Used
"Back problems"

36

19
Chiropractic, massage
Allergies

9

3
Spiritual healing, lifestyle diet
Arthritis

18

7
Chiropractic, Relaxation techniques
Insomnia

20

4
Relaxation techniques, imagery
Strains/sprains

22

10
Massage, Relaxation techniques
Headache

27

6
Relaxation techniques, chiropractic
High blood pressure

11

3
Relaxation techniques, homeopathy
"Digestive problems"

13

4
Relaxation techniques, megavitamins
Anxiety

28

6
Relaxation techniques, imagery
Depression

20

7
Relaxation techniques, self-help groups

But just what are "relaxation techniques?" Medical school graduates, as well as professionals in hospitals would agree that the idea of physical, mental, and emotional rest are components of good medical care. What is "massage?" It is a universal practice, when people injure themselves reflexively resort to "massage" the area. It would be difficult, given this, to find anyone who does not utilize "unconventional medicine." The authors recognized this problem, saying: "Some of the unconventional therapies studied warrant further clarification. For example, 'massage therapy' or 'relaxation therapy' may mean different things to different people." But, again, this was stated only in the small print, not to be found in interpretation and discussion of results.

"Spiritual healing" certainly means different things to different people as well. To most it means a healing ceremony of some sort, usually with a group. But is it also "spiritual healing" when a Catholic priest comes to the hospital bed to administer a sacrament? Or when anyone says a religious prayer? These practices are not taught in medical schools, but it seems doubtful that this deserves to be considered as "unconventional medicine." Likewise, the use of "imagery," especially in the management of insomnia and anxiety, is so customary that it is a cliché to speak of counting sheep to bring on drowsiness or imagining one's audience to be naked in order to allay stage fright.

But most, if not all of these qualifications, and omissions of distinctions had the cumulative effect of maximizing the numbers of survey respondents who could be said to have used "unconventional medicine." This, in turn, was used to support similarly magnified extrapolations of the use and economic impact of "unconventional medicine."

So, for example, the Eisenberg report stated:

The estimated number of ambulatory visits to providers of unconventional therapy in 1990 was 425 million. This number exceeds the estimated 388 million visits in 1990 to all primary care physicians (general and family practitioners, pediatricians, and specialists in internal medicine) combined.

But these "providers" included masseurs and masseuses who make no claims of curing any diseases, commercial weight loss methods, and self-help group meetings. And they might have included many physicians or people who working in conjunction with physicians, delivering appropriate care in conformity with medical scientific standards. Or perhaps they were not included. It is not possible to know from the data presented.

The article compares "alternative" visits to those to general and family practitioners, pediatricians, and internists. As before, there is no mention of obstetrics and gynecology, even though it is well-known that many women depend on their Ob/Gyn physicians as their primary doctors, or to other medical specialists who might serve as primary physicians. Considering the prevalence of "back problems," allergies, and anxiety and depression, why not include the annual number of visits to orthopedists, physiatrists, physical therapists, allergists, psychiatrists, psychologists, and clinical social workers? By including some practitioners and excluding others, the authors are able to pick their way to justifying their major conclusion that, "unconventional medicine has an enormous presence in the U.S. health care system."

The authors go on with their finding that "fewer than 3 in 10 users of unconventional therapy mention its use to their medical doctors," ignoring the fact that some of this "unconventional medicine" may even have been supplied by or at the recommendation of medical doctors. Moreover, the importance of visits to "unconventionals" is variable. A physician may not attach much importance to whether a patient had a massage, attended a Weight Watchers' meeting, or even seen a chiropractor in the previous 12 months. No one study is expected to answer all questions about an investigated matter. But what would be useful to know is how many people, given a serious diagnosis by a physician, seek out acupuncture, homeopathy, or a brown rice diet in lieu of appropriate medical or surgical treatment, or use substances that interfere with prescribed drugs? How many unnecessarily use ineffective methods for disorders that are self-limited?

"Roughly half of those who use unconventional medicine for their principal medical conditions have no supervision of this treatment by either a medical doctor or a provider of unconventional therapy." But, again, what is the significance of someone's using "relaxation techniques" or imagery in an unsupervised setting? "The use of unconventional therapy," they warned, "especially if it is totally unsupervised, may be harmful." But what is this harm? Are the 30% of people who use chiropractic on their own more liable to injure themselves than are chiropractors to injure the other 70% of those who use this form of "unconventional medicine?" Would it be safer for people having a massage, or attending a self-help group? What may truly be harmful where these methods are concerned, is their substitution for proved measures in the setting of serious or potentially serious medical conditions.

Two references were given by the authors in support of the idea that their data show a risk to the public of unsupervised "unconventional medicine." One was the 1984 report of the U.S. Congress' Subcommittee on Long-Term Health and Aging, Quackery: a $10 Billion Scandal, known as the Pepper Report for the late chair of that committee, Claude Pepper. The other is the 1987 report of the Department of Health and Human Services' Health, Information and the use of Questionable Treatment: A Study of the American Public. The adverse effects noted in these sources are primarily from methods -- herbs -- that this article's data indicate were used by only a small minority of their respondents. They also found evidence of harm caused by delay or forgoing of appropriate medical treatment because of the substitution of unproved and ineffective therapies that are deceptively promoted. The present study did not mention those problems.

The present report suggested that "medical doctors should begin to ask patients about their use of unconventional therapy whenever they take a history." Yet delay seeking medical care may arise from "spiritual healing" and "prayer" as from other "unconventional medicine." Yet the authors declined to ask patients details about about these two methods because it "would be inappropriate."

The major problem of the original survey was its exaggeration of the prevalence of questionable, unproven, disproven, and irrational health and nutrition use. The conclusions put forward in this article have been reported repeated since its publication. This has especially taken the form of a subtle term-switching device in which the curious and rather idiosyncratic "unconventional medicine" was converted to the authors' definition of "alternative medicine". The authors have not hesitated to use such language distortion. Dr. Eisenberg has argued strenuously that his data show the need for physicians to cooperate with their patients' use of whatever "alternative medicine" that they may find appealing.

On September 9, 1997, for example, the University of Texas–Southwestern and the Alternative Medicine Research Foundation of Texas sponsored a CME conference in Dallas. Dr. Eisenberg was the keynote speaker, and he used his time to review his 1990 data. He admitted that much of the "unconventional medicine" considered in the survey was essentially "extended self-care." He then, without further explanation, began referring to "alternative medicine" and focused on herbs, acupuncture, and homeopathy. There was no discussion of "relaxation techniques," which accounted for the largest use of "unconventional medicine" in his 1993 article. There was no consideration of chiropractic, massage, imagery, or "spiritual healing" or of commercial weight-loss programs. Dr. Eisenberg, at this same conference in 1997, ridiculed critics of homeopathy as having taken the position that, "it can't work, so it doesn't work?"

This paper was presented at the "Science Meets Alternative Medicine" conference sponsored by the Scientific Review of Alternative Medicine in Philadelphia on February 27th of 1999. At that time, one commentator suggested that Dr. Eisenberg could hardly help how others might misinterpret his work. But in September of 1998 Dr. Eisenberg was a co-author of another article that appeared in the Journal of the American Medical Association [JAMA 280:784-787]. The second sentence of that report stated:

We have known for several years that approximately 1 in 3 adults in the United States uses chiropractic, acupuncture, homeopathy, or one of many other treatment modalities.

The reference for this fact being "known," of course, was the 1993 NEJM article. Fortunately, this deception did not appear in Dr. Eisenberg's follow up survey, a report of which appeared last November in JAMA. But in this updated report building on the 1990 data, there was no pretense about "unconventional medicine." Non-rational methods were referred to as "alternative medicine" throughout. There was also, this time, an acknowledgement that there are "more alternative" and "less alternative" therapies among those considered in both the 1990 data and in the follow-up survey of 2055 respondents in 1997.

But there was no attempt to sort out which therapies were "more alternative" or "less alternative." There was no consideration given to what distinguishes "more alternative" from "less alternative" therapies. There was no attempt to identify whether a particular application of a therapy was consistent with facts and reason or not.

The principal data presented in the follow-up survey are shown in Table 4.

Table 4 (From Eisenberg et al, 1998; JAMA 280:1569-1575)
Type of Therapy

Used in past 12 months: 1990

Used in past 12 months: 1997
Relaxation techniques

13.1

16.3 *
Herbal medicine

2.5

12.1 ‡
Massage

6.9

11.1 ‡
Chiropractic

10.1

11.0
Spiritual healing

4.2

7.0 †
Megavitamins

2.4

5.5 ‡
Self-help group

2.3

4.8 ‡
Imagery

4.2

4.5
Commercial diet

3.9

4.4
Folk remedies

0.2

4.2 ‡
Lifestyle diet

3.6

4.0
Energy healing

1.3

3.8 ‡
Homeopathy

0.7

3.4 ‡
Hypnosis

0.9

1.2
Biofeedback

1.0

1.0
Acupuncture

0.4

1.0 *
1 or more therapy (excluding
   exercise and prayer)

33.8

42.1 ‡

* P .05; † P .01; ‡ P .001

Many of the same problems of the 1993 article apply to this updated survey. For example, relaxation therapy and massage have still not been clarified, although this was said to be warranted five years previously in the 1993 paper.

On the whole, the 1998 article is slightly less polemical that that of 1993. Given the promotional success of the earlier report, perhaps there was no need to be more emphatic. In 1998 it is merely stated that, "the prevalence and expenditures associated with alternative medical therapies in the United States have increased substantially," and that, "the magnitude of the demand for alternative therapy is noteworthy." Eisenberg and his co-authors do not speculate on why this is so, but express a sort of uncomprehending interest in their findings, "in light of the relatively low rates of insurance coverage for these services."

They do speculate that increased insurance coverage of "alternative medicine" would almost surely further increase its use. But they do not address the question of whether the increased use of "alternative medicine" since the 1990 survey has had any health benefits. On the contrary, they draw proper attention to the hazards of adverse effects, particularly in the elderly and infirm. But their data are of no help in assessing this very serious risk and no suggestions for addressing it are offered.

Elsewhere, Dr. Eisenberg has suggested that physicians "supervise" their patients' use of "alternative medicine" and even co-manage their care with "alternative medicine" practitioners. But even in the updated 1998 survey report, Eisenberg and his collaborators do not suggest that physicians make an attempt to educate their patients or try to discourage or curtail their use of unproven and ineffective methods.

Most concerning is absence of comment on major driving force behind alternative medicine - relaxed regulatory environment of the marketplace. The 1998 report mistakenly assumes "alternative" medicine use reflects demand for it and/or dissatisfaction with biomedical care.

Two years after the collection of the Eisenberg's original survey data, Congress appropriated funds for what was to become the Office of Alternative Medicine (OAM). The impetus was the will of one senator, Harkin of Iowa who used his committee chairmanship to appropriate funds because of his personal belief in bee pollen and other ineffective remedies. OAM has now been upgraded with increased funding to the Center for Complementary and Alternative Medicine.

A year after the 1993 NEJM report, the Dietary Supplement Health and Education Act (DSHEA) freed the health food industry from much of the regulatory oversight of the FDA. An avalanche of uncritical media coverage of unproved claims and practices followed. A swell of advertising dollars poured into the print and electronic media. This was a potent, and not coincidental combination. Ideologically motivated private foundations such as the $300 million Fetzer Foundation had poured millions of dollars into medical school courses and research, had sponsored the Bill Moyers' PBS series "Healing and the Mind," sponsored the Eisenberg study plus his Harvard Medical School course and annual post-graduate courses for physicians and other health professionals. These and the first Eisenberg paper itself acted to create a perception that "alternative medicine" is safe, effective, and at least as good, if not better than, scientific biomedicine.

The authors did not comment on the fact that use of acupuncturists, homeopaths, and chiropractors remained stable over the ten years, while the "alternative medicine" that drew the greatest increase in use, more than tripling between 1990 and 1997 were over-the-counter herbal medicine, folk remedies, and homeopathic products, all of which have been aggressively mass-marketed in the wake of DSHEA. Major manufacturers are rolling out herbal and homeopathic versions of their established trade-named products. Vitamin sales lagged by comparison, having been previously marketed maximally for years. On the other hand, many "nutritional supplements" combine herbs, vitamins, as well as additional ingredients and may claim to be homeopathic as well. Thus, someone's purchasing a single over-the-counter "nutritional supplement" of this kind would contribute to an increased prevalence of use of several different forms of "alternative medicine."

In 1996, under pressure from organized acupuncturists, the FDA reclassified acupuncture needles as Class II "medical devices," which permitted their use and associated claims of various health benefits. In 1997 the OAM, in association with its former interim Director, carefully orchestrated a seemingly biased panel that issued a consensus statement in favor of acupuncture. The 1998 Eisenberg article mentioned none of these things, while marveling at the public's alleged demand for "alternative medicine." The impression remains that acupuncture and "natural" products are safe, effective, and even constitute "cutting edge" medical science.

Eisenberg is silent on how these factors may have affected his updated survey information of 1997. Yet they are crucial to any evaluation. Manufacturers are now selling products which are, for all intents and purposes, medications. But, legally, as "nutritional supplements," they are "foods." Even food products are now appearing which contain herbal ingredients, such as soup containing Saint John's Wort and corn chips laced with kava.

Any serious discussion of the prevalence, let alone the medical and public health implications of "alternative medicine" will be non-productive until and unless their terminology is clarified. For if "unconventional" medicine cannot be distinguished from "alternative medicine," "complementary medicine," "integrative medicine," and quackery, then there will be nothing left to call quackery or illegal promotion. Likewise, if medicine cannot be distinguished from exercise, prayer, relaxation, self-help practices, and other "extended self-care," then there will be nothing left that will not be medicine. These issues are important not only scientifically, but also in the social, cultural, and economic context of continued medical scientific progress amid rising medical care costs. Unfortunately, Dr. Eisenberg, his collaborators, the lay and medical press and other commentators have been obscuring rather than clarifying, hindering instead of helping, honest efforts to confront them.

________________________

Dr. Gorski practices obstetrics and gynecology in Arlington, Texas, and is president of the Greater Dallas-Fort Worth Council Against Health Fraud.

Quackwatch Home Page

This article was posted on March 16, 2002.