Trying to confirm what you already believe is a dangerous practice
By Cary Gutbezahl, MD
Recently published research in the Archives of Internal Medicine has attracted attention because it concludes that small gifts influence physician choice of medications. I am certain that this study will be used to justify future public and organizational policy decisions.
The focus of my comments is not related to influence on physician decision making. The issue is the misuse of research findings and extending conclusions beyond the conditions of the research. That this research is publicized is evidence of the failure of our educational system! It’s time to stand up and scream “Foul!”
This research used compared medical students, not physicians, at two medical schools that had different policies about small gifts from pharmaceutical companies. Based upon psychological tests of positive and negative associations, not prescribing patterns, the researchers concluded that even small gifts influence opinions.
There are a lot of reasons to believe that there are significant flaws in drawing conclusions about practicing physicians from this study.
First, the subjects were students not practicing physicians. Relying upon them as representative of physicians would be like assessing how well students can perform laparoscopic cholecystectomies and concluding that physicians are not skilled enough in performing surgery. Is there anyone who does not recognized that medical students differ from practicing physician? Medical students are not engaged in “self-learning” and evaluating journal articles or practice guidelines. They are passive recipients of information. Physicians learn how to become life-long learners in their residencies. In addition, medical students have not gone through the socialization processes that are part of post-graduate medical education that prepare the physician to have a more questioning attitude to what they are told. And medical students are in an economically different place than practicing physicians and might be subject to greater influence by lower cost items (determining whether that’s true is a research study, in itself).
Second, if that were not enough to debunk this study, a fundamental flaw of experimentation was overlooked. Only two medical schools were studied. While these schools differed in their policy on small gifts, it’s very likely that they varied in other characteristics also. For example, faculty attitudes toward drug expenses might be different (which would align with the differences in policies). The problem is that there may be many unknown dimensions in which the two schools differed. To overcome the unknown differences, one needs to study students from more schools. Even that might not be sufficient, depending upon what the data reveals.
More than likely, the researchers had their minds made up before they conducted the research. That is, they were biased. When researchers are biased, it influences the quality of their research design, their analysis of data and the conclusions they draw. Ultimately, their research is not research, but rhetoric. Research requires skepticism.
The same is true in management. When management seeks to confirm what they believe to be true, they begin a path down a road that leads to missing important warning signs. Good research and good management research challenges pre-existing belief. Trying to confirm what you already believe is a dangerous practice. Instead, try to disprove what you believe to be true.
Filed Under: Clinical Improvement • Featured Articles
Case studies to learn more about ways in which Compass Clinical has worked to create better American hospitals.