Many journal authors have the bad habit of looking just at the p-value of a study and ignoring the clinical importance of their findings. If they get a small p-value, which indicates a statistically significant difference between the new therapy and the standard therapy, they dance in the streets, they pop open the champagne bottles, they celebrate wildly, and they publish their results in an "A" journal. If they get a large p-value, they rend their clothes, they throw ashes on their heads, they wail and moan, and they publish their results in a "C" journal.
An article about measurement of fatigue
- Measurement of Fatigue Determining Minimally Important Cllinical Differences. Schwartz AL, Meek PM, Nail LM, Fargo J, Lundquist M, Donofrio M, Grainger M, Throckmorton T, Mateo M. Journal of Clinical Epidemiology 2002: 55(3); 239 - 244. [Medline]
offers some valuable lessons about clinically relevant differences.
Cancer patients have major problems with fatigue. The only good measure is a self-report, and this can be measured in several different ways:
- Profile of Mood States, a 65 item scale with a subscale of five items representing fatigue. Each item is rated from 0 to 4.
- Schwartz Cancer Fatigue Scale, a 28 item scale with four subscales: physical, emotional, cognitive, and temporal. Each item is rated from 0 to 4.
- General Fatigue Scale a ten item scale with no subscales. Each item is rated from 1 to 10.
- A ten point single item scale.
The last scale asked the question "what is your level of fatigue today" with 0 representing "no fatigue" and 10 representing "the greatest possible fatigue." There's a slight error here, because if you count properly, there are 11 numbers in the range from 0 to 10.
The researchers measured a group of 103 cancer patients before and after initiation of chemotherapy. In addition to getting the four scales, the patients were asked at follow-up whether their fatigue levels had changed and by how much. Interestingly, 30 subjects reported a decrease in fatigue, but the average scores on all four scales for these patients did not differ from their peers who reported no change in fatigue. Those who reported an increase in fatigue did differ from those reporting no change. What this means is difficult to interpret, but the authors feel that patients may perceive increases in fatigue differently than decreases in fatigue.
If you look at the average change in each scale for those patients who report a small change in fatigue, this represents a minimally important clinical difference. The numbers don't seem to quite match the tables, but the authors suggest that a 5.6 unit shift in POMS, 5.0 for SCFS, 9.7 for GFS, and 2.4 for the single item scale. If you divide each of these values by the number of items in the scale, you get values that hover around 1.0 for the first three scales, which is similar to a recently published paper in BMJ.
- Interpreting treatment effects in randomised trials. Guyatt GH, Juniper E, Walter S, Griffith L, Goldstein R. British Medical Journal 1998: 316(7132); 690-693. [Medline] [Full text] [PDF]
- Clinically significant changes in pain along the visual analog scale. Bird SB, Dickson EW. Ann Emerg Med 2001: 38(6); 639-43. [Medline]
- The visual analog scale for pain: clinical significance in postoperative patients. Bodian CA, Freedman G, Hossain S, Eisenkraft JB, Beilin Y. Anesthesiology 2001: 95(6); 1356-61. [Medline]
- How well is the clinical importance of study results reported? An assessment of randomized controlled trials. Chan KB, Man-Son-Hing M, Molnar FJ, Laupacis A. Cmaj 2001: 165(9); 1197-202. [Abstract] [Full text] [PDF]
- Setting the minimal metrically detectable change on disability rating scales. Hebert R, Spiegelhalter DJ, Brayne C. Arch Phys Med Rehabil 1997: 78(12); 1305-8. [Medline]
- Interpreting thresholds for a clinically significant change in health status in asthma and COPD. Jones PW. Eur Respir J 2002: 19(3); 398-404. [Medline]
- Quality of life questionnaires: does statistically significant = clinically important? Juniper EF. J Allergy Clin Immunol 1998: 102(1); 16-7. [Medline] [Full text] [PDF]
- Does the clinically significant difference in visual analog scale pain scores vary with gender, age, or cause of pain? Kelly AM. Acad Emerg Med 1998: 5(11); 1086-90. [Medline]
- A proposal to use confidence intervals for visual analog scale data for pain measurement to determine clinical significance. Mantha S, Thisted R, Foss J, Ellis JE, Roizen MF. Anesth Analg 1993: 77(5); 1041-7. [Medline]
- Can there be a more patient-centred approach to determining clinically important effect sizes for randomized treatment trials? Naylor CD. J Clin Epidemiol 1994: 47(7); 787-95. [Medline]
- Determining the minimum clinically significant difference in visual analog pain score for children. Powell CV, Kelly AM, Williams A. Ann Emerg Med 2001: 37(1); 28-31. [Medline]
- Assessing clinically significant change: application to the SCL-90-R. Schmitz N, Hartkamp N, Franke GH. Psychol Rep 2000: 86(1); 263-74. [Medline]
- Clinical utility and clinical significance in the assessment and management of pain in vulnerable infants. Stevens B, Gibbins S. Clin Perinatol 2002: 29(3); 459-68. [Medline]
- What is the relationship between the minimally important difference and health state utility values? The case of the SF-6D. Walters SJ, Brazier JE. Health Qual Life Outcomes 2003: 1(1); 4. [Medline]
- Minimum clinically significant VAS differences for simultaneous (paired) interval serial pain assessments. Yamamoto LG, Nomura JT, Sato RL, Ahern RM, Snow JL, Kuwaye TT. Am J Emerg Med 2003: 21(3); 176-9. [Medline] [Abstract]
You can find an earlier version of this page on my original website.