Jump to comment:

• Statistical problems in the vitamin C and common cold trial with South Korean army recruits
  Harri Hemilä
  Published on: 25 March 2022

• Published on: 25 March 2022

  Statistical problems in the vitamin C and common cold trial with South Korean army recruits

  • Harri Hemilä, Adjunct professor University of Helsinki

  The report on vitamin C for preventing the common cold in the Republic of South Korea army recruits by Kim et al. [1] has several statistical problems.

  First, Kim et al. did not follow the intention-to-treat [ITT] approach. Figure 1 shows that 49 participants were excluded because they “stopped intake of vitamin C”, and 84 participants were excluded because they “stopped intake of placebo” [1]. The CONSORT recommendation for ITT analysis states as follows [2, Box 6]: “participants who … did not take all the intended treatment ... exclusion of any participants for such reasons is incompatible with intention-to-treat analysis”.

  Second, Altman et al. pointed out that “The odds ratio should not be interpreted as an approximate relative risk [RR] unless the events are rare in both groups (say, less than 20-30%)”[3]. The common cold is not rare. Over 50% of the participants in the Kim et al. trial had the common cold during the trial period which greatly exceeds the 20-30% limit. Furthermore, there is no need to use the OR as the approximation for RR, because the RR can be calculated from the trial data in Table 1, RR = 0.916 (= 0.538/0.587) [1].

  Third, in their abstract, Kim et al. wrote “the vitamin C group had a 0.80-fold lower risk of getting a common cold” implying that vitamin C decreased the incidence of colds by 20%. However, the correct effect estimate is given by the RR above, which indicates only 8.4% lower risk of colds in the vitamin C group....

  Show More

  The report on vitamin C for preventing the common cold in the Republic of South Korea army recruits by Kim et al. [1] has several statistical problems.

  First, Kim et al. did not follow the intention-to-treat [ITT] approach. Figure 1 shows that 49 participants were excluded because they “stopped intake of vitamin C”, and 84 participants were excluded because they “stopped intake of placebo” [1]. The CONSORT recommendation for ITT analysis states as follows [2, Box 6]: “participants who … did not take all the intended treatment ... exclusion of any participants for such reasons is incompatible with intention-to-treat analysis”.

  Second, Altman et al. pointed out that “The odds ratio should not be interpreted as an approximate relative risk [RR] unless the events are rare in both groups (say, less than 20-30%)”[3]. The common cold is not rare. Over 50% of the participants in the Kim et al. trial had the common cold during the trial period which greatly exceeds the 20-30% limit. Furthermore, there is no need to use the OR as the approximation for RR, because the RR can be calculated from the trial data in Table 1, RR = 0.916 (= 0.538/0.587) [1].

  Third, in their abstract, Kim et al. wrote “the vitamin C group had a 0.80-fold lower risk of getting a common cold” implying that vitamin C decreased the incidence of colds by 20%. However, the correct effect estimate is given by the RR above, which indicates only 8.4% lower risk of colds in the vitamin C group. Thus, using the OR as an inappropriate measure of effect gave a 2.4-fold multiplication of the actual percentage benefit from vitamin C administration. Furthermore, Table 1 shows that the 8.4% difference between the vitamin C and placebo groups was not statistically significant with P = 0.059 [1].

  Fourth, standard textbooks on controlled trials are critical about adjustments of the effect estimate by numerous baseline variables, since “it is often possible to select specific covariates out of a large set in order to achieve a desired result” [4, p. 368]. Kim et al. used 15 variables to adjust the effect of vitamin C in their Table 2 [1]. Furthermore, 4 of those 15 variables were not baseline variables, as they were determined at the end of the trial: 1) “Do you think vitamin C intake helps prevent diseases such as a common cold? (End point)”, 2) “Do you prefer to take vitamin C? (End point)”, 3) “Did you experience physical vitality such as decreased fatigue after taking vitamin C? (End point)” and 4) “Intent to continue intake Vitamin C”. Therefore, they are unambiguously inappropriate to be used as baseline adjustment variables. Thus, the marginally significant P-value calculated for the over-adjusted statistical model in Table 2 (P = 0.0419) is statistically much less sound than the non-significant P-value calculated in Table 1 (P = 0.059) [1].

  Fifth, at the start of the trial, Kim asked the participants the question: “Do you think vitamin C intake helps prevent diseases such as common cold?” and 1098 participants agreed or strongly agreed. At the end of the trial, only 320 participants agreed or strongly agreed in response to the same question. Thus, participation in the one-month trial led to a 71% decline in the opinion that vitamin C might prevent diseases such as the common cold. Half of the participants were given placebo and if all the placebo participants had lost their belief in vitamin C because of their personal experience of not getting any benefit, that would explain a decrease by only 50%. This implies that a great proportion of the vitamin C participants also changed their mind when they had actually taken 6 g/day of vitamin C for 30 days.

  In parallel with the dramatic decline in the “agree” responses, there was an even more dramatic increase in the “disagree” responses to the same question. At the end of the trial, the “disagree” responses were 12 times higher than at the start (365 vs. 30) and the “strongly disagree” responses were 33 times higher than at the start (236 vs. 7) [1]. These dramatic changes towards negative opinions after having taken high doses of vitamin C for one month were not discussed by Kim et al.

  Sixth, in the text section, the first paragraph of the Results, Kim et al. wrote “895 subjects were smokers …... 99 subjects were never smokers”. However, Table 1 states “never smoker: 895” and “former smoker: 99” and there is no category for current smokers in that table [1]. Evidently, one of those has been transposed but we do not know which one. The confusion in the smoking classification also makes the reader doubt the validity of the statement that the effect of vitamin C was stronger “among never smokers”. Does that refer to 99 or to 895 participants?

  Finally, Kim et al. do not describe the background of the topic in sufficient detail to highlight the nuances of those findings. In 1996, a meta-analysis of 3 trials with 475 participants under short-term acute physical stress calculated that vitamin C decreased the risk of the common cold by RR = 0.50 (95% CI 0.35-0.69, P = 0.00003) [5]. Two later trials with similar participants found consistent results so that, after the inclusion of the two later trials, the estimate of effect was RR = 0.48 [6].

  The 1996 paper stated [5] “One study was identified in which the experimental conditions are quite close to those in the [three] studies ... Pitt and Costrini carried out a randomized double-blind study with military recruits in a training camp in South Carolina [7]. They administered 2 g/day of vitamin C to the study group, but there was no difference (0%) in common cold incidence when compared to the placebo group. There were over 1200 common cold episodes in the study… and thus this study has great weight as regards the possible role of vitamin C in subjects under heavy stress. Nevertheless, there are several noteworthy differences between the Pitt and Costrini study and the three studies... Pitt and Costini's subjects were under a regular training program, the study lasted for 2 months and, furthermore, the tablet administration did not begin until their third week at the training camp. In contrast, the subjects of the [three] studies ... were under acute and unusual stress and the studies lasted for just few weeks”.

  The Pitt and Costrini trial with US Marine recruits was large and recorded 1219 common cold episodes. The confidence interval around the null effect was very narrow with RR = 1.00 (95% CI: 0.90-1.12) [7,8]. This negative result with the narrow confidence interval is inconsistent with universal benefits of vitamin C supplementation for army recruits, but the US Marine recruit study was dismissed by Kim et al. Although there is strong evidence that vitamin C decreases common cold incidence in people under short-term physical stress [5,6], that benefit does not seem to extend substantially to conditions of long-term physical stress. Possibly the body may adapt more efficiently to regular stress than to acute stress [5].

  References

  [1] https://doi.org/10.1136/bmjmilitary-2019-001384
  [2] https://doi.org/10.1136/bmj.c869
  [3] https://doi.org/10.1136/bmj.317.7168.1318
  [4] https://doi.org/10.1007/978-3-319-18539-2_18
  [5] https://doi.org/10.1055/s-2007-972864 and https://helda.helsinki.fi/handle/10138/225881
  [6] https://doi.org/10.1002/14651858.CD000980.pub4 and https://helda.helsinki.fi/handle/10138/225864
  [7] https://doi.org/10.1001/jama.1979.03290350028016
  [8] https://doi.org/10.1017/S0007114500002889

  Show Less

  Conflict of Interest:
  None declared.

  • Back to top