After adding the data from the two studies Ferrari cited (as well as the seven additional studies I found), only two data points make it into the five-10-percent range at a time point relevant to performance in a grand tour. The majority of points again fall consistently below five percent and the data again does not support his claim.
Shifting from the mean (average) data, Ferrari pointed to “individual increases up to 25 percent, with four athletes over 10 percent” in Friedmann (2005), as well as “individual improvements of up to 14 percent” in (Saunders 2010), and “seven-to-eight percent in the initial set of studies” (Wehrlin 2006, Garvican 2012). He said, “the reasons for these different responses may be genetic, nutritional (adequate intake of iron and protein), or related to training loads.”
However, increases as large as eight-to-10 percent can be found even in the control group, if looking at individuals (Robertson 2010a, Saunders 2010).
Ferrari correctly responded that “the method of measurement of Hgb mass is definitely subject to errors (see article on 53×12.com).”
However, it is not correct to blame errors or factors besides altitude for the 10-percent increase in the control subject while crediting altitude alone for the 25-percent increase in the experimental subject.
In this example, Ferrari is doing the scientific equivalent of flipping a coin 100 times and only counting the result if it comes up heads. The whole point of using the mean (average) values from large numbers and control groups is to try to get rid of the influence of noise on individual results so the real effect can be seen.
We followed up with two very direct questions:
“First, do you not consider the average value of the cohort as stronger evidence than an individual case?”
“Do you not consider studies with control groups to be stronger evidence than studies without a control?”
Ferrari responded, saying, “obviously from a scientific and statistical point of view, the average behavior of a group of subjects is more significant than individual behavior. But as a physician evaluating the individual patient, I have to further study the case if I did measure an effect on the individual, before ruling out that this cannot be true because it’s not ‘statistically significant.’ In the case of the effects of altitude, it is certain that there are ‘responders’ and ‘non-responders,’ as evidenced first by the study of Levine, as well as being confirmed by anyone who has experience as a trainer on the field.”
His response raised two final questions in this thread:
“Do you have data showing Armstrong’s response to altitude?”
“Do you have data showing Armstrong’s response to EPO?”
We also let him know that, “if not, [we] have to point out the problem in using the individual data points to suggest Armstrong was a super responder to altitude just as it would be wrong to use individual results to suggest he was a super responder to EPO.”
Ferrari did not respond further to this thread.
In the absence of Armstrong’s personal data, it is impossible to know where he might fall on the spectrum of altitude response. Without Armstrong’s data, credible science would consider the means (averages) of all the data at each time point to minimize the effect of uncontrolled variables, as we’ve done in Figure 3.
FILED UNDER: Commentary