This assignment is based on a recent study at Beijing Sport University on the effects of altitude training on 3000-m treadmill running performance and VO2max. The study was a controlled trial, but I have used the means, standard deviations and correlations in the data to simulate a simpler uncontrolled time series, of the kind that a sport scientist might have to analyze for a squad of athletes. I have changed some of the details of the design for this assignment, but I am grateful to my colleagues at BSU, especially Dr Bing Yan, for permission to use the descriptive data.
The analyses are similar to those in last year’s assignment. I have generated fresh data, but from the same simulation spreadsheet I used last year (not available for downloading). In other words, it’s like repeating the study with similar subjects and the same underlying effects on running performance and VO2max, hence differences in the effects from last year are all due to… sampling variation!
Download Assignment 7 data.xlsx. You will see that the data consist of repeated measurements of 3000-m times and VO2max in 13 athletes. The two pre-tests were at sea level. A few days later, the athletes went to a live-high train-high altitude camp for 4 wk. A few days after returning they performed the first post-test, then lived in the altitude house at BSU for 3 wk of live-high train-low training before performing another post-test. They resumed normal living and training for 4 wk before the final post-test. Your task is to quantify the mean changes and individual differences in the changes in time-trial performance, and to determine the extent to which the changes in time-trial performance were explained by changes in VO2max.
VO2max would be a bit of an irrelevant luxury in an applied sport setting, but if you want to build a career as a sport researcher, measuring and analyzing potential mechanism (mediator) variables is a smart move, especially if you don’t have a control group. Evidence of a physiological, psychological or biomechanical mediator of a performance change is evidence that at least part of the effect of the intervention is not due simply to the athletes making a bigger effort in the post-test. You, the athletes, the coach, and reviewers of any manuscript you submit to a good journal, will therefore have more confidence that the effect is “real”. The evidence is not so good if the mediator itself could show a bigger change with a bigger effort. VO2max is not perfect in this regard, although in principle you get the same value for near-maximal efforts as for maximal efforts. A submaximal measure, such as exercise economy or fractional utilization (or its surrogate, anaerobic threshold), is better for endurance performance. The evidence for a mediator takes two forms: a change in the mean of the potential mediator in the right direction to account for the change in performance, and individual changes in the potential mediator tracking individual changes in performance. We’ll see both here.
Evidence for a change in the mean of the potential mediator requires a smallest important change in the mediator, which here is the change associated with the smallest important change in performance. Usually you don’t know this smallest important change in the mediator. You may be able to derive a value from theory; for example, other things being equal, a 0.3% change in VO2max must translate into a 0.3% change in running time-trial time (the smallest important change). Unfortunately, other things may not be equal: your athletes might increase their VO2max, but that increase might be offset to some extent by reductions in economy or fractional utilization, so maybe you need a 0.6% change in VO2max with your intervention to get a 0.3% change in performance, on average.
You can get evidence of tracking of changes in the potential mediator and performance only if there are individual responses to the treatment. Make sure you “get” this point. The individual responses themselves, expressed as a standard deviation, don’t have to be clear, but the clearer the relationship between them (those of performance and the potential mediator), the better the evidence for individual responses. The relationship is simply the slope between the change scores, and that slope itself can provide you with the smallest important change in the mediator. For example, if the percent per percent slope is 1.0, then the smallest important change in the mediator is the same as that for performance. But if the slope is 0.7 %/%, a 0.7% change in performance requires a 1% change in the mediator, so a 0.3% change in performance requires a (1/0.7)*0.3 = 0.43% change in the mediator. (You have to think this through yourself.) Fine, except that there is another “unfortunately”: this slope is reduced (“attenuated”) by noise in the potential mediator, so the observed slope tends to be less than the true slope. Correcting for the attenuation is a bit tricky: you divide it by the intraclass correlation coefficient (ICC) for the change score of the mediator. The ICC is (SD2-2e2)/SD2, where SD is the standard deviation of the change scores and e is the typical error from a reliability study of the mediator. (The factor of 2 comes from the fact that the error in a change score is Ö2 times the typical error.) Doing this correction is way too complicated for this assignment, but you will derive the %/% slope, and you will see that it is fairly obviously attenuated.
Now download the post-only crossover spreadsheet. Copy-insert an extra column for the raw data and for the log-transformed data in Sheet1, then repeat exactly for the Time-series deltas sheet. Now delete 7 rows to accommodate the 13 rows of data, and immediately delete the same 7 rows in the Reference and Time-series deltas sheets. Now copy and paste the VO2max data, the values for time in weeks of the tests, and the subject identities (just numbers). If you paste just the values, you will keep the nice blue formatting. Save the spreadsheet as VO2max analysis.xlsx or similar, then Save as… Time-trial data.xlsx or similar, and copy-paste in just the time-trial times.
Both spreadsheets are set up with the default custom effect Post1 minus the mean of Pre1 and Pre2, which is fine, but you will be changing the custom effect, and you will eventually want to determine the mediating effect of the change in VO2max on the change in time-trial time. So here’s what you do next. Highlight and copy the five custom-effect cells above the raw data in the time-trial sheet, open the VO2max sheet, click on the first of the custom-effect cells above the raw data in that sheet, then Paste Special… and click on the Paste Link button in the bottom left of the window. Now, whenever you choose a custom effect for a change in time-trial times, you will automatically get the same custom effect for the corresponding change in VO2max. But you will want the values of that custom effect in VO2max to be pasted into in the X1 column of the time-trial sheet, to allow you to quantify VO2max as a mediator. So while you have the VO2max sheet open, go across to the AD column, highlight and copy AD52 to AD64, switch back to the time-trial sheet, then Paste Special… Paste Link into the first cell of X1 (Cell D52). The values of X1 are now the percent changes in VO2max for each athlete, so change the name of the variable to dVO2max or something similar (not too long, or it doesn’t fit in some cells properly). This variable is used at the end of the assignment.
Finally, the threshold value for the smallest important harmful effect on time-trial time… You should be able to do that, but if you get it wrong, you’ll get wrong answers for most of the assignment. So put the value 0.3 in the percent cell. At this stage don’t insert any smallest important values in the VO2max sheet.
- The smallest important change in time-trial time of 0.3% is for top middle-distance runners, who are in contention for medals at world champs or Olympics. Most of the runners in this study would have been at a lower level. Why do we use a smallest important for top athletes, even though we are studying sub-elite athletes? xxxx
- If we did want to use a smallest important that applied to sub-elite runners, how would we define it, and how would we get it. xxxx
- With the mean of Pre1 and Pre2 as baseline (the default setting), what is the effect of the altitude camp on 3000-m time-trial time, expressed as an MBD? Show in this format: x.x, 90% confidence limits ±x.x units; qualitative magnitude, qualitative probability of substantial or trivial (or unclear if it’s unclear): xxxx
- Show a similar MBD for the individual responses, but show the confidence interval as lower and upper limits: (Hints: thresholds for SDs are not the same as for mean changes; and the MBD probability is shown lower down in the panel of cells for the ratio of groups SDs.)
xxxxx
- The scatterplot for the custom effect vs X2 not adjusted for dVO2max shows the moderating effect of baseline in this sample. Put the appropriate value of the baseline (X2) into Cell AE44 to get the effect of altitude adjusted to baseline.
What is the adjusted effect and confidence limits: xxxx
(You should find little or no change from the unadjusted effect next door and in Question 3.
Take a look at the slope of the line in the graph of custom effect vs X2 not adjusted for dVO2max. Complete this sentence: The slope shows that the effect of the altitude camp on runners with slower baseline time-trial times tended to be… xxxx
What would you normally expect for the effect of baseline, with regression to the mean:
xxxx
Ah yes, but is the modifying effect of baseline clear? Evaluate the effect of 2SD of baseline by putting the appropriate value in Cell AE47. Show x.x, 90% confidence limits ±x.x units; qualitative magnitude, qualitative probability of substantial or trivial (or unclear if it’s unclear):
xxxx
Note that a proper analysis for the potential modifying effect of baseline would require a control group, especially if there was sufficient noise in the dependent variable to produce a substantial effect of regression to the mean.
NOW CLEAR CELL AE47 to stop getting the modifying effect of baseline.
Then scroll down and you will see this:
Adjusting for X2 (baseline) is having no effect on the mean (-0.9), but it makes the precision of the estimate a bit worse (±1.23 rather than ±1.22). If baseline or any other covariate was having a substantial effect, you would expect an improvement in precision when the effect is adjusted for the covariate. Taken together with the fact that we get an unclear effect of baseline, and going in the wrong direction (we expect bigger benefits for subjects with worse baseline performance), it’s reasonable not to adjust for baseline for the remaining analyses. This decision also means that, when we adjust for changes in VO2max, we won’t need the cells for adjusting for both predictors. So, clear the mean value of X2 in AE44 (and if you had something else there, get it right and re-do Question 5).
- Repeat Questions 3 and 4 for the effect immediately following the live-high train-low period compared with baseline (the mean of the two pre-tests).
Mean effect: xxxx
Individual responses: xxxx
- You cannot easily estimate the uncertainty in each athlete’s change in performance, but you can identify those athletes who appeared to benefit most. What were the three largest changes in performance at the end of the live-high train-low period, in descending order, in log units (approximate percents): xxxx
Now use the back-transformation formula 100*exp(x/100)-100 to express these changes as exact percents: xxxx
- Two athletes had impairments in performance at the end of the live-high train-low period. What were the impairments, as approximate (almost exact, actually) percents: xxxx
- Now, let’s look at the effects on VO2max. First, the values for X1 in this spreadsheet have nothing to do with these subjects, so ignore the graphs showing custom effect vs X1. Do look at the graph custom effect vs X2 (baseline). The line is practically flat, so there is apparently no substantial modifying effect of baseline, regression-to-the-mean or otherwise. Try adjusting to baseline by putting the mean value in AE44 and all you will get is a slight worsening of precision. So, let’s not adjust for baseline.
State the mean effect and confidence limits for the effect of the altitude camp plus live-high train-low training on VO2max: xxxx
- You can’t do an MBD for the effect on VO2max, because you don’t yet know the smallest important change in VO2max. Return to the time-trial sheet and check out the scatterplot of the custom effect vs dVO2max not adjusted for X2. What is it about this graph that shows that VO2max could be a mediator: xxxx
- To quantify the potential mediating effect of the change in VO2max on the change in 3000-m time-trial time, adjust the effect of altitude-camp plus live-high train-low training to zero change in VO2max (using Cell AD44). The spreadsheet will now tell you the effect of training that is independent of State its MBD: xxxx
- How much of the effect of training was due to VO2max? To find out, put the mean change in VO2max in the appropriate cell (AD47). State its MBD: xxxx
Check: this effect plus that in Question 11 should equal the mean effect in Question 6 (the value you are currently seeing right next door in Cell AE97–the slight difference is just due to rounding). The effect due to VO2max is underestimated, if the slope is attenuated by noise. So, what’s important here is that the clear contribution from VO2max is evidence for mediation of individual responses that makes the mean effect in the study less likely to be due to a placebo effect.
- For the percent per percent relationship between change in VO2max and change in 3000-m time, put an appropriate value in Cell AD47 to get the value and its confidence limits. You will need to get the spreadsheet to give an extra decimal place: xxxx
Note that the usual %/% value is around -0.5. That is, a 1% change in VO2max gives a -0.5% change in running time-trial time. Assuming -0.5 is the slope after correction for attenuation, what would be the smallest important harmful change in VO2max (without confidence limits): xxxx
Put this harmful value into the VO2max sheet and report the clinical MBD for the mean change in VO2max at Post2 vs baseline (mean of Pre2 and Pre2).
First, what are the thresholds for moderate, large, and very large: xxxx
Now the MBD (you already have the mean and confidence limits in Question 9:
xxxx
There is reasonable evidence from the mean change in VO2max that VO2max is a mediator.
A final note… I could have asked you questions about the extent to which the benefit of altitude training was still present in the last post-test. Check out the plot of mean values and change scores to see what is going on. The change between baseline and Post3 is clear, but there is zero change in the mean between the last two tests, so you are bound to get an unclear effect for the change between these tests with this sample size. So you can’t make a definitive conclusion about the effect of altitude training not declining 4 wk after the last altitude exposure, but obviously you can say that there is still a clear benefit at that time, assuming that the athletes did not change their usual training.