Improving Access to Psychological Therapies (IAPT) in the UK
A good example of the reporting of data from the Improving Access to Psychological Therapies (IAPT) national treatment initiative in England. If you read this here you’ll see how the effect size of treatment is reported.
Caveat
Effect size can be both important and helpful. However, we add the caveat that nothing should ever be considered in isolation. Effect size is one important aspect of assessing impact of our care but there are many others; the individual journey of any particular client won’t be reflected in effect size. For this it’ll be far better to consult that particular client’s chart.
Standard deviation
Standard deviation is a measure of how spread out a collection of numbers are. The formula is the square root of the variance of the numbers. The variance is defined as the average of the squared differences from the mean of the numbers. Note that we use the 'sample' standard deviation formula where we subtract 1 from the total number (N) of measures available. The pooled standard deviation is then calculated from the pre-treatment and post-treatment standard deviations.
Pooled standard deviation
The pooled standard deviation is in fact the average of the pre-treatment and post-treatment standard deviation. Pooling in this way has the effect of typically increasing the size of the denominator which results in a more conservative treatment effect.
Effect size - Cohen's D & Hedges' g
We provide two different calculations for effect size because these are the two most common ways effect sizes are reported in published papers. The Hedges' g calculation has a correction factor for smaller sample sizes to prevent inflation of the result.
The Hedges' correction factor
When sample sizes are small this factor also has the effect of reducing the effect size to make the result more conservative. Typically, you'll notice that once there are around 20 or more cases the result will be the same regardless of which effect size calculation is used.
Why we explain the numbers
We feel that it's important that anyone can replicate our findings by using the formulas published here on this help page so you know exactly what the numbers actually mean.
Measuring the effect of treatment
Measuring the ‘effect’ of our treatment is important, particularly at the service or organisation level, when we have higher numbers of clients and we can aggregate their scores. Higher numbers will always be a more reliable indicator of the value of treatment and having some way of indicating this allows us to look at our results compared to how results are commonly reported in academic literature.
A choice of methods
We choose to use two different effect size calculations in Pragmatic Tracker; Cohen’s d and Hedges’ g, because these are commonly used as the method of reporting effect size.
Cohen’s d or Hedges’ g?
We offer two methods simply because Cohen’s D is better generally for larger sample sizes (more than 50 clients) where Hedges’ g is better for much smaller sample sizes (less than 20 clients). However, you’ll notice that often these numbers come out the same.
How we calculate the effect sizes
In Pragmatic Tracker we include all scores that a client has filled in for any particular measure where there have been at least 2 measure scores recorded for the same measure. Typically, this will be the 1st time a client filled in a measure and the last available time they filled it in.
