There are many methods of identifying change in treatment according to different measures. Some proprietary measurement systems use algorithms based on very large normative data samples. Others use what is known as 'reliable change indices'. More consider 'moving to recovery' rates. More again consider measurement of 'effect size' as being the important point to highlight. All of these are valid and have their place.
However, in our single user system, since we have many different measures available all with their own characteristics, we decided to use a very simple traffic light system based on the percentage reduction in distress from where one starts. This provides a simple, useful clinical guide to whether a client is moving in the right direction - by a lot, by a bit or hardly at all or deteriorating.
For a more complete explanation of how the calculations work read the small print below.
The system also gives give alerts with respect to appointment times to flag clients who have not been seen for a while. Amber shows clients who have not been seen for 30 days - changing to red after 60 days.
On any particular measure, a green signal indicates that there has been a greater than 25% change in reduction of distress from the initial score. An amber signal indicates a between 6% and 25% change and a red signal indicates a change of 5% or less, or, indeed, deterioration. In spite of the fact that our system is basic and very simple we wanted to make sure that the calculations on any instrument were based on movement in the same direction. As MOST measures reduce score with improvement we adopted the strategy of always measuring REDUCTION in distress to create consistency.
Lets take an example. GAD-7 measures anxiety. It has a maximum score of 21 (maximum anxiety) and a minimum score of 0 (no anxiety). A client attends with an initial score of 16. By the 3rd appointment the GAD-7 score is now 12. There has been a 4 point improvement. 4/16 = 1/4 = 25% (amber signal).
Lets take an example. SWEMWBS measures wellbeing. It has a minimum score of 7 (least wellbeing) and a maximum score of 35 (most wellbeing). A client attends with an initial score of 14. By the 3rd appointment his score is shifted to 21. Because the scores go up with improvement we need to calculate the DIFFERENCE between the scores and the maximum score at each time point. So, the initial score of 14 is subtracted from 35 (35-14 = 21). Then the score at the 3rd session is subtracted from the maximum of 35 (35-21 = 14). The REDUCTION in distress is from 21 to 14. This represents a 7 point improvement. 7/21 = 33.3% (green signal)
We do not make any claims about scientific rigour with this basic system. It is what it intended to be, a simple flag of whether a client is moving in the right direction or not on any particular measure. Please don't get hung up on the % numbers. The actual measure scores are right there to look at anyway Remember, your data exports to excel and if you wish to analyse your data according to other criteria you can of course do so.
Measuring change through questionnaires is not the same as measuring body temperature using a thermometer. Scientists argue amongst themselves as to how to calculate effect sizes and as to what constitutes reliable improvement. This can be confusing and frustrating. We've chosen a very simple way of looking at whether change is happening or not. We fully recognise that change characteristics of different instruments vary but we're confident that our approach provides a simple, user-friendly, and, dare we say it, 'pragmatic' clinical tool.
We've adopted this straightforward approach in the single user systems for simplicity. However, for any customers wishing to have traffic light alerts based on different criteria, such as algorithms or reliable change indices, we will build to your specification. Please contact us for more information.