When I first started www.cpnhelp.org one of my dreams was to be able to gather some real data about users experiences and results on cpn protocols. I know from my training as a psychologist that having such a ready data pool is a priceless opportunity. It would allow us to go beyond mere anecdote to some real, statistically relevant results. I was joined in this by Marie Rhodes whose training and intelligent understanding of medical research gave her an interest in gathering some "hard" data, i.e. medically sanctioned measures such as EDSS scores, blood results and the like.
But when I started the site 33 weeks ago and put up the first survey, we had only 4 people who had met the three month criteria and could fill out the survey. It took us 8 months to make for reasonable numbers that I felt were worth reporting on. The results of that survey can be found at Survey #1 Results.
Since that report, I've seen others referring to it, like me grateful to have some kind of data they can point to. We aren't just going on our personal impressions any more.
Having been worried about the lack of data, now I'm getting worried about how data is being used and misused. In case you haven't gathered by now, I'm a worrier by nature. This could be a genetic defect on my part. You've heard the joke about the Jewish mother who gives her son two ties for his birthday? The next morning he comes down proudly wearing one of them, and she says "What's the matter, you didn't like the other one?"
My worries- that these very preliminary results will be referred to as firm findings. The data group is so small (23) that anything in this survey is merely suggestive. And barely that.
This is even more true with any results derived from breaking down findings into subgroups, like the MS/CFS-FM groups. In that case we have even tinier groups, and less validity to draw on. Such fndings are even less generalizable because each group is now so small that any large differences by any one member within the group will skew the results of that group.
I almost regret having broken the results down this way, especially with those damned charts of which I was so proud, because the charts make everything look so official. Slick presentation impresses us beyond the actual validity of the data itself.
However, our numbers are growing every day. I put up a poll up recently to get a clearer idea of how many people are actually doing the protocol: 35, plus 3 people additional actually done with the thing! Thirty-eight is not bad. When we hit 50+, we might even have some data worth doing statistical analysis on. Getting closer to that every day.
The results of this first survey have inspired Marie and I to take the next step, and build a survey which is much more detailed and will include some hard measures of functioning in it as well. It will take time to develop, but it's coming.
In the mean time, I want to ask everyone to please be cautious about how you refer to any findings of the current survey. If you refer to these findings, please, please use qualifiers like "It looks like..." and "preliminary survey results have suggested..." and the like.
It's good to be enthusiastic about the CAP's and your own personal experience, but be tentative about how firmly you make claims based on this data. Make sure newbies know that, while the survey found one thing, their own results may differ, and that there is a big range of response within the survey reports.
Thanks to everyone who contributed to Survey #1. I'll be relying on you all to provide even more data next time!