Nitpicking Sloppy Science
April 30, 2006 by Do'C 
Printer-Friendly Version
Readers may remember that I originally asked the journal Medical Hypotheses to consider contacting the study authors of “Hyperbaric oxygen therapy may improve symptoms in autistic children” and publishing an erratum about the overstated treatment pressure. That didn’t go far with Medical Hypothesis (they don’t do peer review). I subsequently sent the request to the study authors.
I received a prompt and courteous reply from Dr. Rossignol.
Hi,
Our HBOT pressure gauge reads about 4.15 PSI. Furthermore, the average measured atmospheric pressure where the chamber is located is 1.003 to 1.006 ATA. Therefore, this converts to 1.285 to 1.288 ATA. HBOT measurements are typically reported to 2 significant figures, i.e. 1.3, 1.5, 2.0, 2.4, 3.0 ATA, etc… Therefore, this properly rounds to 1.3 ATA (we did not report the pressure to 3 significant figures as would be the case with 1.30 ATA). Therefore, an “erratum” in this case is not necessary.Thanks,
Dan
In a way, the good Dr. Rossignol is right, inasmuch as an “erratum” may not be “necessary” given the likely insignificance of the pressures involved in general. Unfortunately for me, I wasn’t looking for “necessary”. I was looking for what’s ethical, and what is scientifically accurate. I’m not going to bother him with any further request for erratum, and I promise my next HBOT post will dig into this paper in much more detail. Here’s my reply to Dr. Rossignol:
Hi Dr. Dan,
Thanks for your reply.
You said, “HBOT measurements are typically reported to 2 significant
figures, i.e. 1.3, 1.5, 2.0, 2.4, 3.0″Now that’s kind of interesting for two reasons:
1. It’s not entirely true. At least two of the papers cited in your references (Collet et al., and Montgomery et al.) show treatment pressures of 1.75 atmospheres. There might even be more. Would those have been acceptable to be published at 1.8? Do you think a real peer-reviewed journal would be requiring that you publish an erratum? I think so, but I could be wrong. In many of the “typical” numbers you list, reporting to 2 significant figures does not change the accuracy anyway (2.0 ATA is stated instead of 2.00 ATA, 3.0 ATA instead of 3.00 ATA, etc.). Your study overstates the treatment pressure beyond the physical capabilities of the equipment, and since the pressure change is so low in the first place, that overstatement has greater mathematical significance. Converting the pressure increase to mmHg (for arterial blood gases calculations) clearly illustrates an overstatement of about 10%, maybe more (but impossible to know without the actual atmospheric data).
and
2. Invoking the, “that’s how they’re typically reported” argument is a fallacious appeal to popularity. No matter how many times anyone says “well that’s how pressures are typically reported”, or no matter how many times pressures are reported that way, it will not make the actual treatment pressure any closer to the rounded up and asserted 1.3 ATA, especially since it’s a physically impossible maximum with that equipment. That appeal to typicality (popularity) may make you “feel” like you’re being scientifically accurate, but then I can’t and wouldn’t dictate your ethics. Medical Hypotheses certainly won’t either. The science will speak for itself.
Am I nitpicking? Absolutely yes, and admittedly so. I’m tired of seeing sloppy science, and attitudes like, “it’s good enough, because everyone else does it that way”. You could have used a term like “approximately” rather than asserting rounded-up absolutes that were impossible with the equipment used. At any rate, we both know that pseudo-HBOT treatment with ~29% O2 enriched air at 1.25-1.30 ATA is probably insignificant anyway. Higher plasma O2 concentrations are achievable with simple O2 therapy (even simple mask at 10 lpm), and then there’s always our good friend hemoglobin, so I won’t bother you any further.
If you change your mind about publishing an erratum to reflect scientific reality, please let me know.
Again, I really do appreciate that you took the time to respond. I also appreciate Ms. Rossignol’s taking the time to drop by and comment to readers at my blog (she has open questions from readers if she’s interested).
Sincerely and best regards,
Dad Of Cameron
Reality counts!
13 Comments
RSS feed for comments on this post.
Sorry, the comment form is closed, see "Comments/Contact" under "About".
Comment by Camille — 30 April, 2006 @ 10:08 pm
Pressure makes diamonds, Dr. R.
Nice work, as usual, D o’ C.
Comment by Robert P — 1 May, 2006 @ 12:00 pm
It’s great to see you debunking and reading through the science. I’ve been doing short blurbs on autism papers over on my site and what I find more often than not is Duh science or things lacking controls. And, that is from the peer reviewed journals.
I’m reading fMRI stuff now, which is a bit more controlled, if not …. informative.
BTW, kick ass work on the blog. Are you doing it yourself? Like the comment preview???? I’ve only seen that a few other places.
Comment by Dad Of Cameron — 1 May, 2006 @ 12:28 pm
Hi Robert.
Thank you for your kind words. I’ll be sure to stop by your site - sounds interesting.
I did assemble this blog from various Wordpress open source tools and plugins.
As an example the comment preview is an AJAX comments plug-in, based on a modified Flex>Tonus Wordpress theme (by Kashou).
Comment by Big White Hat — 1 May, 2006 @ 8:23 pm
I have been looking around your blog and I am impressed. I am an autism dad and a blogger. I am not an autism blogger. I am currently writing a post about Autisitcs’ rights to life and liberty. I may or may not have it done by the time you check it out.
I would really like to get in touch with you. So please send an email to the adress I listed in the email field.
Comment by Dad Of Cameron — 1 May, 2006 @ 10:07 pm
Hi BWH,
Thank you as well for the kind words. You are welcome to contact me at comments at autismstreet daht org. No guarantees, but I try to answer most e-mail within a couple of days.
Comment by TheProbe — 3 May, 2006 @ 4:48 pm
Re HBOT…this treatment had been hyped to parents of kids with cerebral palsy. Then, there was a study at McGoll Univeristy where they found that the ‘results’ were the same regardless of whether they used room air or O2. While it is stil being hyped to an extent, I do not see the spam articles like I used to.
Comment by Dad Of Cameron — 3 May, 2006 @ 5:02 pm
Hi Probe,
I predict pretty big hype on the horizon in the autism world for this pseudoHBOT. It seems that parents are often quick to pay for “hope”. I’m going to dig into the original paper that prompted my article much deeper in the near future. Thanks for reading and sharing your perspective about the past history with CP ‘treatment’.
Comment by Robert P — 5 May, 2006 @ 7:06 am
Have you seen this clinical trial? It doesn’t appear to be the same group.
Comment by Dad Of Cameron — 5 May, 2006 @ 8:23 am
That depends what you mean by same group.
Another DAN! doc, the one you mention is Bradstreet and crew (1st Arizona chelation study that never went to real peer-review, exorcisms?, affiliated with “Defeat Autism In The Name Of Jesus”? - Bradstreet’s CFO, many other wacky angles.
Comment by Robert P — 5 May, 2006 @ 9:00 am
I haven’t read through their protocol, but it will be interesting to see if they are held to the same standard as others.
Comment by Ian — 11 May, 2006 @ 11:46 am
DOC,
Linking from the previous post on this subject, the Vitaeris320 does indeed have 2 pressure valves, but these can also be used to allow oxygen to be fed into the chamber. Some people use 100% oxygen from cylinders, others use portable oxygen concentrators that generate 90% oxygen from the air. Generally the oxygen is passed via tube, through this inlet, feeding either a mask or nasal cannula..
Ian
Comment by Dad Of Cameron — 11 May, 2006 @ 12:54 pm
Ian,
I respect the fact that you are bringing detail and fact about the equipment to the discussion. But, specific to this particular study, what’s your take on the fact that the same plasma O2 concentration can be acheived with simple O2 therapy? (Never mind that hemoglobin is still carrying the Lion’s share of oxygen in a non-anemic patient).
Comment by Dad Of Cameron — 14 May, 2006 @ 6:52 pm
I just got an e-mail back from the manufacturer (OxyHealth).
So there you have it from the manufacturer, the FDA-approved max pressure of the device is 4.26 psi above ambient. (~1.289 ATA at sea level, assuming max pressure is maintained).