Bonzai,

Some in the "evidence based movement" woke up to the point you're making about statistical limitations* when studies of magic water began appearing in prestigious journals.

Science-based medicine's message in a nutshell: data are only useful when the prior probability for the hypothesis under study is pretty good.

If we can't imagine a plausible mechanism for some therapy, we probably shouldn't bother with a controlled trial. The signal-to-noise ratio in medicine is low and without the constraint of plausibility, most positive results will be false positives.

"Evidence-based medicine" has been a gravy train for the pseudoscientists. Occasionally the magic water or the magic herbs seem to work. And even when they don't work, well, those federal dollars can make a facility look a lot more pretty.

SBM kicks EBM's ass. Too bad SBM is only like 12 dudes.
______________
*OK, not your exact point, which has more to do with access to first person data. But you were talking limitations, so I'm riffing on that.