An important question: how does one evaluate in the real world if a drug with excellent double-blind RCT results really works? (not necessarily related to Ivermectin)

massimaux | 2 points | Oct 29 2020 14:39:26

An important question: how does one evaluate in the real world if a drug with excellent double-blind RCT results really works? (not necessarily related to Ivermectin)

We all know very well that drug regulatory agencies need multiple peer-reviewed huge multi-center double-blind RCT studies showing safety and effectiveness of a given drug in order to approve it for medical use.

Okay. Let's assume we have such a drug. The question is: how do we know this drug would work in the real world with real patients prescribed/administered by real GPs and/or clinicians?

When, how and who evaluates drug's effectiveness and safety in the post-approval period?

What measured data is compared to what measured data to conclude if the drug is safe and effective when used on a mass scale?

The question is very important. Because, RCT is one space and real world is another space. Good results with RCT do not necessarily replicate in medical practice. And, results in medical practice are those that really matter in the end of the day.

This is worth discussing because, as you will see yourselves, it does imply Ivermectin use by medical practitioners as well.

[-] akaariai | 3 points | Oct 29 2020 18:56:21

The point of RCT is to come up with a treatment plan usable in medical practice (say 24mg ivermectin once immediately at hospitalization, treatment otherwise exactly like normally). Then test the plan with as realistic population as possible (say all patients coming to couple different hospitals) in a setup where the only changing factor is real pills or placebo, and that factor randomized . Then run the tests and compare if there was an effect.

Now if effect was clear, then you have good reason to believe it works in real medical practice, because that was the setup where it was tested.

While the method works great in general there will always be a possibility of different populations in other hospitals, errors in implementing the trial and so on.

All in all well designed and implemented RCT reporting significant and powerful effect and you have very good reason to believe it works well in practice. If you want to be even more sure, then you run another even larger rct in different hospitals.

[-] massimaux | 1 points | Oct 29 2020 19:10:34

Thanks for your input.

The question is, however, how would one know how well the drug/treatment works in medical practice? What is measured and compared with what? You have to know whether it works. You can't just say, I believe the RCT results will replicate in the real world.

[-] TrumpLyftAlles | 3 points | Oct 29 2020 20:52:02

The question is, however, how would one know how well the drug/treatment works in medical practice? What is measured and compared with what? You have to know whether it works.

A great deal has been learned in the past few decades about how medical science gets research wrong, and the result has been increased wisdom about how to do RCTs better so we're more confident in the results. When I was learning about experimental design in the 70's, "multi-center" was not a known desirable feature (or I forgot being taught about it). I still don't know why that's important, but apparently it is. It's also why replication of research is so heavily emphasized, and why every paper has a section that's supposed to expose conflicts of interest. Pre-registration of trials was also not taught in 1974. That came about when scientists became aware of the tendency to not disclose negative results.

Any other kind of inference is going to be inferior to RCTs.

An alternative is epidemiological research after something has been out there a long time. An example is this study that looked at 41 thousand Finns and found that statins and anti-blood pressure meds was associated with greater weight gain. It's unlikely that the statin trials proving safety/efficacy would have picked that up; you need the big numbers.

But no epidemiological study is ever posted without people suggesting that X and Y weren't controlled for and an RCT is needed to verify the results. /r/covid19 has about 70 observational studies that say vitamin D helps with covid19 and without fail, people comment that you can't learn anything from observational studies, only RCTs have value.

Obviously not very relevant to ivermectin.

[-] massimaux | 3 points | Oct 29 2020 21:52:16

I'm on the same page with you regarding what doctors of the world should do with IVM.

With this post I tried to point out the absurdity of blindly insisting on large well designed RCTs in the context of this virus-caused catastrophe that humanity has not faced in the last 70 years at least. Of course, RCT is a gold standard, and it is beyond doubt the best scientific method to test a hypothesis regarding medical treatments.

But, hey, every forking day, over 6000 people all around the world, are dying in unimaginable sufferings. This rate is about to increase in the coming period.

On the other hand, we have a drug that is very very safe and we have about 10 trials that built up the probability that IVM CAN save lives and cut the misery of COVID patients to a large extent.

The RCT fanatics don't understand that the ultimate goal of RCT is to maximize the probability that a certain drug/treatment is safe and effective. For IVM, we already know it's safe and the probability that it's effective for different stages is pretty high. Perhaps not as valid as an RCT would suggest, but still high enough so that doctors don't waste precious time and prescribe/administer it immediately.

Back to the original post, here comes the point. Even if we waited for a great RCT, real world evidence (RWE) is what ONLY matters in the end of the day. RCT results must be compared against the RWE. What is RWE? Measured data for all patients who received the treatment will be recorded and measured data for all patients who did not receive the treatment have already been recorded. Very soon, after treating, say, 10,000 or 20,000 patients, which is a matter of weeks, these two sets of data will be available and ready to be analyzed and compared. That's the RWE. If there is effectiveness, doctors will continue using it. If not, they will stop. But there is nothing to lose and very very much to win.

To sum up, RWE needs to be collected in any case, either with or without RCT. I'm just pointing to the obvious, RCT should be skipped and large RWE on IVM should be collected as soon as possible. F.ck!!!

[-] Haitchpeasauce | 2 points | Oct 30 2020 03:45:56

If observational and molecular studies are literally of no value, we might as well stop doing them. /s

[-] massimaux | 1 points | Oct 29 2020 20:24:49

IMO, the following must be done.

Measured data A should be recorded electronically for all patients who received the treatment (after the approval).

Measured data B should have been recorded electronically for all patients who did not receive the treatment (before the approval).

Data A and data B should be compared in all relevant statistical dimensions. This is actually a huge observational study called real world evidence (RWE).

Finally, the RWE results should be compared with the RCT results and thereafter definitive conclusions can be drawn on how well RWE matches RCT.

Without collecting data A and data B and generating RWE, we may never know how well the treatment works.

[-] TrumpLyftAlles | 3 points | Oct 29 2020 20:37:22

When, how and who evaluates drug's effectiveness and safety in the post-approval period?

I don't know much about this.

Suppose the FDA approves ivermectin for covid-19 (maybe just an EUA), and somehow a good number of MDs / GPs are persuaded to try it with their covid19 patients, since the FDA approval gives them shelter from lawsuits if they try it.

There was an interesting conversation on This Week in Virology about how something like 90% of modern medical practice is not based on RCTs, and how in NYC in March/April when there were no data-based covid19 protocols, doctors figured out what works almost by accident. Roughly: "It might come down to scheduling: two patients share a room, they are in similarly dire condition, both are scheduled to go on ventilators, one IS intubated but the other has not yet been intubated when the doctor comes in and notices that one of the two patients who were about the same status, now have a considerably-worse patient -- the intubated one."

The point being that doctors watch what happens with their patients. If a drug doesn't work, they'll notice. If it's deleterious, they'll notice faster. And of course they notice when a drug works, too. An MD might conduct informal experiments: let's try IVM on these patients and not these others. Doctors talk to each other, post on forums. The truth will come out.

There is a formal process for reporting adverse events. All drugs are subject to this monitoring. It happens that drugs which have gone through the billion dollars of testing and gotten FDA approval turn out to be dangerous to some subset of patients, and that's not discovered until millions of patients are taking the drug. This kind of thing probably is not picked up by individual MDs or posted on forums. <-- my guess.

I don't know if this is responsive to your question. Is it?