the implications of this work are just staggering, to be honest. now that i've slept over it, it's obvious and undeniable actually that the trial was never blinded!
But actually we can check that by only looking at symptom onset more than a week after jab, but n fact the onset of almost all of the symptoms associated with these visits started later than 1 week post jab so it won't change much
Hi OpenVAET, Great work! I'm trying to run your Perl code, but getting: Missing file [public/doc/pfizer_trials/pfizer_trial_demographics_merged.json] at analyse_local_and_central_by_site.pl line 220. Do I need to run some other routine first to create this and other json files?
Regarding the results, to avoid possible criticisms of p-hacking / cherry picking, it would be nice to see a histogram of all of the p-values. Under the null hypothesis of no differences between arms, this distribution should be uniform. If it has an overabundance of small ps, then we have solid evidence across all sites. In addition, it would make sense to apply a multiple testing correction like False Discovery Rate to see how it adjusts the p-values.
Hello Russ ; and thanks for your great feedback - I'll work on that histogram, discuss that with Josh & we will update as soon as we find the time !
Regarding "running on the raw data" - indeed, some additional explanations are required which need to be integrated in a more elegant fashion - I'll add a Substack on technicalities shortly.
There are two steps prior to run the script provided above:
A. Downloading & preparing the XPT files, converting them to raw .CSV.
B. Parsing the .CSV files we care about & converting them to a more optimized JSON structure.
You can jump to the "ready to run" step by unzipping the following .JSON conversions of the raw files in your project root folder (650 Mo - 44 Mo zipped):
Or alternatively you can reproduce all the steps yourself to check for errors earlier in the process:
1. If you're booting from the "project dump" available on Github you'll first need the PHMPT documents - and to convert the .XPT files to .CSV (using R or ReadDirStats).
You can either:
- download the "already converted .CSV" here (12 Go - 300 Mo zipped):
- or execute "perl tasks/pfizer_documents/get_documents.pl" - answering "Y", when the script prompts if it should unzip all the files & convert the PDF files. Having done that, you need to convert every .XPT file you care about with ReadDirStats (for example "readstat FDA-CBER-2021-5683-0772290-0772417_125742_S1_M5_bnt162-01-S-D_supplb.xpt FDA-CBER-2021-5683-0772290-0772417_125742_S1_M5_bnt162-01-S-D_supplb.csv").
2. Having converted the .XPT files to .CSV you need to extract the files to .JSON (landing to the first .ZIP provided) using the various extraction scripts located in "tasks/pfizer_trials/*". Should be quite easy to locate the right extract script searching for the .json input required in the project.
Due to the fact that we were for a long time working without ADSL file, this script still uses .PDF extracts (the end data isn't affected) - and the syntax could be shorter now that new files have been released. I'll program an updated version ASAP. Don't hesitate if something is still unclear meanwhile !
Basically this article is reporting on one tail end of happenings. I'm saying look at the other tail end of happenings and confirm nobody could make the argument that "hey you could have just as easily argued that many sites were skewed in favor of placebo efficacy and vaccine inefficacy".
Great work, but isn't this explained by mild post-vaccination symptoms in the days following vaccinations? Investigators likely told participants to ignore these.
👏👏Great work. Fascinating that there were only 800 central pcrs in the treatment group
the implications of this work are just staggering, to be honest. now that i've slept over it, it's obvious and undeniable actually that the trial was never blinded!
Can't agree more with you.
If only defrauding the efficacy was the worst part and that they had sold a non-dangerous snake-oil...
But nah, as we already have numerous evidences of it, they also defrauded the safety part...
But actually we can check that by only looking at symptom onset more than a week after jab, but n fact the onset of almost all of the symptoms associated with these visits started later than 1 week post jab so it won't change much
Hi OpenVAET, Great work! I'm trying to run your Perl code, but getting: Missing file [public/doc/pfizer_trials/pfizer_trial_demographics_merged.json] at analyse_local_and_central_by_site.pl line 220. Do I need to run some other routine first to create this and other json files?
Regarding the results, to avoid possible criticisms of p-hacking / cherry picking, it would be nice to see a histogram of all of the p-values. Under the null hypothesis of no differences between arms, this distribution should be uniform. If it has an overabundance of small ps, then we have solid evidence across all sites. In addition, it would make sense to apply a multiple testing correction like False Discovery Rate to see how it adjusts the p-values.
Hello Russ ; and thanks for your great feedback - I'll work on that histogram, discuss that with Josh & we will update as soon as we find the time !
Regarding "running on the raw data" - indeed, some additional explanations are required which need to be integrated in a more elegant fashion - I'll add a Substack on technicalities shortly.
There are two steps prior to run the script provided above:
A. Downloading & preparing the XPT files, converting them to raw .CSV.
B. Parsing the .CSV files we care about & converting them to a more optimized JSON structure.
You can jump to the "ready to run" step by unzipping the following .JSON conversions of the raw files in your project root folder (650 Mo - 44 Mo zipped):
https://drive.google.com/file/d/118kVmsuelhOCWEU2bHEq3aFRv-jOIrkK/view?usp=share_link
Or alternatively you can reproduce all the steps yourself to check for errors earlier in the process:
1. If you're booting from the "project dump" available on Github you'll first need the PHMPT documents - and to convert the .XPT files to .CSV (using R or ReadDirStats).
You can either:
- download the "already converted .CSV" here (12 Go - 300 Mo zipped):
https://drive.google.com/file/d/1iqO6-s62wpnEdNq2LDh_YFyvlbuB0RXj/view?usp=share_link
- or execute "perl tasks/pfizer_documents/get_documents.pl" - answering "Y", when the script prompts if it should unzip all the files & convert the PDF files. Having done that, you need to convert every .XPT file you care about with ReadDirStats (for example "readstat FDA-CBER-2021-5683-0772290-0772417_125742_S1_M5_bnt162-01-S-D_supplb.xpt FDA-CBER-2021-5683-0772290-0772417_125742_S1_M5_bnt162-01-S-D_supplb.csv").
All of this is further documented on the "Methodology Details" section of this page: https://openvaet.org/studies/review_nejm_fda_data?currentLanguage=en
2. Having converted the .XPT files to .CSV you need to extract the files to .JSON (landing to the first .ZIP provided) using the various extraction scripts located in "tasks/pfizer_trials/*". Should be quite easy to locate the right extract script searching for the .json input required in the project.
Due to the fact that we were for a long time working without ADSL file, this script still uses .PDF extracts (the end data isn't affected) - and the syntax could be shorter now that new files have been released. I'll program an updated version ASAP. Don't hesitate if something is still unclear meanwhile !
As a control, can you possibly report on the testing rates of the sites that were the most skewed towards testing the vaccine group?
You mean sites where we would have observed a discrepancy in favor of Placebo testing instead of BNT testing ?
The code isn't "arm specific", they would have spawned in the analysis if such discrepancy was observed.
Basically this article is reporting on one tail end of happenings. I'm saying look at the other tail end of happenings and confirm nobody could make the argument that "hey you could have just as easily argued that many sites were skewed in favor of placebo efficacy and vaccine inefficacy".
That's what I mean by "The code isn't "arm specific"".
If sites had been dis-balanced in favor of the Placebo, it would have spawned all the same and would have been reported here.
They don't spawn because there are no such anomalies. Go figure why 😌
I request the highest of fives:
https://www.youtube.com/watch?v=sc0848ovEMo
Great work, but isn't this explained by mild post-vaccination symptoms in the days following vaccinations? Investigators likely told participants to ignore these.
No because in that case they wouldn't have gotten a central test either