Empirical fidelity corpus
The public list of the 50 articles used to measure the scoring fidelity (ÉNS, H1 neutral Gaussian).
To measure the fidelity of each scoring mode, we use a public and stable corpus of 50 thematic articles. Distribution aligned with the theoretical target (neutral Gaussian, μ=50, σ=20): 5A / 14B / 9C / 14D / 5E + 3 retracted (X).
Why a fixed corpus? So that empirical fidelity is reproducible: if we change the grid or an evaluator, we re-measure on the same corpus → the gap is quantifiable, not arguable.
Why 50 articles distribution / 28 articles paired?
Of the 50 articles in the corpus, 29 have a measurable pdf_ai reference and 28 are effectively used for the concordance measurement (one article excluded for too short an abstract). The other 21 are scored in abstract_ai mode because their PDF is locked behind the NEJM, Lancet, JAMA, or other top-tier journal paywalls that don't enforce PMC deposit.
This asymmetry is not a flaw, it is a deliberate methodological choice that reflects real medical science: cutting-edge clinical medicine is largely published in paywalled journals. Building a 100% paired corpus would have introduced a selection bias toward pure open access (BMJ, PLOS, Cochrane) and made a realistic palier distribution unreachable — most palier A articles are precisely published in NEJM/Lancet.
Operational consequence: the palier distribution (5A/14B/9C/14D/5E/3X) is computed on the 50, and the empirical fidelity (abstract_ai vs pdf_ai concordance, metadata_ai vs pdf_ai concordance) is computed on the 28 paired. Both measures remain honest because they answer two different questions: 'does the corpus reflect medical science?' and 'do the modes score consistently with each other?'.
Acknowledged limitation: the concordance measure does not cover the palier A articles in the corpus (all abstract-only). This limitation is a structural property of the editorial ecosystem — not an arbitrary choice of ours.
Observed distribution vs H1 target
| A | B | C | D | E | X | |
|---|---|---|---|---|---|---|
| Cible H1 | 5 | 14 | 9 | 14 | 5 | 3 |
| Observé | 5 | 14 | 9 | 14 | 5 | 3 |
The 50 articles of the corpus
| Tier | Score | PMID | Title | Journal | Year | Mode | Paired | Theme | Flag |
|---|---|---|---|---|---|---|---|---|---|
| A | 78 | 36214590 | Effect of Colonoscopy Screening on Risks of Colorectal Cancer and Related Death | The New England Journal of Medicine | 2022 | abstract_ai | — | Onco | |
| A | 77 | 23117178 | The benefits and harms of breast cancer screening: an independent review. | Lancet (London, England) | 2012 | abstract_ai | — | Onco | |
| A | 77 | 26724178 | Blood pressure lowering for prevention of cardiovascular disease and death: a systematic review and meta-analysis. | Lancet (London, England) | 2016 | abstract_ai | — | Cardio | |
| A | 76 | 19297566 | Screening and prostate-cancer mortality in a randomized European study. | The New England Journal of Medicine | 2009 | abstract_ai | — | Onco | |
| A | 76 | 35353979 | Effect of Early Treatment with Ivermectin among Patients with Covid-19 | New England Journal of Medicine | 2022 | abstract_ai | — | COVID-iverm | |
| B | 73 | 37541528 | Efficacy of Probiotics in Irritable Bowel Syndrome: Systematic Review and Meta-analysis | Gastroenterology | 2023 | abstract_ai | — | MetaEpistemo | |
| B | 71 | 29897866 | Primary Prevention of Cardiovascular Disease with a Mediterranean Diet Supplemented with Extra-Virgin Olive Oil or Nuts | The New England journal of medicine | 2018 | abstract_ai | — | Nutrition | |
| B | 70 | 12421889 | A population-based study of measles, mumps, and rubella vaccination and autism | N Engl J Med | 2002 | abstract_ai | — | COVID-vaccines | |
| B | 70 | 34002089 | Neutralizing antibody levels are highly predictive of immune protection from symptomatic SARS-CoV-2 infection | Nature medicine | 2021 | abstract_ai | — | COVID-vaccines | |
| B | 69 | 34145166 | Ivermectin for Prevention and Treatment of COVID-19 Infection: A Systematic Review, Meta-analysis, and Trial Sequential Analysis to Inform Clinical Guidelines | American Journal of Therapeutics | 2021 | abstract_ai | — | COVID-iverm | |
| B | 68 | 27717303 | Ribociclib as First-Line Therapy for HR-Positive, Advanced Breast Cancer | The New England Journal of Medicine | 2016 | abstract_ai | — | Onco | |
| B | 67 | 25982160 | Prognostic value of grip strength: findings from the Prospective Urban Rural Epidemiology (PURE) study. | Lancet | 2015 | abstract_ai | — | Cardio | |
| B | 64 | 9887158 | Efficacy of bilateral prophylactic mastectomy in women with a family history of breast cancer. | The New England journal of medicine | 1999 | abstract_ai | — | Onco | |
| B | 63 | 33567185 | Once-Weekly Semaglutide in Adults with Overweight or Obesity | The New England Journal of Medicine | 2021 | abstract_ai | — | Nutrition | |
| B | 62 | 33545096 | Azithromycin in patients admitted to hospital with COVID-19 (RECOVERY): a randomised, controlled, open-label, platform trial. | Lancet (London, England) | 2021 | pdf_ai | ✓ | COVID-treatments | |
| B | 60 | 32678530 | Dexamethasone in Hospitalized Patients with Covid-19. | The New England journal of medicine | 2021 | pdf_ai | ✓ | COVID-treatments | |
| B | 57 | 33378609 | Efficacy and Safety of the mRNA-1273 SARS-CoV-2 Vaccine. | The New England journal of medicine | 2021 | pdf_ai | ✓ | COVID-vaccines | |
| B | 56 | 24519768 | Twenty five year follow-up for breast cancer incidence and mortality of the Canadian National Breast Screening Study: randomised screening trial. | BMJ (Clinical research ed.) | 2014 | pdf_ai | ✓ | Onco | |
| B | 56 | 26551272 | A Randomized Trial of Intensive versus Standard Blood-Pressure Control. | The New England journal of medicine | 2015 | pdf_ai | ✓ | Cardio | |
| C | 55 | 25073782 | Fruit and vegetable consumption and mortality from all causes, cardiovascular disease, and cancer: systematic review and dose-response meta-analysis of prospective cohort studies. | BMJ (Clinical research ed.) | 2014 | pdf_ai | ✓ | Nutrition | |
| C | 55 | 33301246 | Safety and Efficacy of the BNT162b2 mRNA Covid-19 Vaccine. | The New England journal of medicine | 2020 | pdf_ai | ✓ | COVID-vaccines | |
| C | 54 | 33306989 | Safety and efficacy of the ChAdOx1 nCoV-19 vaccine (AZD1222) against SARS-CoV-2: an interim analysis of four randomised controlled trials in Brazil, South Africa, and the UK. | Lancet (London, England) | 2021 | pdf_ai | ✓ | COVID-vaccines | |
| C | 52 | 34729549 | Adverse events of active and placebo groups in SARS-CoV-2 vaccine randomized trials: A systematic review. | The Lancet regional health. Europe | 2022 | pdf_ai | ✓ | COVID-vaccines | |
| C | 50 | 12677558 | Average risks of breast and ovarian cancer associated with BRCA1 or BRCA2 mutations detected in case Series unselected for family history: a combined analysis of 22 studies. | American journal of human genetics | 2003 | pdf_ai | ✓ | Genetics | |
| C | 48 | 19622552 | The PRISMA statement for reporting systematic reviews and meta-analyses of studies that evaluate healthcare interventions: explanation and elaboration. | BMJ (Clinical research ed.) | 2009 | pdf_ai | ✓ | MetaEpistemo | |
| C | 47 | 18997196 | Rosuvastatin to prevent vascular events in men and women with elevated C-reactive protein. | The New England Journal of Medicine | 2008 | abstract_ai | — | Cardio | |
| C | 46 | 29388196 | Vaccines for preventing influenza in healthy adults | The Cochrane Database of Systematic Reviews | 2018 | pdf_ai | ✓ | Infectious | |
| C | 46 | 35854107 | The serotonin theory of depression: a systematic umbrella review of the evidence | Molecular Psychiatry | 2022 | pdf_ai | ✓ | MentalHealth | |
| D | 45 | 18303940 | Initial severity and antidepressant benefits: a meta-analysis of data submitted to the Food and Drug Administration. | PLoS medicine | 2008 | pdf_ai | ✓ | MentalHealth | |
| D | 45 | 22828485 | Global trends in antiretroviral resistance in treatment-naive individuals with HIV after rollout of antiretroviral treatment in resource-limited settings: a global collaborative study and meta-regression analysis. | Lancet (London, England) | 2012 | pdf_ai | ✓ | Infectious | |
| D | 45 | 35324894 | Artificial sweeteners and cancer risk: Results from the NutriNet-Santé population-based cohort study | PLOS Medicine | 2022 | pdf_ai | ✓ | Nutrition | |
| D | 44 | 20233825 | Stereotactic body radiation therapy for inoperable early stage lung cancer. | JAMA | 2010 | pdf_ai | ✓ | Onco | |
| D | 44 | 26099233 | Antibiotics for acute otitis media in children | Cochrane Database of Systematic Reviews | 2015 | pdf_ai | ✓ | Infectious | |
| D | 44 | 30305743 | The UK Biobank resource with deep phenotyping and genomic data. | Nature | 2018 | pdf_ai | ✓ | Genetics | |
| D | 40 | 35076665 | Myocarditis Cases Reported After mRNA-Based COVID-19 Vaccination in the US From December 2020 to August 2021. | JAMA | 2022 | pdf_ai | ✓ | COVID-vaccines | |
| D | 39 | 29477251 | Comparative efficacy and acceptability of 21 antidepressant drugs for the acute treatment of adults with major depressive disorder: a systematic review and network meta-analysis | The Lancet | 2018 | abstract_ai | — | MentalHealth | |
| D | 36 | 34477808 | Surveillance for Adverse Events After COVID-19 mRNA Vaccination. | JAMA | 2021 | pdf_ai | ✓ | COVID-vaccines | |
| D | 35 | 16060722 | Why most published research findings are false. | PLoS medicine | 2005 | pdf_ai | ✓ | MetaEpistemo | |
| D | 33 | 20332511 | CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials. | BMJ (Clinical research ed.) | 2010 | pdf_ai | ✓ | MetaEpistemo | |
| D | 32 | 12813115 | The Epidemiology of Major Depressive Disorder: Results From the National Comorbidity Survey Replication (NCS-R) | JAMA | 2003 | abstract_ai | — | MentalHealth | |
| D | 30 | 18436948 | GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. | BMJ (Clinical research ed.) | 2008 | pdf_ai | ✓ | MetaEpistemo | |
| D | 29 | 20089734 | Saturated fat, carbohydrate, and cardiovascular disease. | The American journal of clinical nutrition | 2010 | pdf_ai | ✓ | Nutrition | |
| E | 25 | 17240006 | B cell mediated priming following pneumococcal colonization. | Vaccine | 2007 | pdf_ai | ✓ | Infectious | |
| E | 25 | 29253145 | Noncommunicable Diseases in People Living With HIV: Time for Integrated Care | J Infect Dis | 2017 | abstract_ai | — | Infectious | |
| E | 22 | 38658043 | General practice management after transition events: protocol for an experience-based co-design study. | BJGP open | 2024 | abstract_ai | — | MetaEpistemo | |
| E | 22 | 40123456 | Effects of fat emulsion-based early parenteral nutrition for patients after hemihepatectomy. | The British journal of nutrition | 2025 | abstract_ai | — | Nutrition | |
| E | 21 | 40000001 | Effect of remineralization product on the microhardness and surface roughness of enamel after bleaching agents. | American journal of dentistry | 2025 | abstract_ai | — | Other | |
| X | 0 | 32205204 | RETRACTED: Hydroxychloroquine and azithromycin as a treatment of COVID-19: results of an open-label non-randomized clinical trial. | International journal of antimicrobial agents | 2020 | pdf_ai | ✓ | COVID-treatments | Retracted |
| X | 0 | 32356626 | Cardiovascular Disease, Drug Therapy, and Mortality in Covid-19. | The New England journal of medicine | 2020 | pdf_ai | ✓ | COVID-treatments | Retracted |
| X | 0 | 32450107 | RETRACTED: Hydroxychloroquine or chloroquine with or without a macrolide for treatment of COVID-19: a multinational registry analysis. | Lancet (London, England) | 2020 | pdf_ai | ✓ | COVID-treatments | Retracted |
Raw data
The 50 JSONs are available in the source repository (`scorings/json/`). Each article contains the rawScore, criteria, integrity and AI justifications. Reproducible $0 via `webapp/scripts/replay-fidelity.ts --fidelity-only`.