Phase 3 Search · July 2026
This document explains what the Complete Response Letter Report measures, where every number comes from, exactly how the openFDA dataset was classified, and what the analysis is not. It exists so a reader (or a fact-checking journalist) can trace any figure in the report back to its source without asking us to take it on faith.
Building this dataset was not glamorous work: writing the header-matching logic, then spot-checking dozens of individual letters by hand to make sure the two lists below held up against real CRL text. I did it once so nobody has to take my word for the 62.6% figure.
This report measures one thing directly and reports another from the literature. Directly: how many of the FDA's published Complete Response Letters carry a manufacturing or quality deficiency section, counted from the section headings the agency writes into each letter. From the literature: which deficiency was the primary reason, which only peer-reviewed coded studies can establish.
Two separate things, kept deliberately distinct. First, peer-reviewed, cohort-coded findings from published literature on why the FDA issues Complete Response Letters (CRLs) and what happens next: McCarthy & O'Boyle (2026, Drug Discovery Today), Dilek et al. (2026, Ther Innov Regul Sci), Jefferies via BioSpace (2026), and RSM US LLP (2026). Second, Phase 3 Search's own text-structure analysis of the 457 letters published in the openFDA CRL transparency dataset, which we built ourselves because no equivalent public breakdown of the full archive by year, section-header content, or filer existed.
These two things are never blended into one number. Where the report cites a peer-reviewed percentage, it names the study. Where it cites Phase 3's own analysis, it is labeled as such, every time.
The primary dataset is the openFDA Complete Response Letter transparency dataset (open.fda.gov/apis/transparency/completeresponseletters), downloaded July 2026: 457 published letters, letter years 2002-2026, with the bulk released in the last three years (69 in 2024, 59 in 2025, 31 so far in 2026) as the FDA works through its transparency-initiative backlog. 444 of the 457 carry the FDA letter_type value "COMPLETE RESPONSE". The other 13 are the action types openFDA records in the same dataset: 4 tentative approval, 1 provisional determination, 1 corrected provisional determination, 1 refusal to file, 1 rescinded complete response, 4 with an empty letter_type field, and 1 (the Chiesi record described under Known limitations) with no letter_type field at all.
Counts reflect openFDA's published set of 457 letters as-is; four predate the Aug 2008 CRL rule (Not Approvable letters retained in the same dataset). Those four are the empty-letter_type records: Organon, Inc. (15 March 2002 and 23 June 2004), Fabre-Kramer Pharmaceuticals, Inc. (2 November 2007), and Kyowa Pharmaceutical, Inc. (25 February 2008). Each was issued as not approvable under 21 CFR 314.125(b), before the FDA's 2008 final rule amending 21 CFR 314.110 replaced the Approvable and Not Approvable letters with the Complete Response Letter on 14 August 2008. We keep them in the archive total rather than silently dropping four rows the FDA itself publishes. None of the four carries a manufacturing or quality section header, so none of them sits inside the 286 counted below, and the 62.6% figure is unaffected either way.
Each record includes the letter's full text, the letter year, the naming company, the approval status FDA currently has on file, and administrative metadata (approver, review office, application number). All figures in this report drawn from this dataset use the text, letter_year, company_name, letter_type, and approval_status fields exactly as published; nothing is imputed or estimated.
Reproduction, August 19, 2026. The whole analysis is now regenerated by a published script, tools/build_crl_dataset.py, which rebuilds every table on this report from the raw openFDA file and validates itself against this page's July 2026 baseline before it will write output: 75 checks, all passing, covering the section-header headline, all four exclusive groups, the letter-structure counts, the boilerplate control, and every by-year row. The published pages are rendered from that script's output by tools/render_crl_asset.py, so a figure on the page cannot drift from the data behind it.
Refresh note, August 19, 2026. The archive stood at 459 records on August 13, 2026 (live check against the openFDA API: two letters added since the July 22 pull, Osteal Therapeutics NDA 219644 dated July 29 and ITM Solucin NDA 218705 dated August 7, both typed Complete Response, both currently Unapproved). Applying the section-header rule to the additions gives a conservative 287 of 459 (62.5%); the ITM letter carries a FACILITY INSPECTIONS header, and the Osteal letter is placed clinical-only pending a full-text bulk pull. The published figure is therefore stable at 62.5-62.7% across both pulls. Every number on these pages regenerates from tools/recompute_crl_stats.py, which reproduces the July figures exactly (286/457, groups 202/84/58/113); run it on each new FDA batch and update from its output only.
The FDA's own CRLs are internally structured with named deficiency sections. In the letter text as openFDA publishes it, a section title sits on its own line and is set in full capitals (PRODUCT QUALITY), while the same words inside running prose are not (Product Quality (Melblez Constituent Part)). The rule below uses that difference, so it counts section headers and not passing mentions.
The rule, written so anyone can re-run it:
text field of each of the 457 records in the July 2026 download.line.strip().isupper()). Lines with any lowercase character are skipped.| Side | Section-header labels matched (case-insensitive, full line) |
|---|---|
| Manufacturing / quality | PRODUCT QUALITY · FACILITY INSPECTIONS · MICROBIOLOGY |
| Clinical | CLINICAL · NONCLINICAL · CLINICAL PHARMACOLOGY · CLINICAL/STATISTICAL · HUMAN FACTORS · DEVICE |
Run that rule over the archive and you get 286 of 457 letters (62.6%) carrying a manufacturing- or quality-side header, split as 202 manufacturing/quality header only, 84 carrying both sides, 58 clinical header only, and 113 with neither. Those four groups are the four rows published on the report page, and they sum to 457.
Why the uppercase condition earns its place. The FDA sets a letter's deficiency-section titles in full capitals. The same labels also turn up in mixed case, 60 times across 45 letters, and 29 of those 60 sit underneath the letter's ADDITIONAL COMMENTS heading, which the FDA introduces with the words "comments/recommendations that are not approvability issues". Counting them would count material the FDA has explicitly marked as not blocking approval. Drop the uppercase condition and the figure rises to 296 of 457 (64.8%). Keep the condition but narrow the denominator to the 444 letters typed COMPLETE RESPONSE and it rises to 286 of 444 (64.4%). Both alternatives are higher than the published number, which is the point: 62.6% is a conservative floor, not a number chosen because it flattered the argument.
Headers that appear in nearly every letter regardless of the actual deficiency, such as SAFETY UPDATE, PRESCRIBING INFORMATION, CARTON AND CONTAINER LABELING, PROPRIETARY NAME, and ADMINISTRATION, are administrative boilerplate common to the CRL template itself. They were deliberately excluded from the classification because their near-universal presence (SAFETY UPDATE alone appears in 394 of 457 letters, 86.2%, under the same rule) would swamp any real signal.
The other two classifications follow from the same pass:
What the flag does and does not mean. It is a keyword and section-header presence signal, not the FDA's own primary-reason coding, and the FDA does not publish primary-reason coding for the full archive. A flagged section means the FDA opened a section on that area in that letter. It does not mean the finding in it was the sole or blocking reason for the rejection, and severity was not coded. The areas are not exclusive either: a single letter can carry headers from more than one area, and 84 of the 457 carry both a manufacturing/quality and a clinical header, which is why these groups are reported as counts of letters rather than as slices of a pie.
Where a peer-reviewed, cohort-level figure exists for a smaller, audited sample (CMC cited in 45% of McCarthy & O'Boyle's 30 approved NME CRLs; CMC in 51% of Dilek's 43 NMEs), the report cites that figure separately and names it as such. The 62.6% figure answers a different, narrower question: does this letter contain an FDA-labeled manufacturing or quality section at all, across every letter type in the public archive, not just NMEs.
An earlier pass at this dataset tried simple keyword-presence counting (does the word "manufacturing" appear anywhere in the letter). It does not discriminate, and the same 457 letters show why. Searching each letter's text field for a case-insensitive substring: "clinical" appears in 424 letters (92.8%), "process" in 443 (96.9%), and at least one of "manufactur", "cmc" or "quality" in 350 (76.6%). The FDA's own template makes the point from the other direction: the administrative SAFETY UPDATE section header appears in 394 of 457 letters (86.2%) whatever the letter is actually about. CRLs are long, multi-section documents that discuss several categories of issue regardless of which one drove the decision, so a raw keyword count measures document length more than it measures deficiency. Reporting a single "X% of CRLs are about manufacturing" from keyword hits would have been a misleading number dressed up as a fact. The section-header rule above was adopted instead because it reads the FDA's own internal document structure rather than counting words that appear for unrelated reasons.
| Source | Cohort | What it establishes |
|---|---|---|
| McCarthy & O'Boyle, Drug Discovery Today (2026) | 358 first-cycle small-molecule NME NDAs, CDER, 2013-2023 | CMC is the single most common deficiency area, cited in 45% of 30 approved CRL'd NDAs (areas overlap; 53% of letters cited more than one); CMC recovers at 75%, safety at 44.4%; median delay ~709 days |
| Dilek et al., Ther Innov Regul Sci 60(3) (2026) | 43 novel therapeutics | 65% facility-related, 51% CMC; median 1.28 years CRL-to-approval |
| Jefferies (Andrew Tsai) via BioSpace, 11 May 2026 | Full public CRL archive (analyst review) | >50% cite manufacturing; 41% cite product quality; 27% cite need for more clinical data |
| RSM US LLP, 5 February 2026 | First 200+ released CRLs | >2.5 years average CRL-to-approval; >half of facility deficiencies traced to an incomplete pre-approval inspection |
| PharmaSource CDMO survey | CDMO bid outcomes | Regulatory violations are the #1 cited reason (26%) a CDMO loses a bid |
Not FDA's official coding. The 62.6% section-header figure is Phase 3 Search's own read of the FDA's own document structure. It is disclosed, repeatable, and labeled everywhere it appears, but it is not a claim the FDA itself has published about the archive as a whole.
Not a single "% of CRLs are manufacturing" headline. No such number exists across the whole archive, official or otherwise. The report always names which study or method a percentage comes from.
Not free of recency bias. Approval-status figures for 2024-2026 letters are right-censored: median resolution takes roughly 1.28 to 2 years (Dilek 2026; McCarthy & O'Boyle 2026), so recent letters have not had time to resolve. Do not read a low approved-rate in 2024-2026 as a change in outcomes; it is a timing artifact of when the dataset was pulled.
Not de-duplicated by corporate family. Company names in the repeat-filer count are exactly as filed with the FDA. A parent company and its subsidiaries, or a company under two legal-entity names, are counted separately unless the company_name field is identical.
Not investment, legal, or regulatory advice. This is a structural analysis of public FDA data intended for operational and hiring decisions, not a substitute for regulatory counsel.
Pre-2013 sparsity. The dataset carries only 14 letters across 2002-2012 combined, reflecting which older letters the FDA chose to digitize and release, not actual historical CRL volume. Percentages for that period are noisy and should be read as directional only.
Section-header method blind spots. A minority of letters (24.7%) use different internal structure or are non-CRL action types without a deficiency-section body. They are classified "neither flagged" rather than force-fit into one side, but they remain inside the 457-letter denominator used to compute 62.6%, which makes 62.6% a conservative floor rather than an inflated number. Separately, presence of a section header means the FDA opened that section. It does not mean the finding in it was the reason for the letter, or that it was blocking. We did not code severity.
One letter has no parseable year. 456 of the 457 letters carry a letter_year value. The remaining letter (Chiesi Farmaceutici S.p.A., letter_date 27 February 2026, filed by the FDA as "Under Review for Release") has neither a letter_year nor letter text in the source record, so it cannot be placed in a by-year breakdown or scanned for section headers. It is included in the 457-letter archive total and in the "neither flagged" count; any by-year table on the report or in the downloadable data sums to 456 as a result.
One letter's year field disagrees with its own date. The record for DBV Technologies S.A. carries letter_year 2019, but its letter_date is 3 August 2020 and its file name is CRL_BLA125713_20200803.pdf. By letter date, and by the FDA's own file name, this is a 2020 letter. The by-year tables here group on the published letter_year field, without exception, because that is the rule a reader can re-run against the raw file and reproduce; regrouping this one record by hand would make the tables un-reproducible from any stated rule. So the letter sits in the 2019 row. Moving it to 2020 would shift one letter from the 2019 count to the 2020 count and change nothing else in the analysis.
Not a substitute for FDA's own coding. If the FDA ever publishes its own primary-reason coding for the full public archive, that would supersede this analysis for archive-wide claims. Until then, the peer-reviewed, audited-cohort studies (McCarthy & O'Boyle; Dilek et al.) remain the strongest deficiency-area evidence available, and this report treats them as such.
The full computed dataset, by year, by section-flag classification, and by repeat filer, is available as CSV and JSON. Every number in the report traces to a row in those files, to a named peer-reviewed study cited above, or to a count you can recompute yourself from the same openFDA download using the rule written out above. Nothing on these pages rests on a keyword list we have not published. If you believe a classification is wrong, contact us and we will review and correct promptly.
Compiled by Phase 3 Search · July 2026. This document is the methodology companion to The Complete Response Letter Report.