--- name: k-dense-ai/analytical-method-validation source: https://app.decimal.ai/s/k-dense-ai-analytical-method-validation@1/SKILL.md source_sha256: 4558133703dc --- # Analytical Method Validation ## When to use Any time the question is whether an analytical procedure is fit for its intended purpose: designing a validation study, evaluating validation data, verifying a compendial procedure, transferring a procedure to another laboratory or instrument, or defending any of these in a report. ## The two rules **1. Establish which framework governs before designing anything.** The same assay validates differently under ICH Q2(R2), USP <1225>, ICH M10, CLSI EP, and ISO/IEC 17025. They differ in which characteristics are required, how the studies are laid out, and whether numeric acceptance criteria are supplied at all. Blending them produces a protocol that satisfies none of them. **2. State acceptance criteria before collecting data.** Criteria chosen after seeing results are not acceptance criteria, and deciding them post hoc is a standing audit finding. ICH Q2(R2) deliberately supplies almost no numeric criteria — they have to come from the specification, the analytical target profile (ICH Q14 section 3), or development data. ICH M10 is the exception: it supplies explicit numbers, and they differ between chromatographic assays and ligand binding assays. ## Scope This skill plans studies, computes the statistics correctly, and structures the documentation. It does **not** decide that a procedure is validated, release a batch, accept or reject a run, close an investigation, or substitute for the analyst, the technical reviewer, the quality unit, or the regulator. Every script reports; none of them concludes. ## Copyright boundary ICH guidelines are published openly and licensed for reuse with acknowledgement, so their requirements are encoded directly in this skill. **USP general chapters, CLSI EP documents, and ISO standards are copyrighted and paywalled.** For those, this skill supplies the designation, scope, and where to obtain an authorised copy — never the text, never invented thresholds. Do not ask an agent to retrieve, transcribe, or reconstruct their content. If a number matters and it lives in a paywalled document, read it from the authorised copy. ## Frameworks ```bash cd skills/analytical-method-validation/scripts python3 plan_validation.py --list-frameworks ``` | Key | Governs | Numeric criteria supplied | | --- | --- | --- | | `ich-q2r2` | Release and stability testing of drug substances and products | Almost none — you derive them | | `ich-m10` | Bioanalytical concentration measurement (PK, TK, BE) | Yes, and they differ by modality | | `usp-1220` | Compendial procedure lifecycle, three stages | Paywalled | | `usp-1225` / `usp-1226` | Validation / verification of compendial procedures | Paywalled | | `clsi` | Clinical laboratory measurement procedures (EP series) | Paywalled | | `iso-17025` | Lab-developed and modified methods under accreditation | No — "to the extent necessary" | **Q2(R2) replaced Q2(R1) in November 2023 and restructured the characteristics.** Range is now the parent characteristic (section 3.2), containing *response* (linearity) and *validation of lower range limits* (DL/QL). Accuracy and precision are section 3.3 and may be evaluated in combination against a single criterion. Robustness is treated as a development activity and cross-refers to ICH Q14. Multivariate procedures are addressed explicitly (2.5 and 3.2.2.3), and Annex 2 adds worked examples for techniques Q2(R1) never covered — quantitative ¹H-NMR, NIR, quantitative LC/MS, qPCR, biological assays, and particle size. A Q2(R1)-shaped protocol — a flat list of linearity, range, accuracy, precision, specificity, LOD, LOQ, robustness — is out of date. Note also the error correction dated 30 November 2023 to Table 5 and Tables 6–11. ## Scripts ```bash cd skills/analytical-method-validation/scripts ``` | Script | Question answered | | --- | --- | | `plan_validation.py` | Which framework, which characteristics, what study layout, what protocol? | | `check_response.py` | Does the calibration model actually hold across the range? | | `check_accuracy_precision.py` | What is the recovery, and how much of the variability is between days? | | `check_detection_limits.py` | What are DL and QL by each allowed approach, and do they serve the reporting threshold? | | `check_bioanalytical_run.py` | Does this run meet ICH M10 for its modality? | | `compare_methods.py` | Are two procedures equivalent, at a pre-stated margin? | All take `--format table|tsv|json`. Provenance, guideline citations, and caveats go to stderr; data goes to stdout, so `> out.tsv` keeps them separate. Exit code is `0` for no findings, `1` when findings were raised, `2` for bad input — so any of them can gate a workflow. ## Workflow ### 1. Fix the framework and the required characteristics ```bash python3 plan_validation.py --framework ich-q2r2 --attribute assay --technique hplc --range-use assay ``` Q2(R2) Table 1 decides what is required from the *measured attribute*, not from the technique. For an assay: specificity, response, accuracy, repeatability, intermediate precision. For a limit test: specificity and DL only. For an identity test: specificity alone. Attributes accepted include `assay`, `impurity` (quantitative), `impurity-limit`, and `identity`. Reportable range comes from the specification. Q2(R2) Table 2 gives worked examples — 80–120% of declared content for an assay, 70–130% for content uniformity, reporting threshold to 120% of the specification for an impurity. ### 2. Generate the protocol and fill in the criteria ```bash python3 plan_validation.py --framework ich-q2r2 --attribute impurity --protocol > protocol.md ``` Every bracketed field is a decision to make and record *before* data collection. The protocol skeleton deliberately refuses to pre-fill acceptance criteria for Q2(R2) work, because there is no defensible default. ### 3. Evaluate the response ```bash python3 check_response.py -i calibration.csv --max-back-calc-error 2 ``` Input is `level,response`, one row per injection; repeated rows at the same level are replicates, and supplying them is what makes the linearity test possible. Real output from a curve that a coefficient of determination would wave through: ``` statistic value distinct levels 5 slope 166.6000 intercept 2495.0000 intercept CI includes 0 no coefficient of determination (r2) 0.9830 lack-of-fit F 469.5294 lack-of-fit p 1.5139e-06 runs test p 0.0492 level n mean_response mean_back_calculated relative_error_pct 50.0000 2 10075.0000 45.4982 -9.0036 75.0000 2 15150.0000 75.9604 1.2805 100.0000 2 20050.0000 105.3721 5.3721 125.0000 2 24050.0000 129.3818 3.5054 150.0000 2 26450.0000 143.7875 -4.1417 ``` r² = 0.983 and the model is unusable: −9.0% back-calculated error at the bottom of the range, lack-of-fit p = 1.5 × 10⁻⁶, non-random residual signs. **r² is not evidence of linearity** — it rises with range and is nearly insensitive to curvature. The lack-of-fit F test against pure error and the residual pattern are the evidence, which is why Q2(R2) 3.2.2.1 asks for an analysis of the deviation of points from the line rather than a correlation coefficient alone. Add `--weight 1/x2` for a wide-range curve. The script flags heteroscedasticity when the residual variance in the top third of the range exceeds the bottom third by more than 10×, because an unweighted fit then biases exactly the low end where a reporting threshold lives. ### 4. Evaluate accuracy and precision ```bash python3 check_accuracy_precision.py -i ap.csv --accuracy-limit 2 --rsd-limit 1.0 --design-check assay ``` Input is `level,measured,group`, where `group` is the intermediate-precision factor — day, analyst, or instrument. ``` level component sd rsd_pct df ci90_low_sd ci90_high_sd 100 repeatability (within group) 0.0707 0.0707 3 0.0438 0.2065 100 between-group 1.6515 1.6515 2 n/a n/a 100 intermediate precision (total) 1.6530 1.6530 2.0037 0.9554 7.2821 ``` Repeatability of 0.07% RSD looks superb; intermediate precision is 1.65%, twenty-three times larger, because the variability lives entirely between days. Reporting the within-day figure as the procedure's precision would understate routine performance by more than an order of magnitude. This is why the script fits a one-way random-effects model rather than pooling. Two traps the script handles for you: - **Precision is estimated within each level, never pooled across levels.** Pooling 80/100/120% results into one standard deviation turns the range itself into apparent imprecision. The script reports per level, plus a level-independent view as percent of nominal. - **`--require-ci-within-limit`** enforces that the whole confidence interval sits inside the limit, not just the mean. Q2(R2) 3.3.1.4 asks for the interval to be *compatible with* the criterion; a mean that scrapes inside on six replicates has not demonstrated much. ### 5. Establish DL and QL, and confirm them ```bash python3 check_detection_limits.py --calibration lowcal.csv --blanks blanks.csv \ --confirm-ql 0.05 --confirm-data ql_check.csv --reporting-threshold 0.05 ``` ``` approach sigma slope DL QL sd-and-slope (sigma = residual SD of regression) 7.2816 5033.3490 0.0048 0.0145 sd-and-slope (sigma = SD of y-intercept) 4.3303 5033.3490 0.0028 0.0086 sd-and-slope (sigma = SD of 8 blanks) 3.7702 5033.3490 0.0025 0.0075 ``` The same data give QL estimates spanning 1.9×, purely from the choice of σ. Q2(R2) 3.2.3.5 therefore requires the limit **and the approach used to determine it** to be reported, and an estimated limit to be confirmed with samples at or near it. For an impurity procedure the QL must be at or below the reporting threshold. Reaching for `3.3σ/slope` reflexively, reporting one number with no named approach, and never confirming it are three separate findings. ### 6. Bioanalytical runs under ICH M10 ```bash python3 check_bioanalytical_run.py --modality chromatographic --run run1.csv python3 check_bioanalytical_run.py --modality lba --isr isr.csv python3 check_bioanalytical_run.py --modality lba --criteria ``` `--modality` is mandatory and has no default, because the criteria genuinely differ: | | Chromatographic | Ligand binding assay | | --- | --- | --- | | Calibration tolerance | ±15%, ±20% at LLOQ | ±20%, ±25% at LLOQ and ULOQ | | Accuracy / precision | ±15% / ≤15% CV (±20% / ≤20% at LLOQ) | ±20% / ≤20% CV (±25% / ≤25% at LLOQ and ULOQ) | | A&P design | 4 QC levels, 5 replicates/run, ≥3 runs over ≥2 days | 5 QC levels, 3 replicates/run, ≥6 runs over ≥2 days | | Total error | no such criterion | ≤30%, ≤40% at LLOQ and ULOQ | | ISR agreement | ±20% for ≥2/3 of repeats | ±30% for ≥2/3 of repeats | Applying the ±15% chromatographic numbers to a ligand binding assay, or importing the LBA total-error criterion into a chromatographic method, are both common and both wrong. The run check enforces the per-level rule that gets missed: at least 2/3 of *all* QCs **and** at least 50% at *each* level. A run can pass the overall fraction while a single level fails completely. ``` finding: QC level high: 0/2 within tolerance (0%); M10 requires at least 50% at each level ``` ### 7. Transfer and method comparison ```bash python3 compare_methods.py -i paired.csv --margin 2 --relative --slope-tolerance 0.05 ``` ``` mean difference (%) 1.4646 TOST margin 2.0000 TOST p-value 1.0528e-13 90% CI (TOST) 1.44127 to 1.48797 equivalent at stated margin yes --- for contrast only --- paired t-test p (NOT equivalence) 0.0000 OLS slope (biased here) 1.0396 Deming slope 1.0398 Passing-Bablok slope 1.0351 ``` Two errors this replaces: - **"p > 0.05, no significant difference, therefore the methods are equivalent."** Failing to detect a difference is not evidence of equivalence, and on a small transfer dataset that outcome is close to guaranteed. TOST tests the hypothesis that matters — that the true difference lies inside a pre-stated margin. Here the t test says the difference is highly significant *and* TOST says the methods are equivalent at ±2%; both are true, and only one answers the question. - **Ordinary least squares for method comparison.** OLS assumes the reference values carry no error, which is false when comparing two procedures, and biases the slope toward zero. Deming (with a stated error-variance ratio) and Passing–Bablok (non-parametric, outlier-resistant) are the appropriate regressions and are reported side by side with OLS for contrast. The script also flags proportional bias — when the difference trends with concentration, a single mean bias and its limits of agreement are misleading regardless of how tight they look. ## What this skill exists to prevent 1. Validating against ICH Q2(R1)'s structure three years after Q2(R2) replaced it. 2. Acceptance criteria written after the data were seen. 3. r² presented as evidence of linearity. 4. Repeatability reported as the procedure's precision, with the between-day component invisible. 5. One DL/QL number with no named approach and no confirmation. 6. Chromatographic M10 criteria applied to a ligand binding assay, or the reverse. 7. A t test's non-significance presented as equivalence at a method transfer. ## References - `references/framework-selection.md` — which framework governs, and the questions that decide it - `references/ich-q2r2.md` — structure, Table 1 and Table 2, per-characteristic recommended data - `references/ich-m10-bioanalytical.md` — the full chromatographic and LBA criteria side by side - `references/compendial-and-clsi.md` — USP, CLSI and ISO designations, scope, and how to cite them - `references/statistics.md` — the statistical methods, why each one, and the common errors - `references/source-ledger.md` — provenance and research dates for every claim in this skill ## Assets - `assets/validation-protocol-template.md` — protocol structure with criteria stated up front - `assets/validation-report-template.md` — report structure with raw-data traceability