---
name: lingtai-ai/academic-research
source: https://app.decimal.ai/s/lingtai-ai-academic-research@2/SKILL.md
source_sha256: 4bc5f19ee355
---

# Academic Research

> **Nested swiss-knife reference.** A modular skill: try the bundled script
> first, then load specific reference files only when you need to escape it.

## Try this first

For 80% of "get me this paper" requests, the bundled script is the right answer.
It walks the open-access ladder, falls back automatically, and writes a manifest
the next session can resume from.

```bash
# Fetch by any identifier
python3 <skill-path>/scripts/fetch_paper.py 10.1103/PhysRevLett.125.015001
python3 <skill-path>/scripts/fetch_paper.py arXiv:2301.00001
python3 <skill-path>/scripts/fetch_paper.py PMID:12345678

# Batch (one identifier per line in the file)
python3 <skill-path>/scripts/fetch_paper.py --batch dois.txt --out papers/

# Resolve metadata only (no PDF download)
python3 <skill-path>/scripts/fetch_paper.py 10.1038/nature12373 --dry-run

# Skip LibGen (e.g. legal-sensitive environment)
python3 <skill-path>/scripts/fetch_paper.py <id> --no-libgen
```

**Output layout** (idempotent — re-runs skip entries with `status: ok`):

```
papers/{first-author-year-firstword}/
├── paper.pdf  |  paper.md      # full-text artifact
├── metadata.json                # CrossRef-normalized
└── manifest.json                # {status, tier, source, ts, doi}
```

**Tier ladder** (script stops at first hit):

| Tier | Source | Best for |
|------|--------|----------|
| 1 | arXiv direct | Preprints (physics, CS, math, q-bio, econ) |
| 2 | Unpaywall | Publisher-blessed gold/green OA |
| 3 | Europe PMC | Biomedical full-text + PMC mirror |
| 4 | CORE | Institutional repositories (needs `$CORE_API_KEY`) |
| 5 | Publisher-page extract | Nature/APS/AIP/IOP/Cambridge → in-house extractor (stdlib + `requests`, no third-party deps). Fetches the **already-accessible** official article / DOI landing page and parses `citation_*` metadata + the article body into structured Markdown. **No paywall/CAPTCHA bypass, no cookies/credentials** — official pages only. A login/paywall page is a clean miss; the ladder then falls through. Opt out with `--no-publisher-extract`. |
| 5b | Authorized publisher | **Licensed/institutional access only** — official DOI landing page → same-host publisher PDF → `%PDF-` validation, full provenance. Recovers paywalled-but-subscribed papers without shadow libraries. Never bypasses paywalls or handles credentials. Opt out with `--no-institutional`. See [authorized-publisher-access.md](reference/authorized-publisher-access.md). |
| 6 | LibGen | Last resort; opt out with `--no-libgen` |

**Set `$LINGTAI_RESEARCH_EMAIL` to a real address** before first use — Unpaywall
rejects placeholder emails with HTTP 422. The default falls back to
`lingtai-agent@example.org` with a warning.

Read on only if: the script fails on your paper, you need a custom query shape,
or you're composing a multi-step workflow (search → fetch → cite → write).

## Before drafting citation-bearing academic writing

> **Verify sources before you write prose.** If you are about to draft a paper,
> related-work section, literature review, References list, or any author–year /
> citation-bearing manuscript, pass the **evidence-verification gate first**:
> [reference/evidence-verification-gate.md](reference/evidence-verification-gate.md).
> No verified evidence → no confident prose. Peer-reviewed and preprint sources
> must not share the same evidence layer; search results are leads, not citations.
> If the user asks to "go fast," reduce *scope*, not *verification*. Produce a
> verified literature matrix **before** any submission-like draft.

## Escape hatch — quick paths

```
I'm about to write a paper / references / related-work section → reference/evidence-verification-gate.md (verify FIRST)
The user wants a "fast" / "readable" paper draft → reference/evidence-verification-gate.md (reduce scope, not verification)
I have a DOI                → reference/api-doi-resolver.md → api-crossref.md
I have an arXiv ID          → reference/api-arxiv.md (direct PDF link)
I have a PMID               → reference/api-europe-pmc.md
I have a bibcode            → reference/api-nasa-ads.md (requires free key)
I only have keywords        → reference/decision-tree.md → pick API by discipline
I need a citation network   → reference/api-semantic-scholar.md (single paper) or api-semantic-scholar-batch.md (large corpus) or api-openalex.md
I need to override the PDF ladder → reference/pipeline-obtain-pdf.md
Tier-5 publisher-extract failed and I want to retry it manually → reference/publisher-page-extraction.md
OA + authorized-publisher failed and I have a batch + the user's Zotero → reference/zotero-institutional-fulltext-handoff.md
All OA chains failed        → reference/libgen-fallback.md (last resort)
I need astrophysics         → reference/api-nasa-ads.md
I need high-energy physics  → reference/api-inspire-hep.md
I need biomedical           → reference/api-europe-pmc.md or api-pubmed.md
I need to write/compile a paper → reference/pipeline-latex-writing.md
My empirical draft keeps getting reframed / reviewers "agree" → reference/anti-pattern-text-consistency-vs-data-correspondence.md
I hit an API error          → reference/error-handling.md
```

## Reference index

### API references (13)

Each card includes endpoint parameters, runnable code, response shape, rate limits, and fallbacks.

| API | File | Best for | Key? |
|-----|------|----------|------|
| arXiv | [api-arxiv.md](reference/api-arxiv.md) | Preprint retrieval | No |
| CrossRef | [api-crossref.md](reference/api-crossref.md) | DOI metadata, funder queries | No (mailto recommended) |
| DOI Resolver | [api-doi-resolver.md](reference/api-doi-resolver.md) | Batch DOI → structured citation | No |
| OpenAlex | [api-openalex.md](reference/api-openalex.md) | Discovery, institution/concept analysis | No |
| Semantic Scholar | [api-semantic-scholar.md](reference/api-semantic-scholar.md) | Citation networks, TLDR | No (tight limits) |
| Semantic Scholar Batch | [api-semantic-scholar-batch.md](reference/api-semantic-scholar-batch.md) | Citation-graph construction from large ID lists | No (tight limits) |
| CORE | [api-core.md](reference/api-core.md) | OA full-text downloads | Optional (recommended) |
| PubMed | [api-pubmed.md](reference/api-pubmed.md) | Biomedical search, PMC full text | No |
| Unpaywall | [api-unpaywall.md](reference/api-unpaywall.md) | OA versions / PDFs | email (real) |
| Google Scholar | [api-google-scholar.md](reference/api-google-scholar.md) | Broadest discipline coverage | No (needs stealth) |
| Europe PMC | [api-europe-pmc.md](reference/api-europe-pmc.md) | Biomed, PMID, full-text XML | No |
| NASA ADS | [api-nasa-ads.md](reference/api-nasa-ads.md) | Astrophysics, BibTeX export | Yes (free) |
| INSPIRE-HEP | [api-inspire-hep.md](reference/api-inspire-hep.md) | High-energy physics | No |

### Pipeline workflows (6)

| Pipeline | File | Purpose |
|----------|------|---------|
| Paper discovery | [pipeline-discovery.md](reference/pipeline-discovery.md) | Keywords → candidate papers |
| PDF acquisition | [pipeline-obtain-pdf.md](reference/pipeline-obtain-pdf.md) | Metadata → full text (manual ladder) |
| Citation tracking | [pipeline-citation-tracking.md](reference/pipeline-citation-tracking.md) | Forward/backward citation networks |
| Scholar analysis | [pipeline-scholar-analysis.md](reference/pipeline-scholar-analysis.md) | Impact, trends, h-index |
| LaTeX writing | [pipeline-latex-writing.md](reference/pipeline-latex-writing.md) | Compile, bibliography, figures, debug |
| Decision tree | [decision-tree.md](reference/decision-tree.md) | "I have X — which API should I use?" |

### Standalone references

- [evidence-verification-gate.md](reference/evidence-verification-gate.md) — **verify-before-drafting gate** for citation-bearing academic writing: detection triggers, the verified-literature-matrix schema, A/B/C evidence tiers, "go fast = reduce scope not verification," matrix-before-draft artifact convention, pre-draft lint
- [publisher-page-extraction.md](reference/publisher-page-extraction.md) — Tier-5 manual escape hatch (Nature/APS/AIP/IOP/Cambridge → structured Markdown)
- [authorized-publisher-access.md](reference/authorized-publisher-access.md) — Tier-5b automated probe: official DOI landing page → same-host publisher PDF, licensed access only, no paywall/credential handling
- [zotero-institutional-fulltext-handoff.md](reference/zotero-institutional-fulltext-handoff.md) — Tier-6a human-in-the-loop: stage a failed batch into Zotero with tags, the **human** runs Find Full Text (institutional access), agent harvests PDFs back with provenance. No UI automation / TCC bypass / credential handling.
- [libgen-fallback.md](reference/libgen-fallback.md) — Last-resort PDF source with legal/safety notes
- [error-handling.md](reference/error-handling.md) — 429 backoff, 403 publisher blocks, timeout patterns
- [anti-pattern-text-consistency-vs-data-correspondence.md](reference/anti-pattern-text-consistency-vs-data-correspondence.md) — empirical-writing failure mode: prose drifts from the data while reviewer rounds make it more polished. Trigger pattern, re-anchoring steps, detection checklist.

## Relationship to built-in web search

- **`web-search-manual`** (returned by `web_search(action="manual")`) is the routing layer ("which tier to use for this URL?")
- **academic-research** is the deep-dive layer ("how do I write OpenAlex filter parameters? what email does Unpaywall want?")
- The two are complementary. If you're just scraping one publisher page and don't need the OA ladder, the built-in manual's `scripts/extract_page.py` is lighter.

## Known caveats

- **Unpaywall's `email` parameter is required and must be real** — placeholder addresses get HTTP 422. Set `$LINGTAI_RESEARCH_EMAIL` once.
- **CORE without an API key is harshly rate-limited** (~100/day vs 10,000/day with a free key from https://core.ac.uk/services/api).
- **Semantic Scholar free tier is very tight** (~100 req/day/IP, ~1 req/s practical). For large-corpus citation graphs use the batch endpoint — [api-semantic-scholar-batch.md](reference/api-semantic-scholar-batch.md). Request a key for any serious citation-network work.
- **Google Scholar requires a stealth browser** (camoufox or playwright-stealth v2); legacy `playwright_stealth` API does not work.
- **arXiv enforces HTTPS** — HTTP requests are 301-redirected automatically.
- **Library Genesis legality varies by jurisdiction** — use is the user's responsibility. Pass `--no-libgen` to opt out.
- **Publisher-page extraction (Tier 5)** — in-house stdlib+`requests` extractor (nothing to install; replaced the broken `zhiping0913/Download_paper`, issue #136). Fetches the already-accessible official landing page → structured Markdown. **No paywall/CAPTCHA bypass, no cookie/credential handling** — official pages only; login/paywall/unsupported-prefix/too-short is a clean miss. Heuristic (equations/figures may be lost). `--no-publisher-extract` to skip. Full contract: [reference/publisher-page-extraction.md](reference/publisher-page-extraction.md).
- **Authorized-publisher tier (5b)** — uses only access you already have: official DOI landing → same-host PDF, `%PDF-`+Content-Type validated. **Never bypasses paywalls/CAPTCHAs, never reads/stores/replays cookies/credentials** (cookies/auth never written to provenance). Most institutional access is IP-based, so a plain GET works on a licensed network and harmlessly misses off it. `--no-institutional` to disable. Full contract: [reference/authorized-publisher-access.md](reference/authorized-publisher-access.md).
- **Zotero institutional full-text (Tier 6a) is a human-in-the-loop handoff, not an agent tool** — for a **batch** where OA+5b both missed and the user runs Zotero Desktop on an institutional network: the agent stages rows via `/connector/saveItems` with a dated tag, the **human** clicks **Find Full Text**, the agent harvests PDFs back with provenance (copy-not-move, `resolved_by: human Find Full Text`). **Never drive the UI via AppleScript/Accessibility** (macOS TCC `-1719`/`-1743`, out of policy); no paywall bypass, no credential handling, no auto-LibGen. Full contract: [reference/zotero-institutional-fulltext-handoff.md](reference/zotero-institutional-fulltext-handoff.md).
- **Drafting citation-bearing academic writing without verifying sources first** — under "fast"/"readable draft" pressure the model flattens DOI-verified papers, preprints, and search leads into one confident evidence layer, which reads as submission-ready and invites reviewer rejection. Pass the [evidence-verification gate](reference/evidence-verification-gate.md) before drafting: verify a literature matrix first, keep A/B/C reliability tiers separate in the prose, and reduce scope (not verification) when speed is requested.
- **Writing an empirical paper iteratively can drift from the data** — reviewer rounds make prose more polished and internally consistent without verifying it still matches the experiments on disk. Reviewer agreement is text-consistency evidence, not data-correspondence evidence. Anchor every claim to data files/runner code *before* writing, and re-derive (don't just rewrite) when feedback flags confusion. See [reference/anti-pattern-text-consistency-vs-data-correspondence.md](reference/anti-pattern-text-consistency-vs-data-correspondence.md).

---
> **Found a bug or issue?** If you encounter any problems with this skill, load the `lingtai-issue-report` skill and follow its instructions to report it.