Loading skill
Install any skill in seconds. Free to start, no credit card required.
Get Started Free →jeremylongshore/langchain-ci-integration
Wire LangChain 1.0 / LangGraph 1.0 tests into a GitHub Actions pipeline — unit tests with FakeListChatModel, VCR-gated integration tests, warning-filter policy, and eval-regression merge gates. Complements langchain-local-dev-loop (F23) which covers the inner loop; THIS covers the CI wire-up.
Passed 17 of 22 test cases with the skill, and 8 of 22 without it.
and 15 more in the full report
9 fixed8 passed either way5 still failing0 broke
Tokens, turns and time describe resource use, not a dollar price.
One of 9 test cases the skill fixed. Same model, same prompt, with and without the skill.
To prevent hanging test processes in CI, we want to enforce execution timeouts for both unit tests and integration test replays using pytest flags. What timeout limits should be specified in the GHA workflow step `run` commands?
| Checked | Without skill | With skill |
|---|---|---|
| Adds --timeout=30 flag to the unit test pytest execution step. | Passed | Passed |
| Adds --timeout=60 flag to the integration test pytest execution step. | Missed | Passed |
The instructions your model reads when the skill loads.
A PR passes every test on your laptop. You push. GHA runs pytest and aborts
during collection — before a single test executes — with:
PytestUnraisableExceptionWarning: Exception ignored in: ...
DeprecationWarning: langchain_community.llms ...
The org runs pytest -W error and a provider SDK emitted a DeprecationWarning
at import time, which the warning filter promoted to an exception while pytest
was still walking the test tree. This is P45 and it blocks every PR for the
team until someone pins a filterwarnings config.
Meanwhile the integration suite has its own failure mode: a VCR cassette
recorded three months ago at temperature=0 against Anthropic is now flaking
against a snapshot. temperature=0 is not deterministic on Claude — it still
nucleus-samples (P05) — so the cassette captured one valid completion, not
the valid completion. And yesterday a reviewer caught
Authorization: Bearer sk-ant-... in a cassette file that had been committed
six weeks ago (P44) because vcrpy records all request headers by default.
This skill covers the outer loop: the GitHub Actions workflow, the unit /
integration / eval gate separation, VCR cassette hygiene, pytest warning
policy, and a merge-blocking eval regression gate. The inner loop — fake
model fixtures, VCR recording workflow, local determinism tricks — lives in
langchain-local-dev-loop (F23); cross-reference it, do not duplicate it.
Pin: langchain-core 1.0.x, langgraph 1.0.x, actions/checkout@v4,
actions/setup-python@v5, vcrpy 6.x. Pain-catalog anchors: P05, P43, P44, P45.
langchain-core >= 1.0, < 2.0, langgraph >= 1.0, < 2.0pytest >= 8, pytest-asyncio, vcrpy >= 6 (integration)langchain-local-dev-loop (F23) applied locally — fixtures and recording workflowtest cases passed, vs 8 / 22 without
Published skills whose descriptions closely match this one, with their headline benchmark lift.
More skills from the same source and nearby in the registry, with their headline benchmark lift.