LangChain Local Dev Loop (Python)

LangChain Local Dev Loop (Python)

Overview

An engineer writes the most natural assertion possible:

def test_summarize():
    out = chain.invoke({"text": "..."})
    assert out.content == "expected summary"

It passes locally against Claude at temperature=0. It fails in CI on the third run with a one-token delta in the output. That is P05: Anthropic's temperature=0 is not greedy — it still samples. Tests against live Claude are not deterministic, period.

So the engineer swaps in FakeListChatModel(responses=["expected summary"]) and the assertion passes. Then the downstream callback that logs cost blows up in CI with KeyError: 'token_usage' — because FakeListChatModel does not emit response_metadata["token_usage"] (P43). Production code reads that key, so either the fake has to synthesize it or the test has to skip the callback.

Meanwhile, the first integration test under VCR records a cassette that ships Authorization: Bearer sk-ant-api03-... in the repo (P44). PR review catches it; the reviewer revokes the key; the dev loop is hosed for an afternoon.

And none of this matters if pytest cannot even collect the suite because import langchain_community emits a DeprecationWarning that -W error promotes to failure (P45).

This skill installs the four layers that make the whole loop fast and safe: FakeListChatModel / FakeListLLM with a metadata-emitting subclass (fixes P43); VCR with filter_headers plus a pre-commit hook (fixes P44); pytest filterwarnings policy in pyproject.toml (fixes P45); and an env-var-gated integration marker so the default pytest run never touches live APIs.

Speed targets: unit tests with FakeListChatModel run in < 100ms per test; VCR-replayed integration tests run in 500ms – 2s per test; live integration tests (the RUN_INTEGRATION=1 gate) run only in nightly or