Install any skill in seconds. Free to start, no credit card required.
Get Started Free →Python testing framework for writing simple, scalable, and powerful tests
.claude/skills/aiskillstore-pytest/SKILL.md| Test case | Without → With | Effect | Δ tokens | Δ turns |
|---|---|---|---|---|
| case-02 | ✗→✓ | ▲ Improved | 64% | 0% |
| case-11 | ✗→✓ | ▲ Improved | 51% | 0% |
| case-18 | ✗→✓ | ▲ Improved | 86% | 0% |
| case-19 | ✗→✓ | ▲ Improved | 107% | 0% |
| case-03 | ✓→✓ | = Same ✓ | 107% | 0% |
Pytest is a mature Python testing framework that makes it easy to write small tests while scaling to support complex functional testing.
python# test_example.py def test_addition(): assert 2 + 2 == 4 def test_string_operations(): assert "hello".upper() == "HELLO" assert "world" in "hello world"
bash# Run all tests uv run pytest # Run with verbose output uv run pytest -v # Run specific test file uv run pytest test_example.py # Run specific test function uv run pytest test_example.py::test_addition
Basic fixture definition:
pythonimport pytest @pytest.fixture def sample_data(): return {"name": "Alice", "age": 30} def test_user_data(sample_data): assert sample_data["name"] == "Alice" assert sample_data["age"] == 30
Fixture with setup and teardown:
python@pytest.fixture def database_connection(): # Setup conn = create_database_connection() yield conn # Teardown conn.close() def test_database_query(database_connection): result = database_connection.query("SELECT * FROM users") assert len(result) > 0
Fixture scopes:
python@pytest.fixture(scope="function") # Default - created per test def temp_file(): pass @pytest.fixture(scope="module") # Created once per module def module_resource(): pass @pytest.fixture(scope="session") # Created once per test session def session_resource(): pass
Basic parametrization:
python@pytest.mark.parametrize("input,expected", [ ("3+5", 8), ("2+4", 6), ("6*9", 54), ]) def test_eval(input, expected): assert eval(input) == expected
Parametrized fixtures:
python@pytest.fixture(params=["mysql", "postgresql", "sqlite"]) def database(request): if request.param == "mysql": return MySQLConnection() elif request.param == "postgresql": return PostgreSQLConnection() else: return SQLiteConnection() def test_database_operations(database): # Test runs 3 times, once for each database type result = database.execute("SELECT 1") assert result == 1
Stacking parametrization for combinatorial testing:
python@pytest.mark.parametrize("x", [0, 1]) @pytest.mark.parametrize("y", [2, 3]) def test_combinations(x, y): # Runs 4 times: (0,2), (0,3), (1,2), (1,3) assert x + y > 1
Basic async test:
pythonimport pytest @pytest.mark.asyncio async def test_async_function(): result = await async_operation() assert result is not None
Async fixtures:
python@pytest.fixture async def async_client(): client = AsyncClient() await client.connect() yield client await client.disconnect() @pytest.mark.asyncio async def test_async_api(async_client): response = await async_client.get("/api/data") assert response.status_code == 200
Using conftest.py for shared fixtures:
python# conftest.py @pytest.fixture def authenticated_client(): client = create_test_client() client.login("testuser", "password") return client @pytest.fixture(scope="session") def test_database(): db = create_test_database() yield db db.cleanup()
Test classes:
pythonclass TestUserAPI: def test_create_user(self, authenticated_client): response = authenticated_client.post("/users", json={"name": "John"}) assert response.status_code == 201 def test_get_user(self, authenticated_client): user_id = create_test_user() response = authenticated_client.get(f"/users/{user_id}") assert response.status_code == 200
Using monkeypatch fixture:
pythondef test_environment_variable(monkeypatch): monkeypatch.setenv("API_KEY", "test-key") assert get_api_key() == "test-key" def test_file_operations(monkeypatch, tmp_path): test_file = tmp_path / "test.txt" test_file.write_text("test content") monkeypatch.setattr("module.FILE_PATH", str(test_file)) assert read_file_content() == "test content"
Custom markers:
python# pytest.ini [tool:pytest] markers = slow: marks tests as slow integration: marks tests as integration tests unit: marks tests as unit tests # test_file.py @pytest.mark.slow def test_expensive_operation(): pass @pytest.mark.integration def test_database_integration(): pass
Running tests by marker:
bash# Run only unit tests uv run pytest -m unit # Skip slow tests uv run pytest -m "not slow" # Run integration or unit tests uv run pytest -m "integration or unit"
pythonimport pytest from fastapi.testclient import TestClient from myapp import app @pytest.fixture def client(): return TestClient(app) @pytest.fixture def test_user(): return {"username": "testuser", "email": "test@example.com"} def test_create_user(client, test_user): response = client.post("/users/", json=test_user) assert response.status_code == 201 assert response.json()["username"] == test_user["username"] def test_get_user(client, test_user): # Create user first create_response = client.post("/users/", json=test_user) user_id = create_response.json()["id"] # Get user response = client.get(f"/users/{user_id}") assert response.status_code == 200 assert response.json()["email"] == test_user["email"]
pythonimport pytest from sqlalchemy import create_engine from sqlalchemy.orm import sessionmaker @pytest.fixture(scope="function") def test_db(): engine = create_engine("sqlite:///:memory:") Session = sessionmaker(bind=engine) Base.metadata.create_all(engine) session = Session() yield session session.close() def test_user_creation(test_db): user = User(name="John", email="john@example.com") test_db.add(user) test_db.commit() retrieved_user = test_db.query(User).filter_by(name="John").first() assert retrieved_user.email == "john@example.com"
pythondef test_invalid_input_raises_error(): with pytest.raises(ValueError, match="Invalid input"): process_input("invalid") def test_file_not_found(): with pytest.raises(FileNotFoundError): read_nonexistent_file() def test_custom_exception(): with pytest.raises(CustomAPIError) as exc_info: call_api_endpoint() assert exc_info.value.status_code == 404 assert "not found" in str(exc_info.value)
uv add --dev pytest)uv add --dev pytest-asyncio)uv add --dev httpx for async HTTP tests)| Case | Status | Duration (ms) | Turns | Tokens | Tool calls | ||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Without | With | Δ | Without | With | Δ | Without | With | Δ | Without | With | Δ | ||
case-01 | fail→fail | 16,331 | 14,113 | -14% | 1 | 1 | 0% | 2,072 | 3,856 | +86% | 0 | 0 | — |
case-02 | fail→pass | 18,023 | 10,647 | -41% | 1 | 1 | 0% | 2,546 | 4,187 | +64% | 0 | 0 | — |
case-03 | pass→pass | 16,714 | 2,540 | -85% | 1 | 1 | 0% | 1,173 | 2,429 | +107% | 0 | 0 | — |
case-04 | pass→pass | 7,730 | 5,197 | -33% | 1 | 1 | 0% | 1,369 | 3,038 | +122% | 0 | 0 | — |
case-05 | pass→pass | 13,668 | 10,577 | -23% | 1 | 1 | 0% | 1,461 | 2,983 | +104% | 0 | 0 | — |
case-06 | pass→pass | 17,792 | 11,908 | -33% | 1 | 1 | 0% | 2,308 | 3,239 | +40% | 0 | 0 | — |
case-07 | pass→pass | 8,885 | 8,203 | -8% | 1 | 1 | 0% | 1,580 | 2,568 | +63% | 0 | 0 | — |
case-08 | pass→pass | 11,738 | 14,124 | +20% | 1 | 1 | 0% | 1,997 | 3,696 | +85% | 0 | 0 | — |
case-09 | pass→pass | 13,579 | 11,271 | -17% | 1 | 1 | 0% | 1,534 | 3,138 | +105% | 0 | 0 | — |
case-10 | pass→pass | 11,570 | 11,851 | +2% | 1 | 1 | 0% | 2,074 | 3,235 | +56% | 0 | 0 | — |
case-11 | fail→pass | 17,235 | 7,208 | -58% | 1 | 1 | 0% | 2,184 | 3,297 | +51% | 0 | 0 | — |
case-12 | pass→pass | 8,761 | 6,828 | -22% | 1 | 1 | 0% | 1,614 | 3,317 | +106% | 0 | 0 | — |
case-13 | pass→pass | 4,146 | 8,677 | +109% | 1 | 1 | 0% | 660 | 2,610 | +295% | 0 | 0 | — |
case-14 | pass→pass | 13,225 | 10,878 | -18% | 1 | 1 | 0% | 1,468 | 3,046 | +107% | 0 | 0 | — |
case-15 | pass→pass | 5,429 | 2,516 | -54% | 1 | 1 | 0% | 962 | 2,338 | +143% | 0 | 0 | — |
case-16 | pass→pass | 5,009 | 2,093 | -58% | 1 | 1 | 0% | 795 | 2,274 | +186% | 0 | 0 | — |
case-17 | pass→pass | 12,491 | 10,939 | -12% | 1 | 1 | 0% | 2,296 | 4,176 | +82% | 0 | 0 | — |
case-18 | fail→pass | 9,229 | 5,784 | -37% | 1 | 1 | 0% | 1,558 | 2,897 | +86% | 0 | 0 | — |
case-19 | fail→pass | 9,974 | 12,649 | +27% | 1 | 1 | 0% | 1,673 | 3,460 | +107% | 0 | 0 | — |
case-20 | pass→pass | 15,802 | 14,341 | -9% | 1 | 1 | 0% | 2,066 | 3,916 | +90% | 0 | 0 | — |
case-21 | pass→pass | 10,968 | 17,311 | +58% | 1 | 1 | 0% | 2,042 | 4,339 | +112% | 0 | 0 | — |
case-22 | pass→pass | 5,904 | 10,040 | +70% | 1 | 1 | 0% | 1,075 | 2,974 | +177% | 0 | 0 | — |
DecimalAI ran this skill against gemini-3.6-flash twice over the same eval suite — once with the skill loaded and once without — and compared the two runs case by case. 22 cases were attempted. The headline lift of +18 percentage points is the difference between those two pass rates over the 22 comparable cases.
Without the skill loaded, the model failed this case. With it loaded, the same prompt on the same model passed. This is one improved case from the latest verified run; every case, including any that regressed, is in the table above.
Other measured skills in the registry, with their headline benchmark lift.