Ready to supercharge your AI agents?

Install any skill in seconds. Free to start, no credit card required.

Python Resilience Patterns · dicklesworthstone | DecimalAI

dicklesworthstone/python-resilience

SkillScore96Maintained

Python resilience patterns including automatic retries, exponential backoff, timeouts, and fault-tolerant decorators. Use when adding retry logic, implementing timeouts, building fault-tolerant services, or handling transient failures.

codingby dicklesworthstone

Links this skill to your agents and keeps it current with the author’s updates.

Automatic

Coding pack

1 · One-time — install the CLI (docs ↗)

pip install decimalai

2 · Get the skill — writes .claude/skills/dicklesworthstone-python-resilience/SKILL.md

decimalai skills pull dicklesworthstone-python-resilience --out .claude/skills/

No CLI? Get the same file — curl · raw file

curl -fsSL --create-dirs -o .claude/skills/dicklesworthstone-python-resilience/SKILL.md https://app.decimal.ai/s/dicklesworthstone-python-resilience/SKILL.md

Raw file: /s/dicklesworthstone-python-resilience/SKILL.md ↗ · machine-readable summary at /s/dicklesworthstone-python-resilience.json ↗

In productionGlobal

how this skill behaves on real traffic · last 30 days

No production data for this skill yet — usage numbers appear here once agents start running it.

Anonymized aggregate across every org using this skill — proof from real production use, not a demo.

Does this skill help? · Benchmark: with vs. withoutsign in to run this on your agents

24 test cases from the publisher’s eval.yaml · each run twice on gemini-3.6-flash — with the skill injected, then without · last run 24d ago

Yes — +21% lift · at higher cost+127% tokens0% turns96% of whole cases pass with skill (23/24)

Test case	Without → With	Effect	Δ tokens	Δ turns
case-01	✗→✓	▲ Improved	28%	0%
case-03	✗→✓	▲ Improved	157%	0%
case-13	✗→✓	▲ Improved	132%	0%
case-16	✗→✓	▲ Improved	68%	0%
case-19	✗→✓	▲ Improved	162%	0%

Showing 5 of 24 cases

SkillSafety

✓ 3 of 3 passed

Code scan15 of 15 checks clean

0 critical · 0 warning · 0 info — scanner v4 — 8/10/2026

live_secretremote_code_executionreverse_shelldata_exfiltrationssrf_metadataagent_snoopinganti_refusaldestructive_commandsobfuscated_unicodeunicode_homoglyphopaque_blobsuspicious_urltool_overreachtrigger_abuseprompt_injection_phrasing

IntentSafe

LLM judge · does the skill do what it claims

ContentClean

LLM judge · policy & harmful-content review

Source

Imported

dicklesworthstone / pi_agent_rust

tests/ext_conformance/artifacts/agents-wshobson/python-development/skills/python-resilience/SKILL.md

View on GitHub ↗

License NOASSERTIONImported 8/10/2026Version v1

Skill contents

SKILL.md

Python Resilience Patterns

Build fault-tolerant Python applications that gracefully handle transient failures, network issues, and service outages. Resilience patterns keep systems running when dependencies are unreliable.

When to Use This Skill

Adding retry logic to external service calls
Implementing timeouts for network operations
Building fault-tolerant microservices
Handling rate limiting and backpressure
Creating infrastructure decorators
Designing circuit breakers

Core Concepts

1. Transient vs Permanent Failures

Retry transient errors (network timeouts, temporary service issues). Don't retry permanent errors (invalid credentials, bad requests).

2. Exponential Backoff

Increase wait time between retries to avoid overwhelming recovering services.

3. Jitter

Add randomness to backoff to prevent thundering herd when many clients retry simultaneously.

4. Bounded Retries

Cap both attempt count and total duration to prevent infinite retry loops.

Quick Start

python
from tenacity import retry, stop_after_attempt, wait_exponential_jitter

@retry(
    stop=stop_after_attempt(3),
    wait=wait_exponential_jitter(initial=1, max=10),
)
def call_external_service(request: dict) -> dict:
    return httpx.post("https://api.example.com", json=request).json()

Fundamental Patterns

Pattern 1: Basic Retry with Tenacity

Use the tenacity library for production-grade retry logic. For simpler cases, consider built-in retry functionality or a lightweight custom implementation.

python
from tenacity import (
    retry,
    stop_after_attempt,
    stop_after_delay,
    wait_exponential_jitter,
    retry_if_exception_type,
)

TRANSIENT_ERRORS = (ConnectionError, TimeoutError, OSError)

@retry(
    retry=retry_if_exception_type(TRANSIENT_ERRORS),
    stop=stop_after_attempt(5) | stop_after_delay(60),
    wait=wait_exponential_jitter(initial=1, max=30),
)
def fetch_data(url: str) -> dict:
    """Fetch data with automatic retry on transient failures."""
    response = httpx.get(url, timeout=30)
    response.raise_for_status()
    return response.json()

Pattern 2: Retry Only Appropriate Errors

Whitelist specific transient exceptions. Never retry:

ValueError, TypeError - These are bugs, not transient issues
AuthenticationError - Invalid credentials won't become valid
HTTP 4xx errors (except 429) - Client errors are permanent

python
from tenacity import retry, retry_if_exception_type
import httpx

# Define what's retryable
RETRYABLE_EXCEPTIONS = (
    ConnectionError,
    TimeoutError,
    httpx.ConnectTimeout,
    httpx.ReadTimeout,
)

@retry(
    retry=retry_if_exception_type(RETRYABLE_EXCEPTIONS),
    stop=stop_after_attempt(3),
    wait=wait_exponential_jitter(initial=1, max=10),
)
def resilient_api_call(endpoint: str) -> dict:
    """Make API call with retry on network issues."""
    return httpx.get(endpoint, timeout=10).json()

Pattern 3: HTTP Status Code Retries

Retry specific HTTP status codes that indicate transient issues.

python
from tenacity import retry, retry_if_result, stop_after_attempt
import httpx

RETRY_STATUS_CODES = {429, 502, 503, 504}

def should_retry_response(response: httpx.Response) -> bool:
    """Check if response indicates a retryable error."""
    return response.status_code in RETRY_STATUS_CODES

@retry(
    retry=retry_if_result(should_retry_response),
    stop=stop_after_attempt(3),
    wait=wait_exponential_jitter(initial=1, max=10),
)
def http_request(method: str, url: str, **kwargs) -> httpx.Response:
    """Make HTTP request with retry on transient status codes."""
    return httpx.request(method, url, timeout=30, **kwargs)

Pattern 4: Combined Exception and Status Retry

Handle both network exceptions and HTTP status codes.

python
from tenacity import (
    retry,
    retry_if_exception_type,
    retry_if_result,
    stop_after_attempt,
    wait_exponential_jitter,
    before_sleep_log,
)
import logging
import httpx

logger = logging.getLogger(__name__)

TRANSIENT_EXCEPTIONS = (
    ConnectionError,
    TimeoutError,
    httpx.ConnectError,
    httpx.ReadTimeout,
)
RETRY_STATUS_CODES = {429, 500, 502, 503, 504}

def is_retryable_response(response: httpx.Response) -> bool:
    return response.status_code in RETRY_STATUS_CODES

@retry(
    retry=(
        retry_if_exception_type(TRANSIENT_EXCEPTIONS) |
        retry_if_result(is_retryable_response)
    ),
    stop=stop_after_attempt(5),
    wait=wait_exponential_jitter(initial=1, max=30),
    before_sleep=before_sleep_log(logger, logging.WARNING),
)
def robust_http_call(
    method: str,
    url: str,
    **kwargs,
) -> httpx.Response:
    """HTTP call with comprehensive retry handling."""
    return httpx.request(method, url, timeout=30, **kwargs)

Advanced Patterns

Pattern 5: Logging Retry Attempts

Track retry behavior for debugging and alerting.

python
from tenacity import retry, stop_after_attempt, wait_exponential
import structlog

logger = structlog.get_logger()

def log_retry_attempt(retry_state):
    """Log detailed retry information."""
    exception = retry_state.outcome.exception()
    logger.warning(
        "Retrying operation",
        attempt=retry_state.attempt_number,
        exception_type=type(exception).__name__,
        exception_message=str(exception),
        next_wait_seconds=retry_state.next_action.sleep if retry_state.next_action else None,
    )

@retry(
    stop=stop_after_attempt(3),
    wait=wait_exponential(multiplier=1, max=10),
    before_sleep=log_retry_attempt,
)
def call_with_logging(request: dict) -> dict:
    """External call with retry logging."""
    ...

Pattern 6: Timeout Decorator

Create reusable timeout decorators for consistent timeout handling.

python
import asyncio
from functools import wraps
from typing import TypeVar, Callable

T = TypeVar("T")

def with_timeout(seconds: float):
    """Decorator to add timeout to async functions."""
    def decorator(func: Callable[..., T]) -> Callable[..., T]:
        @wraps(func)
        async def wrapper(*args, **kwargs) -> T:
            return await asyncio.wait_for(
                func(*args, **kwargs),
                timeout=seconds,
            )
        return wrapper
    return decorator

@with_timeout(30)
async def fetch_with_timeout(url: str) -> dict:
    """Fetch URL with 30 second timeout."""
    async with httpx.AsyncClient() as client:
        response = await client.get(url)
        return response.json()

Pattern 7: Cross-Cutting Concerns via Decorators

Stack decorators to separate infrastructure from business logic.

python
from functools import wraps
from typing import TypeVar, Callable
import structlog

logger = structlog.get_logger()
T = TypeVar("T")

def traced(name: str | None = None):
    """Add tracing to function calls."""
    def decorator(func: Callable[..., T]) -> Callable[..., T]:
        span_name = name or func.__name__

        @wraps(func)
        async def wrapper(*args, **kwargs) -> T:
            logger.info("Operation started", operation=span_name)
            try:
                result = await func(*args, **kwargs)
                logger.info("Operation completed", operation=span_name)
                return result
            except Exception as e:
                logger.error("Operation failed", operation=span_name, error=str(e))
                raise
        return wrapper
    return decorator

# Stack multiple concerns
@traced("fetch_user_data")
@with_timeout(30)
@retry(stop=stop_after_attempt(3), wait=wait_exponential_jitter())
async def fetch_user_data(user_id: str) -> dict:
    """Fetch user with tracing, timeout, and retry."""
    ...

Pattern 8: Dependency Injection for Testability

Pass infrastructure components through constructors for easy testing.

python
from dataclasses import dataclass
from typing import Protocol

class Logger(Protocol):
    def info(self, msg: str, **kwargs) -> None: ...
    def error(self, msg: str, **kwargs) -> None: ...

class MetricsClient(Protocol):
    def increment(self, metric: str, tags: dict | None = None) -> None: ...
    def timing(self, metric: str, value: float) -> None: ...

@dataclass
class UserService:
    """Service with injected infrastructure."""

    repository: UserRepository
    logger: Logger
    metrics: MetricsClient

    async def get_user(self, user_id: str) -> User:
        self.logger.info("Fetching user", user_id=user_id)
        start = time.perf_counter()

        try:
            user = await self.repository.get(user_id)
            self.metrics.increment("user.fetch.success")
            return user
        except Exception as e:
            self.metrics.increment("user.fetch.error")
            self.logger.error("Failed to fetch user", user_id=user_id, error=str(e))
            raise
        finally:
            elapsed = time.perf_counter() - start
            self.metrics.timing("user.fetch.duration", elapsed)

# Easy to test with fakes
service = UserService(
    repository=FakeRepository(),
    logger=FakeLogger(),
    metrics=FakeMetrics(),
)

Pattern 9: Fail-Safe Defaults

Degrade gracefully when non-critical operations fail.

python
from typing import TypeVar
from collections.abc import Callable

T = TypeVar("T")

def fail_safe(default: T, log_failure: bool = True):
    """Return default value on failure instead of raising."""
    def decorator(func: Callable[..., T]) -> Callable[..., T]:
        @wraps(func)
        async def wrapper(*args, **kwargs) -> T:
            try:
                return await func(*args, **kwargs)
            except Exception as e:
                if log_failure:
                    logger.warning(
                        "Operation failed, using default",
                        function=func.__name__,
                        error=str(e),
                    )
                return default
        return wrapper
    return decorator

@fail_safe(default=[])
async def get_recommendations(user_id: str) -> list[str]:
    """Get recommendations, return empty list on failure."""
    ...

Best Practices Summary

Retry only transient errors - Don't retry bugs or authentication failures
Use exponential backoff - Give services time to recover
Add jitter - Prevent thundering herd from synchronized retries
Cap total duration - stop_after_attempt(5) | stop_after_delay(60)
Log every retry - Silent retries hide systemic problems
Use decorators - Keep retry logic separate from business logic
Inject dependencies - Make infrastructure testable
Set timeouts everywhere - Every network call needs a timeout
Fail gracefully - Return cached/default values for non-critical paths
Monitor retry rates - High retry rates indicate underlying issues

Illustrative Want to see what dicklesworthstone/python-resilience would put in front of your model? Preview the rendered body and check its trigger phrases — no LLM call, no signup.

Latest benchmark · publisher runVerified

8/29/2026, 05:13 UTC · 24 cases

Lift

+21%

96% vs 75%

Agent turns

1.0 vs 1.0

Tokens

+127%

4,964.83 vs 2,188.83

fail → pass · 5pass → pass · 18fail → fail · 1

Per-case results (24)

Case	Status	Duration (ms)			Turns			Tokens			Tool calls
Case	Status	Without	With	Δ	Without	With	Δ	Without	With	Δ	Without	With	Δ
case-01 I am building a Python service that communicates with a volatile third-party API using `httpx`. I need a function that makes HTTP requests while automatically retrying whenever transient connection issues or temporary server-side error codes occur. Please write a Python function using `tenacity` decorated with appropriate exception checks and status code predicates, including jittered exponential backoff and attempt limits. The function should take a method and URL and return the `httpx.Response` object.	fail→pass	21,987	21,901	-0%	1	1	0%	4,346	5,567	+28%	0	0	—
case-02 When integrating with external web APIs, client errors like HTTP 400 Bad Request or HTTP 401 Unauthorized indicate invalid input or bad credentials. A developer suggests configuring an automated retry decorator to catch all non-200 responses up to 5 times. Explain why client error codes must not be retried, and write a predicate function specifying which status codes should be retried.	pass→pass	13,964	12,292	-12%	1	1	0%	2,423	5,215	+115%	0	0	—
case-03 Our microservices experience thundering herd traffic bursts after downstream service outages because retries use fixed intervals. We want to configure a tenacity retry decorator to wait between attempts starting at 1 second up to 30 seconds, but with randomness added to spread out client attempts. Which wait function should be used in the decorator? Write the decorator.	fail→pass	8,309	7,614	-8%	1	1	0%	1,528	3,923	+157%	0	0	—
case-04 To prevent background worker threads from retrying indefinitely during prolonged outages, we need a tenacity decorator that caps retries at EITHER 5 attempts OR a maximum total elapsed duration of 60 seconds. How should these two stopping conditions be combined in the decorator? Show the code.	pass→pass	7,459	9,416	+26%	1	1	0%	1,324	4,007	+203%	0	0	—
case-05 In our Python application, a data pipeline raises ValueError for bad data formats, AuthenticationError for expired API keys, and TimeoutError or ConnectionError for network failures. A developer suggests decorating the function with @retry(retry=retry_if_exception_type(Exception)). Refine this retry configuration to handle exceptions correctly.	pass→pass	10,464	12,920	+23%	1	1	0%	1,839	4,807	+161%	0	0	—
case-06 A team member suggests logging retries by wrapping the function call in a try/except block outside the retry decorator. However, we want tenacity to automatically log a warning message right before it sleeps between retry attempts using our standard Python logger. How can tenacity's pre-sleep logging hook be configured on the retry decorator? Show the decorator configuration.	pass→pass	17,525	6,330	-64%	1	1	0%	1,024	4,033	+294%	0	0	—
case-07 We need a custom callback function to inspect retry details during failures. A developer proposed passing a function that accepts (attempt, exception) as two positional arguments. What parameter structure does tenacity actually pass to a custom before_sleep callback function, and how do you extract the current attempt number and the raised exception from that parameter? Write the callback function.	pass→pass	10,251	9,312	-9%	1	1	0%	1,923	4,651	+142%	0	0	—
case-08 We have three custom async decorators: @traced("fetch_user_data") (adds logging/tracing), @with_timeout(30) (enforces a 30s timeout per call), and @retry(...) (handles transient retries). A developer placed @retry as the outermost top decorator above @traced and @with_timeout. Is this order correct if each individual fetch attempt should be timed out and traced? Show the correct decorator stacking order on an async function.	fail→fail	15,158	20,148	+33%	1	1	0%	2,784	6,706	+141%	0	0	—
case-09 We need a custom, reusable Python decorator with_timeout(seconds) for async functions that cancels execution if a function takes longer than the specified time in seconds. Implement this decorator using standard Python standard libraries and functools.wraps.	pass→pass	9,829	13,542	+38%	1	1	0%	1,975	4,821	+144%	0	0	—
case-10 In our API backend, fetching user recommendation lists is a non-critical feature. If the recommendation service fails due to network or backend errors, we want the system to catch any exception, log a warning, and gracefully return an empty list [] instead of raising an error to the caller. Write a generic Python decorator fail_safe(default=...) that implements this fallback behavior.	pass→pass	16,882	14,126	-16%	1	1	0%	2,971	5,329	+79%	0	0	—
case-11 We are building a UserService class in Python that needs to fetch user data, log operations, and emit metrics. Instead of calling global singletons or instantiating hardcoded loggers and metrics clients inside the method, how should infrastructure dependencies be structured in the class design to allow seamless unit testing with fakes? Write a dataclass implementation showing this architecture.	pass→pass	16,251	14,698	-10%	1	1	0%	3,080	5,774	+87%	0	0	—
case-12 When calling a remote API that returns an httpx.Response object, we want tenacity to retry whenever the returned response object has a status code in {429, 502, 503, 504}. Write a predicate function should_retry_response and show how to wire it into the @retry decorator.	pass→pass	9,828	5,668	-42%	1	1	0%	1,920	4,047	+111%	0	0	—
case-13 We want a single tenacity @retry decorator that retries if EITHER a transient network exception occurs OR the function returns an HTTP response with a status code in {429, 500, 502, 503, 504}. How are exception conditions and result conditions combined in a tenacity decorator? Write the decorator.	fail→pass	11,486	9,426	-18%	1	1	0%	2,112	4,904	+132%	0	0	—
case-14 A developer implemented an API integration using httpx.get decorated with tenacity retries, but omitted the timeout parameter on the httpx.get call under the assumption that tenacity's retry decorator handles all timing issues. Explain why omitting client-level timeouts is dangerous and show the correct httpx.get call.	pass→pass	15,056	14,875	-1%	1	1	0%	2,478	5,470	+121%	0	0	—
case-15 Our background service retries failed HTTP calls up to 5 times. While end users rarely see failures because retries usually succeed, underlying network instability goes unnoticed until major service blackouts occur. What architectural practice should be added to retry handling to detect degraded dependencies early?	pass→pass	12,653	16,009	+27%	1	1	0%	2,058	5,605	+172%	0	0	—
case-16 Write a Python function that makes a network call, decorated with tenacity @retry configured to stop after exactly 3 attempts and wait with exponential backoff starting at 1 second up to 10 seconds with jitter.	fail→pass	10,324	4,547	-56%	1	1	0%	2,274	3,816	+68%	0	0	—
case-17 Write an async decorator traced(name=None) in Python that logs an info message before invoking the wrapped function, logs an info message upon successful completion, and logs an error message if an exception occurs before re-raising the exception.	pass→pass	12,450	9,910	-20%	1	1	0%	2,233	4,944	+121%	0	0	—
case-18 Modify a generic fail_safe fallback decorator so callers can pass log_failure=False when wrapping non-critical batch calls where exceptions are expected and logging would clutter output. Write the decorator function.	pass→pass	12,253	13,111	+7%	1	1	0%	2,224	4,955	+123%	0	0	—
case-19 A team configured an API client retry loop using fixed 5-second delays (wait_fixed(5)). During an upstream outage, 200 service instances retried on identical 5-second cycles, overloading the server as soon as it recovered. How should the delay strategy be adjusted to prevent synchronized client traffic?	fail→pass	9,198	8,537	-7%	1	1	0%	1,716	4,503	+162%	0	0	—
case-20 We are defining a tuple of retryable network exceptions for httpx HTTP calls to pass into retry_if_exception_type. Which specific exception types from standard Python and httpx should be included in this tuple to handle transient socket and read failures while excluding client code errors?	pass→pass	17,520	16,801	-4%	1	1	0%	2,909	6,221	+114%	0	0	—
case-21 A microservice developer asks whether retries with exponential backoff are sufficient when an upstream service experiences a total, multi-hour outage. What complementary pattern prevents flooding a completely dead service with retries, and when should it trip?	pass→pass	11,462	14,987	+31%	1	1	0%	1,870	5,333	+185%	0	0	—
case-22 We are configuring an asyncpg connection pool in Python for an asynchronous web service expected to handle 500 concurrent requests. We need to set up asyncpg.create_pool with appropriate parameters for minimum pool size, maximum pool size, and connection timeout. Write a Python snippet that configures asyncpg.create_pool with these parameters.	pass→pass	11,896	13,229	+11%	1	1	0%	2,162	5,101	+136%	0	0	—
case-23 We have a Python application using Celery for asynchronous task processing. Long-running tasks are stalling because fast workers prefetch too many tasks at once. How should worker_prefetch_multiplier and task_acks_late be configured in Celery settings to ensure fair task distribution across workers? Show the Celery configuration dictionary or settings.	pass→pass	11,757	11,094	-6%	1	1	0%	1,965	4,984	+154%	0	0	—
case-24 Our Python application connects to a MySQL database via SQLAlchemy create_engine. Sockets drop after 8 hours of inactivity due to MySQL's wait_timeout, causing intermittent OperationalError on idle connections. How should SQLAlchemy's engine pool be configured with pool_recycle and pool_pre_ping to automatically recycle stale database connections? Write the create_engine call.	pass→pass	7,913	8,516	+8%	1	1	0%	1,394	4,440	+219%	0	0	—

These cases were also screened by the skill’s intent / safety pass.See the Trust review ↗Reproduce this benchmark in Colab ↗

How this skill was measured

DecimalAI ran this skill against gemini-3.6-flash twice over the same eval suite — once with the skill loaded and once without — and compared the two runs case by case. 24 cases were attempted. The headline lift of +21 percentage points is the difference between those two pass rates over the 24 comparable cases.

Model measured: gemini-3.6-flash
Graded by: gemini-3.6-flash
Grading method: judged
Verification: verified
Pass rate without the skill: 75%
Pass rate with the skill: 95.8%
Cases attempted: 24
Cases counted toward lift: 24
Cases that regressed: 0
Skill version measured: v1
Run completed: 8/29/2026, 05:13 UTC

Measured example — same prompt, with vs. without the skill · from the benchmark run · case case-01

Prompt

I am building a Python service that communicates with a volatile third-party API using `httpx`. I need a function that makes HTTP requests while automatically retrying whenever transient connection issues or temporary server-side error codes occur. Please write a Python function using `tenacity` decorated with appropriate exception checks and status code predicates, including jittered exponential backoff and attempt limits. The function should take a method and URL and return the `httpx.Response` object.

Without the skill loaded, the model failed this case. With it loaded, the same prompt on the same model passed. This is one improved case from the latest verified run; every case, including any that regressed, is in the table above.

Show both answers

SkillScore blends 4 signals1 of 4 complete

BenchmarkPass rate WITH skill on a verified run96

Live evalNeeds ≥10 evaluated traces in 30 days

AI ratingNeeds ≥10 rated traces

AdoptionNeeds ≥10 serves across ≥2 orgs in 30 days

Efficiency · separate axis (at equal correctness)+0% turns+138% tokens

Shown for transparency; not credited unless a verified run shows no correctness regression.

Backed by 1 of 4 signals — each added signal makes the score more robust.

Verified on gemini-3.6-flash · re-tested 23 days ago

Other measured skills in the registry, with their headline benchmark lift.