Self-Healing Plugin

Automatic retry with reflection, exponential backoff, and circuit breaker.

Overview

Implements resilient execution patterns:

• Tenacity-based retry: Exponential backoff with configurable limits
• Circuit breaker: Prevent cascading failures
• Error classification: Smart retry decisions based on error type
• LLM reflection: Modify arguments based on error analysis

Architecture

Function Call
    |
    v
Circuit Breaker Check
    |
    +--> OPEN --> Return error immediately
    |
    v
Execute with Retry
    |
    +--> Success --> Record success, return result
    |
    v
Error Occurred
    |
    +--> Classify Error
            |
            +--> PERMANENT --> Don't retry
            |
            +--> TRANSIENT --> Retry with backoff
            |
            +--> RECOVERABLE --> Reflect & modify args, retry

Components

| File | Purpose | |------|---------| | healer.py | Main SelfHealingExecutor with retry logic | | circuit_breaker.py | Circuit breaker implementation | | decorator.py | Function decorator for easy integration |

Configuration

# Enable/disable self-healing
DRYADE_SELF_HEALING_ENABLED=true

Enable LLM-based reflection for argument modification
DRYADE_REFLECTION_ENABLED=true
Maximum retry attempts
DRYADE_RETRY_MAX_ATTEMPTS=3

Usage

Basic Usage

from plugins.self_healing import execute_with_healing

result = await execute_with_healing(
    func=my_api_call,
    args={"query": "search something"},
    circuit_name="my_api"
)
if result.status == "ok":
    print(result.result)
else:
    print(f"Failed: {result.error}")
    print(f"Healing actions: {result.healing_actions}")

With Fallback

result = await execute_with_healing(
    func=primary_api_call,
    args={"query": "search"},
    fallback=backup_api_call,
    circuit_name="primary_api"
)

Decorator Usage

from plugins.self_healing import execute_with_self_healing

response = await execute_with_self_healing(
    llm_func,
    messages=messages,
    circuit_name="llm_api"
)

Error Classification

Transient Errors (Retry immediately)

• Connection errors
• Timeout errors
• Rate limiting (429, 503, 504)
• "Service unavailable"

Permanent Errors (Don't retry)

• Invalid API key (401)
• Forbidden (403)
• Not found (404)
• Context length exceeded

Recoverable Errors (Retry with modification)

• Malformed output
• Invalid parameters
• Type mismatches

Circuit Breaker

Prevents overwhelming failing services:

from plugins.self_healing import get_circuit_breaker

breaker = get_circuit_breaker("my_service")
States: CLOSED (normal), OPEN (failing), HALF_OPEN (testing)
if breaker.can_execute():
    try:
        result = await call_service()
        breaker.record_success()
    except Exception:
        breaker.record_failure()

Default configuration:

• Failure threshold: 5 failures
• Recovery timeout: 60 seconds
• Half-open test requests: 1

LLM Reflection

When enabled, uses LLM to analyze errors and suggest argument modifications:

Error: "Invalid date format"
Current args: {"date": "01-19-2026"}

LLM Reflection:
"The date format appears incorrect. Suggest changing to ISO format."
Modified args: {"date": "2026-01-19"}

HealingResult

@dataclass
class HealingResult:
    status: str  # "ok" or "error"
    result: Any = None
    error: str | None = None
    attempts: int = 1
    healed: bool = False
    healing_actions: list[str] = field(default_factory=list)
    circuit_breaker_triggered: bool = False

Dependencies

• tenacity: Retry library with exponential backoff