Delegation Masking: Why Your LangChain Callbacks Lie About Sub-Agent Failures
DEV Community

Delegation Masking: Why Your LangChain Callbacks Lie About Sub-Agent Failures

You delegate a task from Agent A to Agent B in LangChain. Agent B fails. Agent A's callback chain fires 'success' anyway. This is the observability blind spot most builders miss in agentic workflows: delegation masking. A sub-agent fails silently, but the parent agent's callback layer never knows because it only watches the delegation call itself, not what the delegated agent actually did.

Let's walk the mechanism.

The Delegation Pattern in LangChain

When you wire up agent-to-agent delegation in LangChain, you're typically using the @tool decorator to wrap a sub-agent invocation:

@tool
def delegate_to_classification_agent(task: str) -> str:
    """Delegate classification to a specialized sub-agent."""
    result = classification_agent.invoke({"input": task})
    return result.get("output", "")

The parent agent treats this as just another tool. It calls it, gets a result, moves on. The parent agent's callback chain - the layer that logs success/failure, fires alerts, measures latency - only sees the function return value, not the internal state of the sub-agent.

Where the Failure Hides

Here's what can happen:

  • Agent B (sub-agent) fails to produce valid output. Its internal chain breaks - maybe a tool call failed, or output parsing broke, or the LLM went silent. Agent B's callback chain logs the failure.
  • But the delegation function still returns something. Maybe it returns an empty string, a cached fallback, or a generic error message. It doesn't raise an exception - it just returns.
  • Agent A's callback layer sees HTTP 200. The delegation tool returned something, so the callback fires on_tool_end with status: "success". The parent agent logs success, moves on.

The actual failure is buried two layers deep. Agent A's monitoring sees "delegation succeeded." Only if someone digs into Agent B's logs does the failure surface.

This is a callback visibility boundary. The parent agent's instrumentation layer is one level too high to catch delegation failures.

The Signal That Catches It

Standard token-counting observability misses this because both agents might report partial token usage (Agent B started, burned some tokens, then failed). A success callback fired, so metrics look nominal.

What actually catches delegation failures:

1. Output validation at the delegation boundary

Track what the delegation function actually returned:

@tool
def delegate_to_agent(task: str) -> str:
    """Delegate with observability."""
    result = sub_agent.invoke({"input": task})
    output = result.get("output", "")
    # Signal: validate the output exists and is non-empty
    if not output or output.strip() == "":
        # This is a silent failure - the sub-agent ran but produced nothing
        log_event("delegation_produced_empty_output", {
            "delegated_agent": "classification_agent",
            "input": task,
            "sub_agent_status": result.get("status"),
            "token_count": result.get("usage", {}).get("output_tokens", 0)
        })
    return output

The signal is: a delegation that returned empty or unchanged input. The parent agent's callback sees "success," but observability knows something went wrong.

2. Cross-agent execution correlation

When Agent A calls Agent B, log a correlation ID that links both agents' execution traces:

import uuid

correlation_id = str(uuid.uuid4())

# In Agent A's tool:
result = sub_agent.invoke(
    {"input": task, "correlation_id": correlation_id},
    metadata={"correlation_id": correlation_id}
)

# In sub-agent's callback handler:
def on_tool_end(self, output: str, **kwargs):
    corr_id = kwargs.get("metadata", {}).get("correlation_id")
    if corr_id:
        log_event("agent_execution", {
            "correlation_id": corr_id,
            "agent": "sub_agent",
            "output_tokens": ...,
            "status": "success" or "failed"
        })

Now you can query: "What sub-agent executions have a correlation_id but show empty output or error status?" That's where delegation failures hide.

3. Aggregate success rate per agent pair

Track success rates at the delegation edge, not just per-agent:

delegation_success_rate = (
    executions where parent_agent = A and delegated_agent = B and output_validation_passed
) / (
    total executions where parent_agent = A and delegated_agent = B
)

If Agent A to Agent B delegation shows 95% success in parent logs but only 70% pass output validation, you've found the callback masking.

Why Frameworks Don't Catch This

LangChain's callback layer is designed to instrument the calling agent's perspective, not the called agent's internal state. That's by design - clean separation of concerns. But it means delegation failures are invisible until they propagate (or don't).

Most frameworks have the same boundary. CrewAI's task delegation, AutoGen's sub-agent calls - they all fire success callbacks when the call itself succeeds, regardless of what the called agent actually did.

The Observability Fix

You need one layer deeper:

  • Instrument sub-agents independently. Log their execution status, output validity, token usage. Don't rely on the parent agent's callback to know what happened.
  • Validate delegation outputs. Don't trust that a delegation function returning a value means the sub-agent actually succeeded. Check the output.
  • Correlate across agents. Link parent and child agent executions so failures propagate upward in your observability dashboard, not just downward in logs.

The callback chain is essential, but it's not enough. Delegation visibility requires you to see both sides of the boundary.

The takeaway: When agents delegate to other agents, callback success is not the same as execution success. Standard monitoring stays silent because the parent agent's callbacks only see the delegation call, not the delegated agent's actual work. Catch it by validating outputs, correlating executions across agents, and measuring success rates at the delegation edge.

Comments

No comments yet. Start the discussion.