[agentrx-optimizer] Daily Workflow Optimization

Executive Summary

AgentRx analyzed recent gh-aw agent session telemetry for github/gh-aw, cross-referencing the pre-downloaded 24h log bundle (302 runs) with a targeted logs MCP pull (workflow_name: "Metrics Collector", artifacts: ["agent"]) for the single most anomalous run in the window. Top finding: the Metrics Collector workflow (.github/workflows/metrics-collector.md) has a 100% failure rate over its last 2 runs and its most recent run burned 1,576,687 tokens (≈30–130x a typical workflow run) before the runner process itself died (failure_kind: driver_exit). The root cause traces to the workflow's own prompt instructing it to scan all workflows with no workflow_name filter and accumulate every paginated logs batch directly in the model's context across dozens of turns.

AgentRx Evidence

Critical step: repeated, unfiltered agenticworkflows.logs() calls inside the Metrics Collector agent loop (7–8 calls per run, preceded/interleaved with status() calls), reconstructed from the MCP tool-call trace and normalized into AgentRx IR.
Failure category: driver_exit (run #258, 2026-08-24) — process-level failure, distinct from the prior day's agent_logic failure (run #257, 2026-08-23). Both classified baseline by AgentRx's cross-run judge; 2 failures / 2 runs = 100% failure rate for this workflow in the analyzed window (flagged by AgentRx as a high-severity reliability hotspot).
Frequency / impact: Working-set telemetry for run #258 shows rebuild_factor: 25.88 — cumulative input tokens (1,550,038) were ~26x the single largest per-call payload (peak_input_tokens: 59,892) across 39 model invocations. This is the signature of context being re-sent near its peak size on almost every turn rather than growing incrementally, i.e. token-heavy, non-summarized context accumulation — not a runaway single call.
Representative run IDs: §32684298301 (driver_exit, 1.58M tokens, 20 min), §32613752970 (agent_logic, prior day, same workflow).

AgentRx Artifacts

IR summary: 2 trajectories built from MCP tool-call + working_set telemetry for the two most recent Metrics Collector runs (7 days of matching prior-day comparison not needed — both available runs in the 48h window failed). Each trajectory normalized cleanly into AgentRx's canonical IR (1 step, 11–13 substeps per run: status → logs → status → logs ×6-7 → noop → outcome), confirming the retry/pagination shape directly from tool-call timestamps.

Static / dynamic / check / judge stages: could not complete. The AgentRx static-invariant generator invokes the Copilot CLI endpoint (--endpoint copilot), which requires COPILOT_GITHUB_TOKEN/GH_TOKEN — unavailable in this sandbox ([CopilotCLI] exit 1: Error: No authentication information found). This blocked static, and downstream dynamic/check/judge/report stages depend on the static invariant output, so they failed with the same root cause. No invariant or judge output was fabricated to compensate — this section is limited to the completed ir stage plus raw MCP-sourced run telemetry (working_set, mcp_tool_usage.tool_calls, observability_insights), which is independently verifiable via the logs/audit tools.

Cross-run observability insights (from the logs MCP pull, not invented):

reliability / high: "Workflow metrics-collector accounted for 2 failure(s) across 2 run(s), a 100% failure rate."
tooling / medium: "Tool read_bash accounted for 100% of observed tool calls" — narrow capability path.
reliability / medium: "3 high-anomaly events across 2 runs" at the tool_result and plan pipeline stages (anomaly score 0.65, "new log template discovered; rare cluster") — consistent with a repeating pagination/retry loop rather than normal turn progression.

Known limitations: only 2 runs of this workflow existed in the analyzed window (both failed), so this is a small-N but 100%-consistent signal, not a large statistical sample. No ground-truth file was supplied, so judge accuracy scoring was unavailable regardless of the auth issue.

Recommended Optimization

Add a hard iteration/context-size guard to the Metrics Collector's own logs pagination loop, and stop scanning all workflows unfiltered.

.github/workflows/metrics-collector.md (lines 76–100, restated at 331–336) explicitly instructs the agent to:

call logs with "Include all workflows (no workflow_name filter)" (line 84), and
keep paginating via continuation "until the oldest collected run is at or before the 24h window start" (lines 86–92) — with no cap on the number of pagination turns or on cumulative context size, only a per-call count: 20 / timeout: 1 limit.

With 120+ active workflows in this repo, an unfiltered 24h scan needs many count: 20 batches, and every batch is appended to and re-sent as part of the growing conversation context on each subsequent model turn — exactly what rebuild_factor: 25.88 shows. This is the same anti-pattern this optimizer workflow's own instructions guard against ("Cap retries at 2 attempts per workflow... always include workflow_name").

Concrete fix (smallest meaningful change): after each logs batch, have the agent write the batch's extracted per-workflow counters to a scratch file (e.g. /tmp/gh-aw/agent/metrics-scratch.json) and drop the raw batch from context instead of accumulating it in-conversation; only re-read the running JSON aggregate, not the raw log payloads, on the next turn. Alternatively/additionally, add an explicit cap (e.g. "after 6 pagination batches, stop and fall back to the GitHub API path already described in this prompt") so a single run can't silently balloon to 1.5M+ tokens before the runner kills it.

Where to implement: .github/workflows/metrics-collector.md, section "Use Agentic Workflows Tool to Collect Workflow Metrics" (lines 73–100) and "Using Agentic Workflows Tool" (lines 329–336) — then recompile via the workflow's normal compile step to regenerate metrics-collector.lock.yml.

Validation Plan

Re-run Metrics Collector on the next scheduled trigger and confirm via audit/logs that conclusion is success (not driver_exit/agent_logic).
Check the new run's working_set.rebuild_factor — expect it to drop well below the current 25.88, ideally closer to 1–3x, since the aggregate JSON re-read each turn should be small relative to any single raw log batch.
Confirm TokenUsage for the run drops from ~1.5M tokens back toward the ecosystem's typical range (most workflows in this window used 5K–60K tokens; even the largest non-anomalous runs stayed under ~60K).
Confirm ActionMinutes drops from 20 back to a duration consistent with the 30-minute timeout-minutes budget having comfortable headroom.

References

§32684298301 — Metrics Collector, driver_exit, 1.58M tokens
§32613752970 — Metrics Collector, agent_logic, prior day
.github/workflows/metrics-collector.md — source prompt containing the unfiltered-scan instruction (line 84) and unbounded pagination loop (lines 86–92)

Generated by ⚡ Daily AgentRx Trace Optimizer · claude · agent · 106.4 AIC · ⌖ 8.03 AIC · ⊞ 8.8K · ◷

expires on Aug 30, 2026, 10:00 PM UTC-08:00