[agentrx-optimizer] Daily Workflow Optimization - 2026-08-26

Metrics Collector · issue · open

Filter2mode:review mode:live
All recorded Export JSON
github-actions[bot]

published Aug 26, 2026, 6:00 AM · updated Aug 26, 2026, 6:00 AM

Executive Summary

AgentRx analyzed the most recent Metrics Collector runs (100% failure rate: 2/2 runs in the sampled window) using harness-level telemetry (agentic-workflows logs/status MCP data plus raw agent-stdio traces), since the AgentRx LLM-backed stages (static/dynamic invariants, judge) could not authenticate against Copilot CLI in this sandbox. The top finding: the workflow's repo-memory tool config raises max-patch-size to 128 KB but never sets the separate max-file-size option, which silently stays at its 100 KB default — smaller than the metrics file the workflow itself produces. This causes a deterministic, recurring push failure as the ecosystem's workflow count grows.

A second, independent bottleneck was also found in the other failing run: a harness-level "context-rebuild circuit breaker" fired after the agent's cumulative input tokens hit 26x its peak single-turn size, killing an otherwise-successful run.

AgentRx Evidence

  • Critical step: push_repo_memory job → step "Push repo-memory changes (default)" (run §32802543912)
  • Failure category: Hard file-size gate violation (deterministic, code-level, not LLM-judged)
  • Frequency / impact: 2/2 (100%) of sampled Metrics Collector runs failed; this is a scheduled daily workflow, so the failure recurs every day and will not self-resolve — the underlying metrics payload only grows as more workflows are onboarded (314 workflows analyzed in this run alone).
  • Representative run IDs: §32802543912 (file-size gate failure), §32924199594 (context-rebuild circuit breaker)
AgentRx Artifacts

IR summary: Built trajectory.json (63 steps) from run 32924199594's actual agent-stdio.log events (agent_message / reasoning / command_execution / runtime-guard termination) and ran it through the AgentRx pipeline at /tmp/gh-aw/agent/agentrx/runs/gh-aw-daily. The ir stage completed successfully, auto-detecting domain=flash and normalizing the trace to trajectory_ir.json.

Invariant/checker stages: static, dynamic, and check all failed identically:

[CopilotCLI] exit 1: Error: No authentication information found.
RuntimeError: Static invariants JSON parse failed: Expecting value: line 1 column 1 (char 0)

No Copilot/Azure/TRAPI credentials were available in this sandbox, so no LLM-synthesized invariants or violations could be generated. This is a known environment limitation, not a data-availability problem.

Judge stage: Ran to completion but returned num_judges: 0 ("Single Pass: Empty response from LLM"), so AgentRx produced no root-cause classification for this trajectory. The report stage generated an empty predicted-failure-frequency plot as a result.

Known limitation: Because static/dynamic/judge require an authenticated LLM endpoint unavailable here, this report's evidence and recommendation are grounded directly in harness telemetry (GitHub Actions job/step conclusions, agentic-workflows logs MCP runs[] records, and raw agent-stdio.log runtime-guard messages) rather than AgentRx's synthesized invariants.

Corroborating telemetry for the second failure mode (run 32924199594, failure_kind: driver_exit):

working_set: { rebuild_factor: 26.23, cumulative_input_tokens: 1537802, peak_input_tokens: 58623, invocations: 41 }
runtime guard requested termination (context-rebuild circuit breaker tripped: rebuild_factor=26.23
  cumulative_input_tokens=1537802 thresholds=25/1000000) (SIGTERM)
runtime guard fired but process exited 0 — normalizing exit code to 1

The agent had already produced a valid noop result before being killed — the run was marked failure purely because of the token-rebuild guard, at a cost of 1.57M input tokens / 25 action-minutes for a task whose peak single-turn context was only 58.6K tokens (a rebuild factor far above the 25x threshold).

Recommended Optimization

Add max-file-size: 524288 (or similar, ≥ current + headroom) to the tools.repo-memory block in .github/workflows/metrics-collector.md.

Today the frontmatter only overrides max-patch-size (10 KB → 128 KB):

tools:
  repo-memory:
    branch-name: memory/meta-orchestrators
    file-glob: "metrics/**"
    max-patch-size: 131072 # 128KB - handles large daily metrics snapshots without patch-size gate failures

But pkg/workflow/repo_memory.go defines two independent size gates: MaxPatchSize (raised here) and MaxFileSize, which defaults to defaultRepoMemoryMaxFileSize = 102400 (100 KB) and was never touched. The push step enforces both, and the per-file check is what actually failed:

##[error]File exceeds size limit: metrics/daily/2026-08-25.json (359299 bytes > 102400 bytes)
##[error]File size validation failed

The comment in the workflow ("handles large daily metrics snapshots") shows the author believed raising max-patch-size alone was sufficient — it wasn't, because the two limits guard different things (total patch vs. single file).

Why this is highest impact: it is a deterministic, code-level gate (not a probabilistic LLM failure), it explains one of the two observed 100%-failure-rate runs directly from GitHub Actions step logs, and — unlike the context-rebuild issue — it will keep failing on every future run since the daily metrics JSON only grows as more workflows are onboarded (already 359 KB for 314 workflows). It is also a one-line frontmatter change plus a gh aw compile recompilation, far cheaper than redesigning the collection/pagination strategy.

Where to implement: .github/workflows/metrics-collector.md (frontmatter tools.repo-memory, currently lines 23–27), then recompile to regenerate .github/workflows/metrics-collector.lock.yml.

Secondary follow-up (not the primary fix, but same workflow): the logs-tool pagination loop this workflow's prompt mandates (repeated count:20/timeout:1 batches across ~117–314 workflows, "process logs in memory") is a plausible driver of the 26x context-rebuild factor seen in run 32924199594. If file-size fixes don't fully resolve the 100% failure rate, consider having the agent aggregate each batch's counts into a small on-disk running total (e.g. via jq) instead of accumulating every batch's raw JSON in its own context.

Validation Plan

  • Re-run (or wait for the next scheduled) Metrics Collector execution after the max-file-size change lands and lock file is recompiled; confirm the push_repo_memory job reaches conclusion: success.
  • Check that metrics/daily/YYYY-MM-DD.json is present on the memory/meta-orchestrators branch after the run (previously silently dropped on failure).
  • Success metric: Metrics Collector failure rate drops from 100% (2/2) to 0% over the next 5–7 daily runs, with push_repo_memory no longer appearing in job_details as failure.
  • If failures persist afterward, pull working_set.rebuild_factor for the next failing run via the logs MCP tool (artifacts: ["agent"]) to check whether the context-rebuild issue is now the dominant cause, and act on the secondary follow-up above.

References

Generated by ⚡ Daily AgentRx Trace Optimizer · claude · agent · 155.8 AIC · ⌖ 7.24 AIC · ⊞ 8.8K · ◷

  • expires on Sep 1, 2026, 10:00 PM UTC-08:00