[copilot-cli-research] Copilot CLI Deep Research - 2026-08-27

Copilot CLI Deep Research Agent · issue · open

Filter2mode:review mode:live
All recorded Export JSON
github-actions[bot]

published Aug 27, 2026, 5:38 AM · updated Aug 27, 2026, 5:38 AM

Analysis Date: 2026-08-27
Repository: github/gh-aw
Scope: 294 total workflows, 108 using Copilot engine (id: copilot or engine: copilot)


📊 Executive Summary

Research Topic: Copilot CLI Optimization Opportunities (5th cycle)
Key Findings:

  1. engine.agent (custom agent personas) IS genuinely used — 7 workflows (archie, contribution-check, daily-file-diet, glossary-maintainer, hourly-ci-cleaner, technical-doc-writer, workflow-generator) — correcting a run-4 undercount that reported 0 due to an overly strict query.
  2. copilot-sdk: true adoption is flat at 61/108 workflows for a 4th consecutive measurement — genuinely plateaued, not growing.
  3. engine.version pinning, --share, and plugins remain at 0% adoption for a 5th consecutive cycle.
  4. shared/copilot-defaults.md still does not exist despite being recommended 3 cycles ago (run 2) and escalated in runs 3 and 4 — now the single most overdue action item.
  5. Timeout-minutes values are widely scattered (5 to 180 minutes) with no apparent standard tiering by workflow complexity.

Primary Recommendation: Create shared/copilot-defaults.md this cycle (lowest effort, highest leverage — unblocks standardized version pinning, harness tuning, and model defaults across all 108 Copilot workflows) or make an explicit decision to deprioritize it.

This is the 5th research cycle tracking Copilot CLI usage in gh-aw. The overall picture remains stable: teams are using the more "invisible" high-leverage features well (cache-memory: 80, repo-memory: 30, max-continuations, max-tool-denials: 55), but low-visibility CLI-level features (--share, --allow-all-paths, plugins, engine.version) continue to see zero organic adoption. The recurring theme across 5 cycles is not lack of awareness but lack of a low-friction on-ramp — there's no shared snippet or template nudging authors toward these features.


Critical Findings

🔴 High Priority Issues

  • shared/copilot-defaults.md not created (3rd cycle overdue): Recommended run 2 (08-24), escalated run 3 (08-25) and run 4 (08-26). Zero action across 3 cycles. This is the top unblocking item.
  • engine.version pinning at 0/108: No Copilot workflow pins a specific CLI version, meaning every run floats to latest — a supply-chain/reproducibility risk flagged for 5 consecutive cycles with no resolution.

🟡 Medium Priority Opportunities

  • copilot-sdk: true plateaued at 61/108 (56%): Growth stopped after run 2; the remaining 47 workflows use the classic CLI wrapper path. Worth investigating whether SDK mode has a known blocker for the holdouts.
  • plugins feature at 0% adoption across all 294 workflows: Documented and implemented (Plugins: true in copilot_engine.go), but genuinely unused. 5th cycle asking whether this feature is still a priority to promote, or should be deprioritized.
  • --share and --allow-all-paths at 0 manual adoption: Both fully supported CLI flags with no workflow opting in.

View Full Analysis

1️⃣ Current State Analysis

View Copilot CLI Capabilities Inventory

Copilot CLI Capabilities Inventory

Sourced from pkg/workflow/copilot_engine.go, copilot_engine_execution.go, copilot_engine_tools.go, copilot_mcp.go:

  • CLI flags available: --add-dir, --share (not found in execution code this cycle — likely compiler-internal or deprecated; flagged for verification), --disable-builtin-mcps, --no-ask-user, --agent <name>, --autopilot --max-autopilot-continues N, --log-level all --log-dir, --allow-all-paths, --no-custom-instructions, --headless --no-auto-update --host --port (SDK server mode)
  • Engine capabilities (EngineCapabilities struct): ToolsAllowlist, MCP, MaxTurns, MaxContinuations (via --autopilot), WebSearch: false (no built-in web search), BareMode (via --no-custom-instructions), BashCommandAllowlist (via --allow-tool shell(cmd)), Plugins: true (Agent Plugins)
  • Engine config options: engine.id, engine.version, engine.model, engine.agent (custom persona file), engine.args, engine.env, copilot-sdk: true (SDK/server mode), BYOK via COPILOT_PROVIDER_* env vars
  • MCP support: full MCP server rendering via copilot_mcp.go, GitHub MCP gateway proxy mode (github.mode: gh-proxy), MCP scripts
  • Sandbox: AWF (Agentic Workflow Firewall) and gVisor (sandbox.agent.runtime: gvisor) supported
  • Other: max-tool-denials, strict: true mode, harness retry script (copilot_harness.cjs) for transient CAPIError 400s
View Usage Statistics

Usage Statistics

  • Total Workflows: 294
  • Copilot Workflows: 108 (~37%)
  • cache-memory usage: 80 workflows
  • repo-memory usage: 30 workflows
  • copilot-sdk: true: 61 workflows (56% of Copilot workflows) — flat for 4 cycles
  • max-tool-denials: 55 workflows
  • engine.agent (custom persona): 7 workflows — archie, contribution-check, daily-file-diet, glossary-maintainer, hourly-ci-cleaner, technical-doc-writer, workflow-generator
  • engine.version pinning: 0 workflows (5th cycle confirming)
  • --share flag manual use: 0 workflows
  • plugins usage: 0 workflows across all 294
  • model overrides: 40+ workflows use model: at various nesting levels (some small/large aliases, some concrete slugs like copilot/gpt-5.4)
  • timeout-minutes: highly scattered — 30 (76×), 20 (47×), 10 (47×), 15 (37×), 45 (33×), 5 (17×), 60 (9×), 25 (8×), 90 (3×), 120 (2×), plus several one-offs (8, 35, 40, 180)

2️⃣ Feature Usage Matrix

Feature Category Available Features Used Not Used Usage Rate
CLI Flags --add-dir, --share, --disable-builtin-mcps, --no-ask-user, --agent, --autopilot, --allow-all-paths, --no-custom-instructions --add-dir (default, always), --disable-builtin-mcps (2), --agent (7), --autopilot/max-continuations (10+) --share, --allow-all-paths ~40%
Engine Config id, version, model, agent, args, env, copilot-sdk id (108), model (40+), agent (7), copilot-sdk (61) version pinning ~70%
MCP Servers GitHub MCP (gh-proxy), custom HTTP MCP, mcp-scripts GitHub MCP gh-proxy (widespread), mcp-scripts (some) high
Network Config network.allowed allowlists 172 workflows set explicit network: remaining ~122 rely on defaults 58%
Sandbox Options AWF, gVisor gVisor referenced in some (e.g. front-page-copy-guard) Most workflows don't specify sandbox.agent.runtime explicitly low-moderate
Plugins Agent Plugins via plugin install 0 all 294 0%

3️⃣ Missed Opportunities

View High Priority Opportunities

🔴 High Priority

Opportunity 1: Create shared/copilot-defaults.md (3rd cycle overdue)

  • What: A reusable shared snippet (like existing shared/otlp.md, shared/reporting.md) that bundles recommended Copilot engine defaults: pinned engine.version, sensible model default, and harness retry tuning.
  • Why It Matters: Currently every Copilot workflow author manually configures (or omits) version pinning and model choice. A shared import would standardize this in one place and make future CLI upgrades a single-file change instead of 108 individual edits.
  • Where: All 108 Copilot workflows could import it; new workflows would get sane defaults automatically.
  • How to Implement: Create .github/workflows/shared/copilot-defaults.md with a documented engine: block; update docs/src/content/docs/reference/engines.md to reference it as the recommended pattern.
  • Example:
    # shared/copilot-defaults.md
    ---
    engine:
      id: copilot
      version: "0.x.y"  # pin to last-verified-good release
    ---

Opportunity 2: Pin engine.version for Copilot workflows

  • What: None of the 108 Copilot workflows pin a CLI version; all float to latest.
  • Why It Matters: Reproducibility and supply-chain risk — an upstream Copilot CLI regression or breaking change can silently affect all 108 workflows simultaneously with no rollback path.
  • Where: All Copilot workflows, ideally centralized via Opportunity 1's shared snippet.
  • How to Implement: Add version: "<pinned>" under engine: in the shared defaults file, override per-workflow only when a newer feature is explicitly needed.
View Medium Priority Opportunities

🟡 Medium Priority

Opportunity 3: Investigate copilot-sdk adoption plateau (61/108, flat for 4 cycles)

  • What: SDK mode (copilot-sdk: true) growth stalled after an initial adoption wave.
  • Why It Matters: If SDK mode has real advantages (session control, structured logs) but the remaining 47 workflows aren't migrating, there may be a friction point (docs gap, missing example, perceived risk) worth investigating directly rather than re-measuring.
  • Where: Non-SDK Copilot workflows, e.g. daily-doc-updater.md.
  • How to Implement: Survey 3-5 non-SDK workflow authors or check for a known blocking bug; if none found, add a migration note to docs.

Opportunity 4: Decide fate of plugins feature (0% adoption, 5th cycle)

  • What: Agent Plugins capability is fully implemented (Plugins: true, GetPluginInstallationSteps) but has zero adoption across all 294 workflows.
  • Why It Matters: Either this is a genuinely valuable feature that needs promotion/examples, or it should be deprioritized to stop it showing up as a false "gap" every cycle.
  • Where: N/A — repo-wide policy decision needed.
  • How to Implement: Maintainer decision: (a) add a documented example workflow using plugins, or (b) mark as experimental/low-priority in docs and stop flagging in future research cycles.

Opportunity 5: Standardize timeout-minutes tiers

  • What: Timeout values are scattered across 12+ distinct values (5 through 180 minutes) with no visible tiering convention.
  • Why It Matters: Inconsistent timeouts make it hard to reason about expected workflow cost/duration and can mask misconfigured retries or unnecessarily generous budgets.
  • Where: Repo-wide.
  • How to Implement: Define 3-4 standard tiers (e.g., quick: 10, standard: 20-30, deep-research: 45-60, long-running: 90+) and document them in docs/src/content/docs/reference/engines.md or the shared defaults file.
View Low Priority Opportunities

🟢 Low Priority

Opportunity 6: Promote --share for conversation tracking

  • What: --share flag exists for shareable conversation links but is unused; note it wasn't found directly in copilot_engine_execution.go this cycle — worth confirming it's still exposed via engine.args passthrough rather than a first-class flag.
  • Why It Matters: Could aid debugging/support workflows if genuinely available; low priority since impact is narrow (debugging convenience).
  • Where: Debugging-focused workflows, or none if verification shows the flag isn't currently wired up.
  • How to Implement: Verify flag still exists in current CLI; if so, document an opt-in pattern via engine.args: ["--share"].

Opportunity 7: Model slug consistency

  • What: Mixed use of alias models (small, large) and concrete slugs (copilot/gpt-5.4, claude-haiku-4.5) across workflows.
  • Why It Matters: Minor developer-experience inconsistency; not a functional problem, but complicates auditing which workflows use which real model.
  • Where: Repo-wide.
  • How to Implement: Document a convention (e.g., prefer aliases for sub-agents, concrete slugs for primary agent) in the shared defaults file.

4️⃣ Specific Workflow Recommendations

View Workflow-Specific Recommendations

Workflow: daily-doc-updater.md

  • Current State: Uses engine: copilot without copilot-sdk: true or version pin.
  • Recommended Changes: Adopt shared/copilot-defaults.md once created; consider SDK mode.
  • Expected Benefits: Standardized configuration, reproducible CLI version.

Workflow: workflow-health-manager.md

  • Current State: Already uses repo-memory with custom max-file-size/max-patch-size tuning — a good example of advanced feature usage worth referencing in docs as a pattern.
  • Recommended Changes: None urgent; could be cited as a best-practice example in engine docs.
  • Expected Benefits: Documentation value for other authors.

Workflow: archie.md

  • Current State: Good example of engine.agent: adr-writer custom persona usage.
  • Recommended Changes: None; flag as a reference example for the 6 other agent-persona workflows and for documentation.
  • Expected Benefits: Helps close the awareness gap for engine.agent seen in earlier cycles' undercounting.

5️⃣ Trends & Insights

View Historical Trends

Comparing against runs on 08-23, 08-24, 08-25, and 08-26:

  • copilot-sdk: true: flat at 61 for a 4th consecutive measurement — confirmed genuine plateau, not noise.
  • engine.agent: corrected this cycle. Run 4 reported 0 due to an overly strict query; ground-truth check confirms 7 genuine engine.agent usages (archie, contribution-check, daily-file-diet, glossary-maintainer, hourly-ci-cleaner, technical-doc-writer, workflow-generator), consistent with runs 2-3's ~7 estimate.
  • shared/copilot-defaults.md: still not created — 3rd cycle since recommendation, 2nd cycle since escalation. This is the most persistent unactioned item across the research history.
  • engine.version pinning, --share, plugins: all confirmed at 0% for a 5th consecutive cycle.
  • max-tool-denials: stable at 55 (matches run 3/4).
  • cache-memory/repo-memory/total workflow counts: this cycle measured 80/30/294 for cache-memory/repo-memory/total, roughly matching run 4's figures (80/30/294), suggesting the run-4 methodology (vs. run 2/3's higher counts) is the more consistent baseline going forward.

6️⃣ Best Practice Guidelines

  1. Centralize engine defaults: Create and adopt a shared engine-config snippet (version, model, harness tuning) instead of repeating boilerplate per workflow — reduces upgrade friction and configuration drift.
  2. Pin CLI versions for reproducibility: Floating to "latest" across 108 workflows means a single upstream regression can affect the entire fleet simultaneously with no easy rollback.
  3. Tier timeout-minutes intentionally: Pick from a small set of standard values based on workflow complexity rather than ad-hoc numbers, to make cost/duration expectations legible at a glance.


7️⃣ Action Items

Immediate Actions (this week):

  • Create .github/workflows/shared/copilot-defaults.md with a pinned engine.version and default model (3rd cycle overdue)
  • Verify whether --share is still exposed as a first-class Copilot CLI flag or requires engine.args passthrough

Short-term (this month):

  • Survey 3-5 non-SDK Copilot workflows to identify why copilot-sdk: true adoption plateaued at 61/108
  • Document standard timeout-minutes tiers in engine reference docs

Long-term (this quarter):

  • Maintainer decision on plugins feature: promote with a working example, or deprioritize and stop tracking
  • Migrate all 108 Copilot workflows to import the new shared defaults snippet once created

View Supporting Evidence & Methodology

📚 References

  • Copilot Engine Documentation: docs/src/content/docs/reference/engines.md
  • Copilot Engine Implementation: pkg/workflow/copilot_engine.go, copilot_engine_execution.go, copilot_engine_tools.go, copilot_mcp.go
  • Related Workflows: archie.md, workflow-health-manager.md, copilot-cli-deep-research.md, craft.md
  • Previous Research: repo-memory branch memory/copilot-cli-research (runs 08-23, 08-24, 08-25, 08-26)

Research Methodology

Static analysis via grep/glob/view over the repository: enumerated Copilot-related Go source files, reviewed engine capability declarations and CLI flag construction in copilot_engine_execution.go, then cross-referenced against all 294 workflow markdown files under .github/workflows/ using targeted grep patterns for engine:, id: copilot, copilot-sdk, engine.agent, engine.version, cache-memory, repo-memory, network:, timeout-minutes, and CLI flag strings. Findings were cross-checked against 4 prior cycles of repo-memory notes to identify trends, confirm plateaus, and correct a prior cycle's engine.agent undercount via direct ground-truth spot-check.


Generated by Copilot CLI Deep Research (Run: 33042746396)

Generated by 🔬 Copilot CLI Deep Research Agent · copilot · auto · 37.7 AIC · ⌖ 8.73 AIC · ⊞ 10.9K ·

  • expires on Aug 27, 2026, 9:38 PM UTC-08:00