[copilot-cli-research] Copilot CLI Deep Research - 2026-08-26

Copilot CLI Capabilities Inventory (from pkg/workflow/copilot_engine_execution.go, copilot_engine.go, copilot_mcp.go)

• Always-on/compiler-managed flags: --add-dir (workspace, /tmp/, cache/drive-memory dirs), --log-level all, --log-dir, --headless, --no-auto-update (SDK server mode), --host/--port (SDK mode).
• Conditional flags:
  • --disable-builtin-mcps — added unless workflow uses web-fetch tool
  • --no-ask-user — added when CLI version supports it (v1.0.19+), for autonomous runs
  • --agent <name> — set only via engine.agent, not via agent-file imports
  • --autopilot --max-autopilot-continues N — set when engine.max-continuations > 1
  • --allow-all-paths — auto-added when edit tool enabled
  • --no-custom-instructions — added when engine.bare: true
• Engine config surface: id, version, model, args (raw passthrough), env, agent, max-continuations, bare, max-tool-denials, harness options, copilot-sdk: true.
• Memory/storage integration: cache-memory and repo-memory both auto-wired into --add-dir.
• BYOK support: COPILOT_PROVIDER_* env vars for custom model providers (seen in smoke-copilot-aoai-*.md, smoke-copilot-mai.md).
• MCP integration: pkg/workflow/copilot_mcp.go configures MCP servers for the Copilot CLI transport.

Usage Statistics (this cycle vs. prior cycles)

Note on count discrepancies: The id: copilot and cache-memory counts dropped this cycle versus runs 2–3 (109→89, 105→80) while total workflow count rose slightly (292→294). This is very likely a query-methodology difference (this cycle used stricter nested-block matching) rather than an actual adoption regression — flagged so the next cycle can reconcile the counting method and confirm.

🔴 High Priority

Opportunity 1: Create shared/copilot-defaults.md (escalated 2nd time — no action after 08-24 recommendation)

• What: A shared import snippet bundling common Copilot engine settings (e.g. a pinned version, standard max-tool-denials, log verbosity) that workflows can imports: instead of repeating ad hoc.
• Why It Matters: 89 Copilot workflows each independently decide these settings; a systemic fix (e.g., responding to a CLI regression) currently requires touching every file individually.
• Where: All id: copilot workflows, especially the daily-* and smoke-copilot-* families that already share structure via shared/ imports for other concerns (otlp, graders).
• How to Implement: Add .github/workflows/shared/copilot-defaults.md with an engine: fragment (version, standard args) following the existing shared/otlp.md import pattern; migrate 3–5 pilot workflows to validate before wider rollout.
• Example:

  # shared/copilot-defaults.md
  ---
  engine:
    id: copilot
    version: "0.x.y"   # pin to last-known-good
    max-tool-denials: 3
  ---

Opportunity 2: Decide on engine.version pinning policy (0/89, 4 cycles)

• What: No Copilot workflow pins engine.version; all float on latest.
• Why It Matters: For compliance/security-sensitive workflows (security-compliance.md, daily-malicious-code-scan.md, daily-secrets-analysis.md), an unannounced CLI behavior change could silently affect audit results.
• Where: At minimum, the security/compliance-tagged Copilot workflows.
• How to Implement: Either (a) explicitly document that "always latest" is the intended policy for this repo (closing the gap by decision, not code), or (b) pin version on the handful of compliance-critical workflows as a pilot.

🟡 Medium Priority

Opportunity 3: Clarify engine.agent vs. agent-file imports in future research

• What: This cycle's stricter query found 0 workflows setting engine.agent (the flag that actually triggers --agent), versus ~7 in prior cycles that may have matched imports: [.github/agents/*.md] instead.
• Why It Matters: 12 custom agent files exist in .github/agents/ but if none actually set the --agent CLI flag, that authoring investment may be under-leveraged.
• Where: N/A — this is a research methodology fix, but worth a manual audit of a sample workflow (e.g. archie.md, which appeared in both agent: searches) to determine ground truth.
• How to Implement: Manually inspect 3–5 workflows previously counted as "agent users" to confirm whether they set engine.agent or merely import agent markdown files.

Opportunity 4: Reconcile id: copilot and cache-memory count discrepancy

• What: Counts dropped 109→89 and 105→80 between this cycle and the last two, while total workflow count grew.
• Why It Matters: Without reconciling, trend tracking for adoption rates is unreliable across cycles.
• Where: Research process itself.
• How to Implement: Standardize the exact grep pattern used for id: copilot (nested under engine: block) and cache-memory: (top-level key) in the next cycle, and record the pattern in notes.md for repeatability.

Opportunity 5: --share flag remains fully unused for debugging

• What: No workflow opts into --share for producing shareable conversation transcripts.
• Why It Matters: For workflows that fail intermittently (e.g. daily-* scheduled jobs), a shareable transcript link could speed up triage without needing full log-dir access.
• Where: High-failure-rate or hard-to-debug workflows — check agent-job-health.md / aw-failure-investigator.md output for candidates.
• How to Implement: Add args: ["--share"] to engine: on 1–2 pilot workflows and confirm the resulting artifact/link surfaces usefully in run logs.

🟢 Low Priority

Opportunity 6: Standardize timeout-minutes selection guidance

• What: Copilot workflow timeouts span 5–120 minutes with no documented rationale tying value to workflow complexity.
• Why It Matters: Minor — inconsistency doesn't break anything but makes it hard for new workflow authors to pick a sensible default.
• Where: All Copilot workflows.
• How to Implement: Add a short guideline table to docs/src/content/docs/reference/engines.md (e.g., "simple triage: 10-15m, multi-step research: 30-60m, deep audits: 60m+").

Opportunity 7: plugins feature at 0% adoption, 3 consecutive cycles

• What: The documented Agent Plugins feature has never been used in any workflow across all measured cycles.
• Why It Matters: Low urgency since no workflow currently needs it, but worth confirming it isn't dead/unsupported functionality being carried in docs.
• Where: N/A.
• How to Implement: Verify with a maintainer whether plugins is still an active, supported feature before continuing to track its adoption in future cycles.

Workflow family: security-compliance.md, daily-malicious-code-scan.md, daily-secrets-analysis.md

• Current State: All run Copilot engine on latest with no version pin.
• Recommended Changes: Pin engine.version to a known-good release for these specifically, since they gate compliance/security signal quality.
• Expected Benefits: Reproducible audit results; CLI regressions caught deliberately via version bump rather than silently.

Workflow family: daily-* scheduled workflows (broad set)

• Current State: Each independently configures Copilot engine settings; no shared defaults snippet exists.
• Recommended Changes: Adopt shared/copilot-defaults.md once created (see Opportunity 1).
• Expected Benefits: Single point of control for engine-wide fixes/upgrades.

• Persistent zero-adoption features across 4 cycles: engine.version pinning, plugins, --share manual use. These have now been flagged in 3+ consecutive reports without any change — this cycle treats them as requiring an explicit decision rather than continued monitoring.
• copilot-sdk: true plateaued at exactly 61 for three straight measurements (runs 2, 3, 4) — genuinely flat, not measurement noise, since the exact same number recurred three times.
• shared/copilot-defaults.md still not created despite being recommended in run 2 (08-24) and escalated in run 3 (08-25). Two cycles of inaction — this is the single most actionable, low-effort item in the backlog.
• Count discrepancy this cycle (id: copilot 109→89, cache-memory 105→80) likely reflects a stricter/different grep pattern rather than real regression; flagged for reconciliation rather than alarm.
• max-tool-denials usage held steady at 55 between runs 3 and 4 — a genuinely stable, moderate-adoption feature, unlike the SDK/version-pin items that are stuck at extremes (flat-high or flat-zero).