[curriculum-eval] curriculum-scorecard: overall page scores and trend analysis

Curriculum Quality Evaluator · issue · closed

Filter2mode:review mode:live
All recorded Export JSON
github-actions[bot]

published Aug 25, 2026, 1:08 PM · updated Aug 26, 2026, 2:55 PM

Curriculum Scorecard

Part Summary

Part Files Mean Score Std Dev
Part 1 — core path (lessons 00–14) 12 5.89 / 10.0 ±0.36
Part 2 — advanced (lessons 15+) 15 6.11 / 10.0 ±0.35
Other (side quests / no lesson number) 62 6.55 / 10.0 ±0.94
Overall corpus 89 6.39 / 10.0 ±0.84

Part 1 Scores (lessons 00–14)

File Overall Score
workshop/04-github-actions-intro.md 5.42 / 10.0
workshop/05-agentic-workflows-intro.md 5.43 / 10.0
workshop/14b-pr-reviewer-workflow.md 5.47 / 10.0
workshop/08-run-your-workflow.md 5.67 / 10.0
workshop/05b-agentic-workflows-security.md 5.75 / 10.0
workshop/14-next-steps.md 5.85 / 10.0
workshop/08b-interpret-your-run.md 5.99 / 10.0
workshop/09-agentic-editing.md 6.03 / 10.0
workshop/02a-setup-codespace.md 6.09 / 10.0
workshop/05c-agentic-workflows-practice.md 6.22 / 10.0
workshop/07-your-first-workflow.md 6.25 / 10.0
workshop/07d-confirm-model-access.md 6.55 / 10.0

Part 2 Scores (lessons 15+)

File Overall Score
workshop/15-conditional-logic.md 5.53 / 10.0
workshop/17-add-mcp-tools.md 5.65 / 10.0
workshop/16-connect-data-source.md 5.71 / 10.0
workshop/20-persistent-memory.md 5.81 / 10.0
workshop/26-manage-costs-and-budgets.md 5.85 / 10.0
workshop/21-inline-sub-agents.md 5.99 / 10.0
workshop/29-skills-and-domain-knowledge.md 6.05 / 10.0
workshop/18-share-and-reuse.md 6.15 / 10.0
workshop/25-audit-and-observability.md 6.23 / 10.0
workshop/28-orchestrate-workflows.md 6.25 / 10.0
workshop/22-error-handling-and-resilience.md 6.27 / 10.0
workshop/19-research-driven-training-node.md 6.37 / 10.0
workshop/23-ab-experiments.md 6.53 / 10.0
workshop/27-evaluate-workflow-quality.md 6.59 / 10.0
workshop/24-self-hosted-runners.md 6.63 / 10.0

Other Scores (no lesson number)

File Overall Score
workshop/side-quest-17-07-repo-poisoning.md 4.99 / 10.0
workshop/side-quest-01-02-environment-reference.md 5.31 / 10.0
workshop/side-quest-11-06-anthropic-key.md 5.39 / 10.0
workshop/side-quest-05-03-two-file-structure.md 5.51 / 10.0
workshop/side-quest-20-01-memory-patterns.md 5.57 / 10.0
workshop/side-quest-11-01b-workflow-structure.md 5.62 / 10.0
workshop/side-quest-11-08-frontmatter-tools-outputs.md 5.64 / 10.0
workshop/side-quest-11-04-annotated-workflow.md 5.72 / 10.0
workshop/side-quest-06-03a-copilot-requests-permission.md 5.75 / 10.0
workshop/side-quest-enterprise-setup.md 5.75 / 10.0
workshop/side-quest-17-02-security-architecture.md 5.77 / 10.0
workshop/side-quest-17-03-prompt-injection.md 5.81 / 10.0
workshop/side-quest-05-01-actions-power-user.md 5.83 / 10.0
workshop/side-quest-06-03-copilot-token.md 5.83 / 10.0
workshop/side-quest-11-05-event-triggers.md 5.87 / 10.0
workshop/side-quest-10-02-jailbreak-brief.md 5.89 / 10.0
workshop/side-quest-11-07-openai-key.md 5.91 / 10.0
workshop/side-quest-17-01-mcp-concepts.md 5.91 / 10.0
workshop/side-quest-25-01-audit-reference.md 5.91 / 10.0
workshop/side-quest-17-04-permission-escalation.md 5.93 / 10.0
workshop/side-quest-09-01f-debugging-checklist.md 5.96 / 10.0
workshop/side-quest-17-06-output-injection.md 5.97 / 10.0
workshop/side-quest-17-05-supply-chain-mcp.md 6.01 / 10.0
workshop/side-quest-08-01-codespaces-actions-write.md 6.03 / 10.0
workshop/side-quest-10-01-agent-brief.md 6.09 / 10.0
workshop/side-quest-05-02-aw-deep-dive.md 6.13 / 10.0
workshop/side-quest-24-01-runner-infrastructure.md 6.13 / 10.0
workshop/side-quest-09-01-debug-output.md 6.15 / 10.0
workshop/side-quest-07d-billing-paths.md 6.19 / 10.0
workshop/side-quest-09-01a-pattern-long-plan-chain.md 6.20 / 10.0
workshop/side-quest-11-03-better-prompts.md 6.24 / 10.0
workshop/side-quest-26-01-forecast-costs.md 6.24 / 10.0
workshop/side-quest-02-01-local-terminal.md 6.25 / 10.0
workshop/side-quest-01-01-terminal-basics.md 6.26 / 10.0
workshop/side-quest-09-01b-pattern-empty-results.md 6.35 / 10.0
workshop/side-quest-09-01c-pattern-safe-output-blocked.md 6.35 / 10.0
workshop/side-quest-09-01d-pattern-permission-denied.md 6.37 / 10.0
workshop/side-quest-09-01e-pattern-done-no-write.md 6.39 / 10.0
workshop/side-quest-06-01-install-troubleshooting.md 6.41 / 10.0
workshop/side-quest-06-03c-copilot-github-token-ui-only.md 6.49 / 10.0
workshop/side-quest-07-01-compile-workflow.md 6.49 / 10.0
workshop/side-quest-06-03b-copilot-github-token.md 6.77 / 10.0
workshop/side-quest-06-04-install-local.md 6.77 / 10.0
workshop/side-quest-01-03-permission-errors.md 6.79 / 10.0
workshop/side-quest-21-01-sub-agent-syntax.md 6.93 / 10.0
workshop/side-quest-11-02-yaml-frontmatter.md 7.00 / 10.0
workshop/side-quest-11-01-frontmatter-deep-dive.md 7.03 / 10.0
workshop/side-quest-06-02-cca-codespace.md 7.25 / 10.0
workshop/side-quest-13-04-token-optimization.md 7.67 / 10.0
workshop/side-quest-11-09-agent-session-phases.md 7.69 / 10.0
workshop/side-quest-16-02-secrets-and-permissions.md 7.79 / 10.0
workshop/side-quest-16-03-token-exfiltration.md 7.85 / 10.0
workshop/side-quest-13-01-schedule-expressions.md 7.89 / 10.0
workshop/side-quest-12-01-iterate-agent-output.md 7.91 / 10.0
workshop/side-quest-16-04-deterministic-vs-agentic-data-ops.md 8.03 / 10.0
workshop/side-quest-15-01-expressions-and-contexts.md 8.05 / 10.0
workshop/side-quest-16-05-long-lived-credentials.md 8.05 / 10.0
workshop/side-quest-15-02-chaining-conditions.md 8.23 / 10.0
workshop/side-quest-13-02-pr-summary-pattern.md 8.39 / 10.0
workshop/side-quest-13-03-pr-checklist-pattern.md 8.47 / 10.0
workshop/side-quest-16-01-github-output.md 8.50 / 10.0
workshop/side-quest-13-01-pr-labeler-pattern.md 8.63 / 10.0

Score Trends (history window: 20 commits)

File Baseline Latest Delta Direction
workshop/02a-setup-codespace.md 8.09 6.09 -2.00 declining
workshop/04-github-actions-intro.md 7.42 5.42 -2.00 declining
workshop/05-agentic-workflows-intro.md 7.45 5.43 -2.02 declining
workshop/05b-agentic-workflows-security.md 7.75 5.75 -2.00 declining
workshop/05c-agentic-workflows-practice.md 8.59 6.22 -2.37 declining
workshop/07-your-first-workflow.md 8.61 6.25 -2.36 declining
workshop/07d-confirm-model-access.md 7.19 6.55 -0.64 declining
workshop/08-run-your-workflow.md 7.67 5.67 -2.00 declining
workshop/08b-interpret-your-run.md 7.99 5.99 -2.00 declining
workshop/09-agentic-editing.md 8.03 6.03 -2.00 declining
workshop/14-next-steps.md 7.91 5.85 -2.06 declining
workshop/14b-pr-reviewer-workflow.md 7.67 5.47 -2.20 declining
workshop/15-conditional-logic.md 7.53 5.53 -2.00 declining
workshop/16-connect-data-source.md 7.71 5.71 -2.00 declining
workshop/17-add-mcp-tools.md 7.75 5.65 -2.10 declining
workshop/18-share-and-reuse.md 8.15 6.15 -2.00 declining
workshop/19-research-driven-training-node.md 8.39 6.37 -2.02 declining
workshop/20-persistent-memory.md 7.81 5.81 -2.00 declining
workshop/21-inline-sub-agents.md 7.99 5.99 -2.00 declining
workshop/22-error-handling-and-resilience.md 8.29 6.27 -2.02 declining
workshop/23-ab-experiments.md 8.53 6.53 -2.00 declining
workshop/24-self-hosted-runners.md 8.63 6.63 -2.00 declining
workshop/25-audit-and-observability.md 8.25 6.23 -2.02 declining
workshop/26-manage-costs-and-budgets.md 7.85 5.85 -2.00 declining
workshop/27-evaluate-workflow-quality.md 8.37 6.59 -1.78 declining
workshop/28-orchestrate-workflows.md 8.25 6.25 -2.00 declining
workshop/29-skills-and-domain-knowledge.md 8.05 6.05 -2.00 declining
workshop/side-quest-01-01-terminal-basics.md 8.25 6.26 -1.99 declining
workshop/side-quest-01-02-environment-reference.md 7.31 5.31 -2.00 declining
workshop/side-quest-01-03-permission-errors.md 8.81 6.79 -2.02 declining
workshop/side-quest-02-01-local-terminal.md 8.25 6.25 -2.00 declining
workshop/side-quest-05-01-actions-power-user.md 7.85 5.83 -2.02 declining
workshop/side-quest-05-02-aw-deep-dive.md 8.13 6.13 -2.00 declining
workshop/side-quest-05-03-two-file-structure.md 7.51 5.51 -2.00 declining
workshop/side-quest-06-01-install-troubleshooting.md 8.43 6.41 -2.02 declining
workshop/side-quest-06-02-cca-codespace.md 9.25 7.25 -2.00 declining
workshop/side-quest-06-03-copilot-token.md 7.83 5.83 -2.00 declining
workshop/side-quest-06-03a-copilot-requests-permission.md 7.77 5.75 -2.02 declining
workshop/side-quest-06-03b-copilot-github-token.md 8.79 6.77 -2.02 declining
workshop/side-quest-06-03c-copilot-github-token-ui-only.md 8.53 6.49 -2.04 declining
workshop/side-quest-06-04-install-local.md 8.77 6.77 -2.00 declining
workshop/side-quest-07-01-compile-workflow.md 8.49 6.49 -2.00 declining
workshop/side-quest-07d-billing-paths.md N/A 6.19 N/A new
workshop/side-quest-08-01-codespaces-actions-write.md 8.03 6.03 -2.00 declining
workshop/side-quest-09-01-debug-output.md 8.15 6.15 -2.00 declining
workshop/side-quest-09-01a-pattern-long-plan-chain.md 8.21 6.20 -2.01 declining
workshop/side-quest-09-01b-pattern-empty-results.md 8.37 6.35 -2.02 declining
workshop/side-quest-09-01c-pattern-safe-output-blocked.md 8.35 6.35 -2.00 declining
workshop/side-quest-09-01d-pattern-permission-denied.md 8.39 6.37 -2.02 declining
workshop/side-quest-09-01e-pattern-done-no-write.md 8.39 6.39 -2.00 declining
workshop/side-quest-09-01f-debugging-checklist.md 8.01 5.96 -2.05 declining
workshop/side-quest-10-01-agent-brief.md 8.09 6.09 -2.00 declining
workshop/side-quest-10-02-jailbreak-brief.md 7.89 5.89 -2.00 declining
workshop/side-quest-11-01-frontmatter-deep-dive.md 9.05 7.03 -2.02 declining
workshop/side-quest-11-01b-workflow-structure.md 7.65 5.62 -2.03 declining
workshop/side-quest-11-02-yaml-frontmatter.md 8.75 7.00 -1.75 declining
workshop/side-quest-11-03-better-prompts.md 8.63 6.24 -2.39 declining
workshop/side-quest-11-04-annotated-workflow.md 7.72 5.72 -2.00 declining
workshop/side-quest-11-05-event-triggers.md 7.91 5.87 -2.04 declining
workshop/side-quest-11-06-anthropic-key.md 7.39 5.39 -2.00 declining
workshop/side-quest-11-07-openai-key.md 7.93 5.91 -2.02 declining
workshop/side-quest-11-08-frontmatter-tools-outputs.md 7.64 5.64 -2.00 declining
workshop/side-quest-11-09-agent-session-phases.md 7.71 7.69 -0.02 stable
workshop/side-quest-12-01-iterate-agent-output.md 7.93 7.91 -0.02 stable
workshop/side-quest-13-01-pr-labeler-pattern.md 8.63 8.63 0.00 stable
workshop/side-quest-13-01-schedule-expressions.md 7.91 7.89 -0.02 stable
workshop/side-quest-13-02-pr-summary-pattern.md 8.39 8.39 0.00 stable
workshop/side-quest-13-03-pr-checklist-pattern.md 8.49 8.47 -0.02 stable
workshop/side-quest-13-04-token-optimization.md 7.71 7.67 -0.04 stable
workshop/side-quest-15-01-expressions-and-contexts.md 8.05 8.05 0.00 stable
workshop/side-quest-15-02-chaining-conditions.md 8.23 8.23 0.00 stable
workshop/side-quest-16-01-github-output.md 8.50 8.50 0.00 stable
workshop/side-quest-16-02-secrets-and-permissions.md 7.79 7.79 0.00 stable
workshop/side-quest-16-03-token-exfiltration.md 7.85 7.85 0.00 stable
workshop/side-quest-16-04-deterministic-vs-agentic-data-ops.md 8.03 8.03 0.00 stable
workshop/side-quest-16-05-long-lived-credentials.md 8.05 8.05 0.00 stable
workshop/side-quest-17-01-mcp-concepts.md 5.91 5.91 0.00 stable
workshop/side-quest-17-02-security-architecture.md 5.77 5.77 0.00 stable
workshop/side-quest-17-03-prompt-injection.md 5.81 5.81 0.00 stable
workshop/side-quest-17-04-permission-escalation.md 5.93 5.93 0.00 stable
workshop/side-quest-17-05-supply-chain-mcp.md 6.01 6.01 0.00 stable
workshop/side-quest-17-06-output-injection.md 5.97 5.97 0.00 stable
workshop/side-quest-17-07-repo-poisoning.md 5.45 4.99 -0.46 declining
workshop/side-quest-20-01-memory-patterns.md 5.57 5.57 0.00 stable
workshop/side-quest-21-01-sub-agent-syntax.md 6.93 6.93 0.00 stable
workshop/side-quest-24-01-runner-infrastructure.md 6.13 6.13 0.00 stable
workshop/side-quest-25-01-audit-reference.md 5.91 5.91 0.00 stable
workshop/side-quest-26-01-forecast-costs.md 6.24 6.24 0.00 stable
workshop/side-quest-enterprise-setup.md 5.75 5.75 0.00 stable

Overall Trend

  • Baseline mean score: 7.79
  • Latest mean score: 6.39
  • Delta: -1.40
  • Direction: declining

Bloom's Taxonomy Distribution

Level Count %
Remember 6 7%
Understand 17 19%
Apply 46 52%
Analyze 7 8%
Evaluate 5 6%
Create 8 9%

The corpus meets the minimum thresholds for both Understand (≥ 5) and Analyze (≥ 2) steps. However, the Apply level dominates at 52% of all pages, which is unusually concentrated for a progressive workshop. The Bloom pyramid gap analysis found no missing levels, but the heavy Apply clustering suggests that intermediate Understand and Analyze steps connecting conceptual introduction to hands-on practice may be underweighted.


Corpus-Level Notes

Systemic score decline: Nearly every core workshop page has declined by approximately −2.0 points across the 20-commit history window. This uniform −2.0 shift is consistent with a rubric weight change or a new scoring penalty (such as the checkpoint_quality dimension returning 0.0 for all pages without an explicit ## :white_check_mark: Checkpoint section) having been introduced mid-history. The pages that remain stable or high-scoring (side-quest-13-xx, side-quest-15-xx, side-quest-16-xx) already have high checkpoint quality scores, which supports this diagnosis.

Checkpoint coverage gap: 100% of Part 1 core pages score 0.0 on checkpoint_quality, indicating no page in the core path currently has a rubric-compliant checkpoint section with ≥ 4 specific checklist items. This is the single largest driver of the corpus-wide score decline.

Warning

Firewall blocked 1 domain

The following domain was blocked by the firewall during workflow execution:

  • awmgmcpg

To allow these domains, add them to the network.allowed list in your workflow frontmatter:

network:
  allowed:
    - defaults
    - "awmgmcpg"

See Network Configuration for more information.

Generated by 🔬 Curriculum Quality Evaluator · 56.4 AIC · ⌖ 8.41 AIC · ⊞ 6.2K ·

  • expires on Aug 26, 2026, 1:08 PM UTC