Skip to content
GitHub Agentic Workflows

Blog

Meet the Workflows: Metrics & Analytics

Don Syme
Peli de Halleux
Mara Kiefer
Peli de Halleux

Excellent journey! Now it’s time to plunge into the observatory - the nerve center of Peli’s Agent Factory! Where we watch everything and know all!

In our previous post, we explored quality and hygiene workflows - the vigilant caretakers that investigate failed CI runs, detect schema drift, and catch breaking changes before users do. These workflows maintain codebase health by spotting problems before they escalate.

But here’s a question: when you’re running dozens of AI agents, how do you know if they’re actually working well? How do you spot performance issues, cost problems, or quality degradation? That’s where metrics and analytics workflows come in - they’re the agents that monitor other agents, turning raw activity data into actionable insights. This is where we got meta and built our central nervous system.

Metrics & Analytics Workflows

Section titled “Metrics & Analytics Workflows”

Data scientists, rejoice! These agents turn raw repository activity into actual insights:

  • Metrics Collector - Tracks daily performance across the entire agent ecosystem
  • Portfolio Analyst - Identifies cost reduction opportunities (because AI isn’t free!)
  • Audit Workflows - A meta-agent that audits all the other agents’ runs - very Inception

Here’s where things got meta: we built agents to monitor agents. The Metrics Collector became our central nervous system, gathering performance data that feeds into higher-level orchestrators. What we learned: you can’t optimize what you don’t measure. The Portfolio Analyst was eye-opening - it identified workflows that were costing us money unnecessarily (turns out some agents were way too chatty with their LLM calls).

These workflows taught us that observability isn’t optional when you’re running dozens of AI agents - it’s the difference between a well-oiled machine and an expensive black box.

Learn More

Section titled “Learn More”

Next Up: Operations & Release Workflows

Section titled “Next Up: Operations & Release Workflows”

Now that we can measure and optimize our agent ecosystem, let’s talk about the moment of truth: actually shipping software to users.

Continue reading: Operations & Release Workflows →

This is part 6 of a 16-part series exploring the workflows in Peli’s Agent Factory.

Meet the Workflows: Quality & Hygiene

Don Syme
Peli de Halleux
Mara Kiefer
Peli de Halleux

Ah, splendid! Welcome back to Peli’s Agent Factory! Come, let me show you the chamber where everything is polished, everything improved, and all sparkles!

In our previous post, we explored issue and PR management workflows - agents that enhance GitHub’s collaboration features by removing tedious ceremony like linking related issues, merging main branches, and optimizing templates. These workflows make GitHub more pleasant to use by eliminating small papercuts that add up to significant friction.

Now let’s shift from collaboration ceremony to codebase maintenance. While issue workflows help us handle what comes in, quality and hygiene workflows act as vigilant caretakers - spotting problems before they escalate and keeping our codebase healthy. These are the agents that investigate failed CI runs, detect schema drift, and catch breaking changes before users do.

Quality & Hygiene Workflows

Section titled “Quality & Hygiene Workflows”

These are our diligent caretakers - the agents that spot problems before they become, well, bigger problems:

The CI Doctor was a revelation. Instead of drowning in CI failure notifications, we now get timely, investigated failures with actual diagnostic insights. The agent doesn’t just tell us something broke - it analyzes logs, identifies patterns, searches for similar past issues, and even suggests fixes. We learned that agents excel at the tedious investigation work that humans find draining.

The Schema Consistency Checker caught drift that would have taken us days to notice manually.

These “hygiene” workflows became our first line of defense, catching issues before they reached users.

Learn More

Section titled “Learn More”

Next Up: Metrics & Analytics Workflows

Section titled “Next Up: Metrics & Analytics Workflows”

With quality and hygiene workflows maintaining our codebase health, we needed a way to understand whether they were actually working. How do you know if your agents are performing well? That’s where metrics and analytics come in.

Continue reading: Metrics & Analytics Workflows →

This is part 5 of a 16-part series exploring the workflows in Peli’s Agent Factory.

Meet the Workflows: Issue & PR Management

Don Syme
Peli de Halleux
Mara Kiefer
Peli de Halleux

Ah! Let’s discuss the art of collaboration at Peli’s Agent Factory! A most delicious topic indeed!

In our previous post, we explored documentation and content workflows - agents that maintain glossaries, technical docs, slide decks, and blog content. We learned that AI-generated docs need human review, but they’re dramatically better than having no docs at all.

Now let’s talk about the daily rituals of software development: managing issues and pull requests. GitHub provides excellent primitives for collaboration, but there’s a lot of ceremony involved - linking related issues, merging main into PR branches, assigning work, closing completed sub-issues, optimizing templates. These are small papercuts individually, but they add up to significant friction. Issue and PR management workflows don’t replace GitHub’s features; they enhance them, removing tedious ceremony and making collaboration feel effortless. Let’s see how automation makes GitHub more pleasant to use.

Issue & PR Management Workflows

Section titled “Issue & PR Management Workflows”

These agents enhance issue and pull request workflows:

Issue management is tedious ceremony that developers tolerate rather than enjoy. The Issue Arborist automatically links related issues, building a dependency tree we’d never maintain manually. The Issue Monster became our task dispatcher for AI agents - it assigns one issue at a time to Copilot agents, preventing the chaos of parallel work on the same codebase. Mergefest eliminates the “please merge main” dance that happens on long-lived PRs. We learned that tiny frustrations add up - each of these workflows removes a small papercut, and collectively they make GitHub feel much more pleasant to use. The Issue Template Optimizer analyzes which fields in our templates actually get filled out and suggests improvements (“nobody uses the ‘Expected behavior’ field, remove it”).

Learn More

Section titled “Learn More”

Next Up: Quality & Hygiene Workflows

Section titled “Next Up: Quality & Hygiene Workflows”

While issue workflows help manage collaboration ceremony, we also need agents that maintain codebase health - spotting problems before they escalate.

Continue reading: Quality & Hygiene Workflows →

This is part 4 of a 16-part series exploring the workflows in Peli’s Agent Factory.

Meet the Workflows: Documentation & Content

Don Syme
Peli de Halleux
Mara Kiefer
Peli de Halleux

Step right up, step right up, and enter the documentation chamber of Peli’s Agent Factory! Pure imagination meets technical accuracy in this most delightful corner of our establishment!

In our previous post, we explored code quality and refactoring workflows - agents that continuously push our codebase toward better design, finding patterns and improvements that humans often miss. These workflows never take a day off, quietly working to make our code cleaner and more maintainable.

Now let’s address one of software development’s eternal challenges: keeping documentation accurate and up-to-date. Code evolves rapidly; docs… not so much. Terminology drifts, API examples become outdated, slide decks grow stale, and blog posts reference deprecated features. The question isn’t “can AI agents write good documentation?” but rather “can they maintain it as code changes?” Documentation and content workflows challenge conventional wisdom about AI-generated technical content. Spoiler: the answer involves human review, but it’s way better than the alternative (no docs at all).

Documentation & Content Workflows

Section titled “Documentation & Content Workflows”

These agents maintain high-quality documentation and content:

Documentation is where we challenged conventional wisdom. Can AI agents write good documentation?

The Technical Doc Writer generates API docs from code, but more importantly, it maintains them - updating docs when code changes. The Glossary Maintainer caught terminology drift (“we’re using three different terms for the same concept”).

The Slide Deck Maintainer keeps our presentation materials current without manual updates.

The Multi-device Docs Tester uses Playwright to verify our documentation site works across phones, tablets, and desktops - testing responsive layouts, accessibility, and interactive elements. It catches visual regressions and layout issues that only appear on specific screen sizes.

The Blog Auditor ensures our blog posts stay accurate as the codebase evolves - it flags outdated code examples and broken links.

AI-generated docs need human review, but they’re dramatically better than no docs (which is often the alternative). Validation can be automated to a large extent, freeing writers to focus on content shaping, topic, clarity, tone, and accuracy.

Learn More

Section titled “Learn More”

Next Up: Issue & PR Management Workflows

Section titled “Next Up: Issue & PR Management Workflows”

Beyond writing code and docs, we need to manage the flow of issues and pull requests. How do we keep collaboration smooth and efficient?

Continue reading: Issue & PR Management Workflows →

This is part 3 of a 16-part series exploring the workflows in Peli’s Agent Factory.

Meet the Workflows: Code Quality & Refactoring

Don Syme
Peli de Halleux
Mara Kiefer
Peli de Halleux

Ah, what marvelous timing! Come, come, let me show you the next wonder in Peli’s Agent Factory!

In our previous post, we explored how triage and summarization workflows help us stay on top of incoming activity - automatically labeling issues, creating digestible summaries, and narrating the day’s events. These workflows taught us that tone matters and even simple automation dramatically reduces cognitive load.

Now let’s turn to the agents that continuously improve code quality. Code quality and refactoring workflows work quietly in the background, never taking a day off - they analyze console output styling, spot semantic duplication, identify structural improvements, and find patterns humans miss because they can hold entire codebases in context. These workflows embody the principle that good enough can always become better, and that incremental improvements compound over time. Let’s meet the perfectionist agents.

Code Quality & Refactoring Workflows

Section titled “Code Quality & Refactoring Workflows”

These agents make our codebase cleaner and our developer experience better:

Code quality workflows represent a new paradigm in software engineering: autonomous cleanup agents that trail behind human developers, constantly sweeping, polishing, and improving. While developers race ahead implementing features and fixing bugs, these agents work tirelessly in the background - simplifying overcomplicated code, detecting semantic duplication that humans miss, and ensuring consistent patterns across the entire codebase. They’re the Marie Kondos of code repositories, asking “does this function spark joy?” and “could this be simpler?”

What makes these workflows particularly powerful is their tirelessness. The Terminal Stylist literally reads every line of console output code, suggesting improvements to make our CLI prettier (yes, it understands Lipgloss and modern terminal styling conventions). The Semantic Function Refactor finds duplicated logic that’s not quite identical enough for traditional duplicate detection - the kind of semantic similarity that humans recognize but struggle to systematically address. The Duplicate Code Detector goes further, using Serena’s semantic analysis to understand code meaning rather than just textual similarity, catching patterns that copy-paste detection misses entirely.

The Go-specific workflows demonstrate how deep these agents can go. Go Pattern Detector ensures consistency in idioms and best practices, Typist analyzes type usage patterns to improve type safety, and Go Fan reviews module dependencies to catch bloat and suggest better alternatives. Together, they embody institutional knowledge that would take years for a developer to accumulate, applied consistently across every file, every day.

Perhaps most intriguingly, these agents excel at cleaning up AI-generated code. As developers write more code with AI, this is over-more important. These workflows trail behind the development team, refactoring their output to match project standards, simplifying overly verbose AI suggestions, and ensuring the AI-human collaboration produces not just working code, but beautiful code. The Code Simplifier analyzes recently modified code (whether written by humans or AI) and creates pull requests with improvements, while the Repository Quality Improver takes a holistic view - identifying structural improvements and documentation gaps that emerge from rapid development.

This is the future of AI-enriched software engineering: developers at the frontier pushing forward, AI assistants helping them write code faster, and autonomous cleanup agents ensuring that speed doesn’t sacrifice quality. The repository stays clean, patterns stay consistent, and technical debt gets addressed proactively rather than accumulating into crisis. These workflows never get bored, never skip the boring parts.

Learn More

Section titled “Learn More”

Next Up: Documentation & Content Workflows

Section titled “Next Up: Documentation & Content Workflows”

Beyond code quality, we need to keep documentation accurate and up-to-date as code evolves. How do we maintain docs that stay current?

Continue reading: Documentation & Content Workflows →

This is part 2 of a 16-part series exploring the workflows in Peli’s Agent Factory.

Made with by GitHub Next & Microsoft Research Terms of Service