Skip to content
GitHub Agentic Workflows

Meet the Workflows: Metrics & Analytics

Peli de Halleux

Let’s dive deeper into Peli’s Agent Factory!

In our previous post, we explored quality and hygiene workflows - the vigilant caretakers that investigate failed CI runs, detect schema drift, and catch breaking changes before users do. These workflows maintain codebase health by spotting problems before they escalate.

But here’s a question: when you’re running dozens of AI agents, how do you know if they’re actually working well? How do you spot performance issues, cost problems, or quality degradation? That’s where metrics and analytics workflows come in - they’re the agents that monitor other agents, turning raw activity data into actionable insights. This is where we got meta and built our central nervous system.

Data nerds, rejoice! These agents turn raw repository activity into actual insights:

  • Metrics Collector - Tracks daily performance across the entire agent ecosystem
  • Portfolio Analyst - Identifies cost reduction opportunities (because AI isn’t free!)
  • Audit Workflows - A meta-agent that audits all the other agents’ runs - very Inception

Here’s where things got meta: we built agents to monitor agents. The Metrics Collector became our central nervous system, gathering performance data that feeds into higher-level orchestrators. What we learned: you can’t optimize what you don’t measure. The Portfolio Analyst was eye-opening - it identified workflows that were costing us money unnecessarily (turns out some agents were way too chatty with their LLM calls).

These workflows taught us that observability isn’t optional when you’re running dozens of AI agents - it’s the difference between a well-oiled machine and an expensive black box.

Now that we can measure and optimize our agent ecosystem, let’s talk about the moment of truth: actually shipping software to users.

Continue reading: Operations & Release Workflows →


This is part 6 of a 16-part series exploring the workflows in Peli’s Agent Factory.