Meet the Workflows: Advanced Analytics & ML
Time to get into data ana analytics at Peli’s Agent Factory!
In our previous post, we explored organization and cross-repo workflows that operate at enterprise scale - analyzing dozens of repositories together to find patterns and outliers that single-repo analysis would miss. We learned that perspective matters: what looks normal in isolation might signal drift at scale.
Beyond tracking basic metrics (run time, cost, success rate), we wanted deeper insights into how our agents actually behave and how developers interact with them. What patterns emerge from thousands of agent prompts? What makes some PR conversations more effective than others? How do usage patterns reveal improvement opportunities? This is where we brought out the big guns: machine learning, natural language processing, sentiment analysis, and clustering algorithms. Advanced analytics workflows don’t just count things - they understand them, finding patterns and insights that direct observation would never reveal.
Advanced Analytics & ML Workflows
Section titled “Advanced Analytics & ML Workflows”These agents use sophisticated analysis techniques to extract insights:
- Copilot Session Insights - Analyzes Copilot agent usage patterns and metrics
- Copilot PR NLP Analysis - Natural language processing on PR conversations
- Prompt Clustering Analysis - Clusters and categorizes agent prompts using ML
- Copilot Agent Analysis - Deep analysis of agent behavior patterns
The Prompt Clustering Analysis uses machine learning to categorize thousands of agent prompts, revealing patterns we never noticed (“oh, 40% of our prompts are about error handling”).
The Copilot PR NLP Analysis does sentiment analysis and linguistic analysis on PR conversations - it found that PRs with questions in the title get faster review.
The Session Insights workflow analyzes how developers interact with Copilot agents, identifying common patterns and failure modes. What we learned: meta-analysis is powerful - using AI to analyze AI systems reveals insights that direct observation misses.
These workflows helped us understand not just what our agents do, but how they behave and how users interact with them.
Learn More
Section titled “Learn More”- GitHub Agentic Workflows - The technology behind the workflows
- Quick Start - How to write and compile workflows
Next Up: Campaigns & Project Coordination Workflows
Section titled “Next Up: Campaigns & Project Coordination Workflows”We’ve reached the final stop: coordinating multiple agents toward shared, complex goals across extended timelines.
Continue reading: Campaigns & Project Coordination Workflows →
This is part 15 of a 16-part series exploring the workflows in Peli’s Agent Factory.