Skip to content

vt-c-learning-metrics

Track and visualize learning system effectiveness with ROI metrics

Plugin: core-standards
Category: Knowledge Capture
Command: /vt-c-learning-metrics

learning-metrics Skill

Purpose: Quantify the ROI of the compound engineering learning system.

What It Does

Reads metrics JSON files and generates: 1. Markdown dashboard (quick report) 2. ROI calculations (time saved, bugs prevented) 3. Compounding impact analysis

Data Sources

  • metrics/solution-reuse.json - Solution reuse tracking
  • metrics/pattern-usage.json - Pattern prevention tracking
  • metrics/verification-loops.json - Verification success tracking
  • metrics/pattern-promotions.json - Auto-promotion tracking

Execution

Step 1: Load All Metrics

Read all 4 JSON files:

Read: metrics/solution-reuse.json
Read: metrics/pattern-usage.json
Read: metrics/verification-loops.json
Read: metrics/pattern-promotions.json

Extract counts: - Total solutions reused: Count of reuses array - Total time saved: Sum of time_saved_estimate_minutes - Bugs prevented: Count where bug_prevented = true - Verification loops: Average loop_count - Pattern promotions: Count of auto_promoted = true

Step 2: Calculate ROI

Time Saved: - Solution reuse: Sum from solution-reuse.json - Pattern prevention: Sum debug_time_saved_minutes from pattern-usage.json - Total: Sum of both

Bugs Prevented: - Count from pattern-usage.json where bug_prevented = true

Pattern Effectiveness: - Average occurrences before promotion - Percentage auto-promoted vs manual

Verification Effectiveness: - Average loop count (lower is better - indicates fewer false completions) - Total false claims prevented

Step 3: Generate Dashboard

Create markdown output:

# Learning System Metrics

**Generated**: {timestamp}

## 📊 Summary

| Metric | Value |
|--------|-------|
| Solutions Documented | {count from docs/solutions if available, else "N/A"} |
| Solutions Reused | {count from metrics} |
| **Total Time Saved** | **{sum minutes} min ({hours} hrs)** |
| Bugs Prevented | {count} |
| Avg Verification Loops | {avg} |

## 🔄 Top Reused Solutions

| Solution | Reuses | Time Saved |
|----------|--------|------------|
{top 5 solutions by reuse count}

## 🛡️ Pattern Prevention

| Pattern | Uses | Bugs Prevented |
|---------|------|----------------|
{patterns that prevented bugs}

## ✅ Verification Effectiveness

- **Total Verification Runs**: {count}
- **Average Loop Count**: {avg}
- **False Claims Prevented**: {sum}
- **Success Rate**: {percentage of loops that eventually passed}

## 🎯 Pattern Promotions

- **Auto-Promoted**: {count}
- **Manual Promotions**: {count}
- **Avg Occurrences Before Promotion**: {avg}

## 📈 Compounding Impact

### Knowledge Base Growth
Jan: ████████░░ {count} solutions Feb: ██████████ {count} solutions (+{percentage}%) Mar: ████████████ {count} solutions (+{percentage}%) 
### Time Savings Trend
Week 1: {minutes} min Week 2: {minutes} min (+{percentage}%) Week 3: {minutes} min (+{percentage}%) Week 4: {minutes} min (+{percentage}%) 
**This is investment #{total_solutions} in your knowledge base.**

---

## 💡 Key Insights

- **Most valuable pattern**: {pattern_name} (saved {minutes} min across {count} uses)
- **Fastest growing category**: {category} ({count} new solutions this month)
- **Compounding rate**: Each new solution saves an average of {avg_minutes} min when reused
- **ROI**: For every 5 minutes spent documenting, save ~{avg_minutes} minutes on future occurrences

---

## 🚀 What This Means

Your team is **systematically getting smarter**. Each documented solution:
1. Prevents future bugs during planning
2. Saves 15-45 minutes per reuse
3. Compounds knowledge for the entire team

**The system is working.** Keep documenting problems when you solve them.

---

*Generated by learning-metrics skill*

Step 4: Display Dashboard

Output dashboard to terminal.

If files don't exist or are empty, display helpful message: 

📊 Learning System Metrics

No metrics data available yet. The learning system will start tracking:
- Solution reuse when /vt-c-wf-plan finds relevant learnings
- Pattern prevention when critical patterns are detected
- Verification loops when using --autonomous mode
- Pattern promotions when 3+ similar issues occur

Run some workflows to generate metrics!

Integration Points

Invoked by: - /vt-c-learning-metrics command - workflows:plan (optional inline feedback in future) - Manual invocation for reports

Success Criteria

  • ✅ Reads all 4 metrics JSON files
  • ✅ Handles empty/missing files gracefully
  • ✅ Calculates accurate ROI metrics
  • ✅ Generates markdown dashboard
  • ✅ Performance < 100ms

Error Handling

Missing files: - Display message: "Metrics file not found. Showing available data only." - Continue with available metrics

Invalid JSON: - Catch parse error - Display: "Metrics file corrupted. Please check {filename}" - Skip that metric

Empty metrics: - Display helpful "getting started" message - Show what actions will populate metrics