Skip to content

toolkit-review

Plugin: core-standards
Category: Other
Command: /toolkit-review

Toolkit Review

Purpose: Close the feedback loop between project experience and toolkit procedures. Collects improvement proposals from all sources and applies approved changes to toolkit skills, agents, and templates.

The Feedback Loop

SOURCES                                    TOOLKIT (V025-claude-toolkit)
───────                                    ──────────────────────────────

Project work                               intake/
  └─ /vt-c-compound (Option 8) ──────────►       ├─ projects.yaml
  └─ /vt-c-journal (auto-detect) ────────►       ├─ pending/
                                             │    ├─ from-projects/
RCA documents                                │    ├─ from-rca/
  └─ /vt-c-toolkit-review --from-rca ────►       │    └─ from-research/
                                             └─ processed/
Research
  └─ /vt-c-research-ingest ─────────────►  intake/pending/from-research/

                         │
                         ▼
              /vt-c-toolkit-review processes all
                         │
                         ▼
              Skills, agents, templates updated

Invocation

/vt-c-toolkit-review                          # Collect from all registered projects + intake/pending/
/vt-c-toolkit-review --scan ~/path/to/project # Scan specific project (also registers it)
/vt-c-toolkit-review --from-rca PATH          # Import learnings from an RCA document
/vt-c-toolkit-review --status                 # Show pending proposals across all sources

Configuration

toolkit_root: TOOLKIT_ROOT
intake_dir: intake/
projects_registry: intake/projects.yaml

Execution Instructions

Step 1: Collect Proposals

Tip: Run /vt-c-project-sync first to ensure all registered projects' proposals and journal entries have been collected into intake/pending/from-projects/. This is especially important if projects have been worked on since the last sync.

1a. Load project registry:

Read: intake/projects.yaml

Parse the projects list. For each project:

# Check if project path exists
ls [project_path]/docs/toolkit-proposals/

# Read all proposal files
Glob: [project_path]/docs/toolkit-proposals/*.md

1b. Load intake/pending/ proposals:

Glob: intake/pending/from-projects/*.md
Glob: intake/pending/from-rca/*.md
Glob: intake/pending/from-research/*.md

1c. Filter out already-processed proposals:

For each proposal file, check YAML frontmatter: - Skip if toolkit_proposal_status: applied - Skip if toolkit_proposal_status: rejected - Skip if toolkit_proposal_status: rejected-at-gate - Skip if toolkit_proposal_status: qualified (already passed merit gate, pending spec creation) - Skip if toolkit_proposal_status: completed - Skip if toolkit_proposal_status: resolved - Skip if toolkit_proposal_status: spec-created - Skip if toolkit_proposal_status: reviewed - Process if toolkit_proposal_status: proposed or field is absent - Process if toolkit_proposal_status: deferred-to-spec (re-evaluate with merit assessment) - Process if toolkit_proposal_status: needs-modification (re-evaluate after changes)

If no proposals found:

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Toolkit Review - No Pending Proposals
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

Scanned:
• N registered projects
• intake/pending/ (0 files)

No proposals to process.
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

Step 2: Group by Target

Group all collected proposals by target_skill from their YAML frontmatter:

pd-5-specs: (2 proposals)
  [2026-02-13] Missing UX quality criteria in specs (high) — V024-Websites-dev
  [2026-02-13] Missing a11y acceptance criteria (high) — V024-Websites-dev

scaffold: (1 proposal)
  [2026-02-13] critical-patterns.md should include UX patterns (medium) — RCA-001

research-ingest: (1 proposal)
  [2026-02-14] New skill idea from article (low) — research

If multiple proposals target the same section of the same skill, merge them into one combined proposal. Note the merge in the presentation.

Step 3: Analyze, Assess Merit, and Generate Edit Proposals

For each target skill:

  1. Read the current skill file: 

    Read: plugins/core-standards/skills/[target_skill]/SKILL.md

  2. Read all proposals targeting it

  3. Read the constitution (if it exists): 

    Read: .specify/memory/constitution.md

  4. Run Merit Assessment — 5 adversarial checks that challenge whether the proposal should be implemented at all:

Check A: Problem Reality - Is this problem reported by a real project, RCA, or user — or is it purely theoretical? - Has the current state actually caused issues in practice (drift, confusion, bugs)? - If the proposal originates from external article comparison with no experienced pain → flag as THEORETICAL - If the proposal originates from an RCA, project feedback, or documented incident → CONFIRMED

Check B: Constitution Compliance - Does the proposed change strengthen or weaken each Constitution principle? - Especially Principle IV (Composability): does it break self-containment of any skill or agent? Does it introduce cross-file dependencies where none existed? - If the change weakens a principle → flag as VIOLATION([principle]) - If there is a tradeoff between principles → flag as TENSION([principle], [explanation])

Check C: Consumer Identification - Who specifically uses the changed component? Name the consumers (skills, agents, workflows, users). - If the proposal creates new files: who reads those files? Name the consumer. - If the only beneficiary is "future flexibility", "potential reuse", or "cleaner architecture" with no current consumer → flag as SPECULATIVE

Check D: Duplication Analysis (only when the proposal addresses duplication) - Do both copies need to exist for their respective components to work standalone? - Has the duplication actually caused drift? Are the copies identical or have they diverged? - If both components need the content inline for standalone operation → flag as FUNCTIONAL - Skip this check (mark N/A) if the proposal is not about reducing duplication.

Check E: Cost-Benefit - Net file count change (more files = more complexity to maintain) - Net cross-file dependency count (new references between files) - Does the value gained justify the complexity added? - If complexity increases without clear, concrete benefit → flag as NET_NEGATIVE

  1. Generate Merit Verdict from the check results:
Verdict Condition Recommended action
SOUND All checks pass (no flags) Approve or defer-to-spec
QUESTIONABLE 1-2 checks flagged concerns Review concerns carefully before deciding
DETRIMENTAL VIOLATION + NET_NEGATIVE, or 3+ flags, or Problem is THEORETICAL + Consumer is SPECULATIVE Reject (user can override)
  1. Generate a specific edit proposal (same as before):
  2. Identify the exact section to modify (by content, not just line number)
  3. Draft the exact content to add or change
  4. Provide rationale from the source proposals

Step 4: Present for Approval

For each target, present both the merit verdict and the edit proposal using AskUserQuestion:

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Proposal 1/N: Update [target-skill]
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

Source: [source] ([count] proposals, [severity])
Target: plugins/core-standards/skills/[target]/SKILL.md

┌─────────────────────────────────────────────────────┐
│  MERIT ASSESSMENT: [SOUND | QUESTIONABLE | DETRIMENTAL]
│                                                     │
│  A. Problem Reality: [CONFIRMED | THEORETICAL]      │
│     [1-line explanation]                            │
│                                                     │
│  B. Constitution: [OK | VIOLATION(N) | TENSION(N)]  │
│     [1-line explanation]                            │
│                                                     │
│  C. Consumer: [IDENTIFIED: x, y | SPECULATIVE]      │
│     [1-line explanation]                            │
│                                                     │
│  D. Duplication: [N/A | FUNCTIONAL | ACCIDENTAL]    │
│     [1-line explanation, if applicable]             │
│                                                     │
│  E. Cost-Benefit: [NET_POSITIVE | NEUTRAL | NET_NEGATIVE]
│     [1-line explanation]                            │
└─────────────────────────────────────────────────────┘

Proposed change:
  [the edit proposal from Step 3.6]

Evidence:
  [source references]
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

Options depend on the merit verdict:

When verdict is SOUND: - Approve — Apply this change to the toolkit file - Edit — Modify the proposed content before applying - Defer to spec — Too large for direct edit, create a spec - Reject — Don't apply (mark as rejected)

When verdict is QUESTIONABLE: - Approve — Apply despite concerns - Edit — Modify to address concerns, then apply - Defer to spec — Create a spec (concerns carry forward for Defense Line 2) - Reject — Don't apply (mark as rejected with merit assessment preserved)

When verdict is DETRIMENTAL: - Reject (Recommended) — Don't apply (mark as rejected with merit assessment preserved) - Edit — Modify to address concerns, then re-assess - Approve anyway — Override the assessment (user has final say) - Defer to spec — Create a spec despite concerns

In all cases, View source is available as a secondary action to show the full source proposal(s).

Learning capture: When a proposal is rejected based on merit assessment, append the verdict and rejection reason to the proposal file as a ## Merit Assessment section. This preserves the reasoning for institutional learning (Constitution Principle III).

Step 5: Apply Approved Changes

For each proposal with a decision:

Approved proposals: 1. Apply the edit to the target toolkit file using Edit tool 2. Mark source proposals as applied: - For project proposals: update toolkit_proposal_status: applied in the source file - For intake/pending/ proposals: move file to intake/processed/

Deferred-to-spec proposals: 1. Update toolkit_proposal_status: deferred-to-spec in the proposal file 2. Move to intake/processed/: git mv [proposal_path] intake/processed/[from-source]/ — the proposal is processed; the spec handles everything from here

Rejected proposals: 1. Update toolkit_proposal_status: rejected in the proposal file 2. Append ## Merit Assessment section with verdict and rejection reason (already documented above) 3. Move to intake/processed/: git mv [proposal_path] intake/processed/[from-source]/ — the proposal was evaluated and dismissed; keeping it in pending is misleading

Step 6: Summary Report

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Toolkit Review Complete
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

Proposals processed: N
  Approved and applied: X
  Skipped: Y
  Edited and applied: Z

Files updated in toolkit:
  - skills/vt-c-pd-5-specs/SKILL.md (added quality criteria section)
  - templates/vt-c-scaffold/coding/critical-patterns.md (added 6 UX patterns)

Source proposals marked as applied in:
  - V024-Websites-dev/docs/toolkit-proposals/ (2 files)
  - intake/pending/from-rca/ → intake/processed/ (1 file moved)

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Ready to commit. Run /vt-c-commit to create a commit on the current branch.

Suggested message: "Apply toolkit improvements from V024 project feedback"
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

--from-rca Mode

Import learnings directly from an RCA (Root Cause Analysis) document.

Execution

/vt-c-toolkit-review --from-rca ~/path/to/RCA-document.md

Steps

  1. Read the RCA document in full

  2. Extract actionable items:

  3. Each root cause (RC-N) that identifies a toolkit gap
  4. Each defect category that could be prevented by a checklist/pattern

  5. Each "should exist" recommendation

  6. Map each to a target skill: | RCA Finding | Target Skill | |---|---| | Missing quality criteria in specs | pd-5-specs, kw-specs | | Missing quality gate | implementation-orchestrator | | Missing pattern in checklist | scaffold (critical-patterns template) | | Missing auto-detection | continuous-learning | | Missing review step | workflow skill (3-build, 4-review) |

  7. Generate proposal files in intake/pending/from-rca/:

Filename: YYYY-MM-DD-[rca-id]-[target-skill].md

---
type: procedure-improvement
date: YYYY-MM-DD
source: RCA-001
source_path: /path/to/RCA-document.md
target_skill: pd-5-specs
severity: high
category: missing-criteria
toolkit_proposal_status: proposed
---

# Procedure Improvement: Add UX quality criteria to specs template

## Source
RCA-001: Repeated UX Defects Across Sibling Apps
Root Cause RC-3: UX quality criteria are not in the specs

## What Went Wrong
[Extracted from RCA]

## Evidence
[Extracted from RCA - defect counts, specific examples]

## Proposed Change
[Specific addition to the target skill]

## Expected Impact
[How many future occurrences this prevents]
  1. Continue with normal review flow (Step 1-6 above)

--scan Mode

Scan a specific project and optionally register it.

Execution

/vt-c-toolkit-review --scan ~/path/to/project

Steps

  1. Check if docs/toolkit-proposals/ exists in the project
  2. If proposals found, process them through the normal flow
  3. If project is not in intake/projects.yaml, ask whether to register it: 
    Project ~/path/to/project is not registered.
Register it for automatic scanning?
  4. If yes, append to intake/projects.yaml

--status Mode

Show pending proposals without processing them.

Execution

/vt-c-toolkit-review --status

Output

━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
Toolkit Review Status
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

Registered projects: N
  - V024-Websites-dev (2 pending proposals)
  - V025-Some-Project (0 pending proposals)

Intake pending:
  from-projects/: 0 files
  from-rca/: 1 file
  from-research/: 3 files

Total pending: 6 proposals

By target skill:
  pd-5-specs: 2 (high)
  scaffold: 1 (medium)
  research: 3 (mixed)

Run /vt-c-toolkit-review to process all pending proposals.
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━

Proposal Format

All proposals (from any source) use this standardized YAML frontmatter:

---
type: procedure-improvement
date: YYYY-MM-DD
source: [project-name | RCA-NNN | research-ingest]
source_path: [path to source file]
target_skill: [skill name in toolkit]
target_file: [specific file path, if known]
severity: [high | medium | low]
category: [missing-criteria | missing-step | wrong-default | missing-check | new-pattern]
toolkit_proposal_status: [proposed | approved | applied | rejected | rejected-at-gate | deferred-to-spec | qualified | needs-modification | spec-created | completed | resolved | reviewed]
tags: [relevant, tags]
---

Integration Points

Invoked by: - /vt-c-toolkit-review command (primary interface) - Recommended after completing a project milestone or retrospective

Receives proposals from: - session-journal (procedure-improvement entries in project's docs/toolkit-proposals/) - compound-docs (Option 8 in decision menu) - --from-rca flag (RCA document import) - research-ingest (proposals in intake/pending/from-research/)

Does not invoke: - research-implement (separate skill for research proposals with full scaffolding logic)

Error Handling

Project path not found: 

Project path does not exist: ~/path/to/project
Skipping. Update intake/projects.yaml to remove stale entries.

No proposals directory in project: 

No docs/toolkit-proposals/ in V024-Websites-dev.
This project has not generated any toolkit proposals yet.

Invalid proposal format: 

Proposal [filename] is missing required fields: target_skill
Skipping this proposal. Fix the YAML frontmatter and re-run.

Target skill not found: 

Target skill "nonexistent-skill" not found in toolkit.
Skipping proposal. Available skills: [list nearby matches]