Copilot PR Prompt Analysis - 2025-12-29

[github-actions[bot]]

🤖 Copilot PR Prompt Pattern Analysis - 2025-12-29

Summary

Analysis Period: Last 30 days
Total PRs: 1000 | Merged: 735 (74.2%) | Closed: 256 (25.8%)

Prompt Categories and Success Rates

Category	Total	Merged	Closed	Success Rate
Documentation	1	1	0	100.0%
Testing	43	40	3	93.0%
Refactor	25	20	5	80.0%
Other	277	217	60	78.3%
Feature	165	119	46	72.1%
Bug Fix	480	338	142	70.4%

Prompt Analysis

✅ Successful Prompt Patterns

Common characteristics in merged PRs:

• Average prompt length: 111.7 words
• File references: 318 PRs (43.3%)
• Issue references: 317 PRs (43.1%)

Top categories by success rate:

• Documentation: 100.0% success rate (1/1 merged)
• Testing: 93.0% success rate (40/43 merged)
• Refactor: 80.0% success rate (20/25 merged)

Example successful prompts:

1. PR Enable GitHub lockdown mode in org-health-report workflow #8069: Use GitHub lockdown mode in https://github.com/githubnext/gh-aw/blob/main/.github/workflows/org-health-report.md... → Merged (link)

2. PR Add mdflow syntax comparison analysis #8066: Review syntax of mdflow compared to agentic workflows https://github.com/johnlindquist/mdflow... → Merged (link)

3. PR Remove obsolete action references and clean up actions build system #8055: Delete actions/setup-safe-inputs, actions/setup-safe-outputs actions. They are not needed anymore.... → Merged (link)

❌ Unsuccessful Prompt Patterns

Common characteristics in closed PRs:

• Average prompt length: 149.4 words (33% longer than merged PRs)
• File references: 109 PRs (42.6%)
• Issue references: 154 PRs (60.2%)

Lower success rate categories:

• Bug Fix: 70.4% success rate (338/480 merged, 142 closed)
• Feature: 72.1% success rate (119/165 merged, 46 closed)
• Other: 78.3% success rate (217/277 merged, 60 closed)

Example unsuccessful prompts:

1. PR Document GitHub Actions security practices #8013: ---- This section details on the original issue you should resolve (issue_title)[plan] Document GitHub Actions securit... → Closed (link)

2. PR Revert "Fix ES6 import syntax in CommonJS test files" #8005: Reference: https://github.com/githubnext/gh-aw/actions/runs/20558896885/job/59046870105#step:38:1... → Closed (link)

3. PR Refactor Docker image download step into reusable shell script #8003: Refactor "name: Downloading container images" steps into a .sh file in the actions/setup action.... → Closed (link)

Key Insights

Based on the analysis of 1000 Copilot-generated PRs:

• Conciseness matters: Merged PRs have prompts averaging 111.7 words vs 149.4 words for closed PRs (25% shorter)
• Testing and refactoring excel: Testing prompts have a 93% success rate, refactoring at 80% - these focused, technical tasks work well
• Bug fixes are challenging: Only 70.4% of bug fix prompts succeed, likely due to complexity and need for deep context
• Specificity helps: 43.3% of successful PRs reference specific files

Recommendations

Based on today's analysis:

1. DO: Keep prompts concise and focused - aim for under 120 words
2. DO: Reference specific files, functions, or code locations when possible
3. DO: Use clear action verbs (add, fix, refactor, validate) at the start
4. AVOID: Overly verbose prompts with extensive background - focus on the specific task
5. AVOID: Vague instructions - be specific about what needs to change

Success Formula

High-success prompt template:

[Action verb] [specific target] [brief context if needed]

Example: "Add validation in bundler.go to prevent mixing Node.js and GitHub Script runtime modes"

What works:

• Short and direct (under 120 words)
• Specific file/function references
• Clear single objective
• Testing or refactoring focus

What struggles:

• Long prompts with extensive context (over 150 words)
• Multiple objectives in one prompt
• Vague descriptions
• Complex bug fixes without clear reproduction steps

---

Workflow Run: §20572287964

AI generated by Copilot PR Prompt Pattern Analysis