🔍 Agentic Workflow Audit Report - 2025-12-28

[github-actions[bot]]

Audit Summary

• Period: Last 24 hours (2025-12-27 to 2025-12-28)
• Runs Analyzed: 81
• Workflows Active: 29
• Success Rate: 90.12% (73 success, 2 failure, 6 in-progress/cancelled)
• Issues Found: 583 (570 errors, 544 warnings, 13 missing tool reports)

Performance Metrics

• Total Token Usage: 36,386,858 tokens
• Average Token Usage: 449,220 tokens/run
• Total Cost (24h): $10.24
• Average Cost: $0.13/run
• Average Turns: 4.2 turns/run
• Total Duration: 5.3 hours (19,031 seconds)
• Average Duration: 3.9 minutes/run

📊 Trend Charts

Workflow Health - Success/Failure Trends

Analysis: The workflow health chart shows consistent high success rates throughout the 24-hour period, with most time windows achieving 100% success. The two failures occurred during the 03:00 and 02:00 UTC windows but did not significantly impact overall system reliability.

Token Usage & Cost Trends

Analysis: Token usage peaked during the 02:00-03:00 UTC window with approximately 10M tokens consumed, corresponding to high-cost workflows like CI Cleaner and Security Fix PR. Most time windows show moderate token usage (500K-2M tokens) with costs between $0.50-$2.00 per window.

Missing Tools

13 missing tool reports were recorded across 4 workflows:

Tool Name	Count	Workflows Affected	Primary Issue
safeinputs-gh	4	Smoke Codex, Smoke Copilot Playwright, Smoke Copilot Safe Inputs	CLI tool not available in toolchain
Build tools (make, go, node, npm, golangci-lint)	8	Tidy	Required for code formatting, linting, and compilation
Playwright MCP	1	Smoke Copilot Playwright	Browser automation tools not available

Impact: The Tidy workflow had 8 reports of missing build tools, indicating it couldn't complete its full code quality checks. The smoke test workflows missing safeinputs-gh couldn't test that specific tool integration.

Error Analysis

Critical Errors: 0

No critical errors that blocked workflow execution.

Warnings: 544

Most warnings (468) are related to empty or malformed GitHub API responses (pattern: blank). These appear to be non-blocking issues with API tool calls that return empty results.

Top Warning Patterns:

1. Blank/Empty responses (468 occurrences): github-get_me and similar API calls returning empty data
2. JSON parse errors (102 occurrences): Unexpected token 'i', "failed to ge"... is not valid JSON

Note: These warnings don't appear to cause workflow failures but may indicate API rate limiting or transient network issues.

MCP Server Failures

✓ No MCP server failures detected in the 24-hour audit period. All MCP servers (github, safeoutputs, gh-aw) remained operational.

Firewall Analysis

• Total Requests: 420
• Allowed Requests: 420 (100%)
• Denied Requests: 0 (0%)

Top Allowed Domains

1. api.githubcopilot.com: 221 requests (52.6%)
2. api.github.com: 105 requests (25.0%)
3. api.enterprise.githubcopilot.com: 46 requests (11.0%)
4. proxy.golang.org: 22 requests (5.2%)
5. registry.npmjs.org: 10 requests (2.4%)
6. github.com: 9 requests (2.1%)
7. raw.githubusercontent.com: 3 requests (0.7%)
8. Other: 4 requests (1.0%)

✓ No denied domains - Firewall configuration is working correctly and all required domains are properly whitelisted.

Affected Workflows

Failed Runs (2)

1. Daily Project Performance Summary Generator (Using Safe Inputs) - §20547997301

   • Errors: 0, Warnings: 2
   • Status: Failed during execution

2. Daily Issues Report Generator - §20541385513

   • Errors: 0, Warnings: 16
   • Status: Failed during execution

High Token Usage Workflows (>1M tokens)

1. CI Cleaner - 8.4M tokens, $0.00 cost - §20542525486
2. Safe Output Health Monitor - 2.8M tokens, $1.50 cost - §20545731238
3. Security Fix PR - 2.0M tokens, $1.52 cost - §20541267975
4. The Great Escapi - 1.8M tokens, $0.00 cost - §20546849596
5. Tidy - 1.7M tokens, $0.00 cost - §20542126722

Recommendations

1. Investigate Failed Workflows

• Priority: High
• Action: Review logs for "Daily Project Performance Summary Generator" and "Daily Issues Report Generator" to identify root cause of failures
• Both workflows failed with only warnings, no errors - suggests configuration or environment issues

2. Address Missing Build Tools for Tidy Workflow

• Priority: Medium
• Action: Ensure make, go, node, npm, and golangci-lint are available in the Tidy workflow environment
• Impact: Without these tools, Tidy cannot perform code formatting, linting, or testing
• Solution: Add setup steps or use pre-configured action runners with these tools

3. Add safeinputs-gh Tool to Smoke Test Workflows

• Priority: Low
• Action: Make safeinputs-gh CLI tool available to smoke test workflows
• Impact: Smoke tests for this tool cannot complete without it
• Affected Workflows: Smoke Codex, Smoke Copilot Playwright, Smoke Copilot Safe Inputs, Smoke Claude

4. Monitor High Token Usage Workflows

• Priority: Medium
• Action: Review CI Cleaner (8.4M tokens), Safe Output Health Monitor (2.8M tokens), and Security Fix PR (2.0M tokens) for optimization opportunities
• Impact: These workflows account for 34% of total token usage and could benefit from efficiency improvements
• Estimated Savings: Reducing token usage by 20% could save ~$2/day

5. Investigate GitHub API Warning Pattern

• Priority: Low
• Action: Investigate 468 occurrences of blank API responses and 102 JSON parse errors
• Root Cause: Likely API rate limiting, transient network issues, or incorrect error handling
• Solution: Add retry logic and better error handling for API calls

Historical Context

This is the first automated audit report. Historical trends will be available in future reports as data accumulates over the 90-day retention period.

Baseline Metrics Established:

• Success Rate: 90.12%
• Average Cost: $0.13/run
• Average Tokens: 449K/run
• Issues: 583 total

Future audits will compare against these baselines to identify trends and regressions.

Next Steps

• Audit completed and report generated
• Findings stored in repository memory (memory/audit-workflows branch)
• Trend charts uploaded and embedded
• Follow up on failed workflow investigations
• Address missing tools in Tidy workflow
• Monitor token usage trends in high-consumption workflows

References:

• Agentic Workflow Audit Agent (Current Run) - §20548565786
• Daily Project Performance Summary Generator (Failed) - §20547997301
• Daily Issues Report Generator (Failed) - §20541385513

AI generated by Agentic Workflow Audit Agent