|
| 1 | +# Output Contracts |
| 2 | + |
| 3 | +Two machine-readable artifacts plus one human-readable Markdown narrative. |
| 4 | + |
| 5 | +## 1. `candidates.json` — input to the LLM (produced by `scripts/cluster_and_seed.py`) |
| 6 | + |
| 7 | +```json |
| 8 | +{ |
| 9 | + "schemaVersion": "1.0", |
| 10 | + "generatedAt": "2026-04-30T16:00:00Z", |
| 11 | + "newsDataDir": "/abs/path/to/server/data", |
| 12 | + "totalScanned": 124, |
| 13 | + "totalUnique": 87, |
| 14 | + "clusters": [ |
| 15 | + { |
| 16 | + "clusterKey": "llm__long-context", |
| 17 | + "clusterLabel": "Long-context LLMs", |
| 18 | + "matchedDomain": "Large Language Models", |
| 19 | + "primaryKeyword": "long-context", |
| 20 | + "items": [ |
| 21 | + { |
| 22 | + "canonicalId": "arxiv:2604.12345v1", |
| 23 | + "title": "...", |
| 24 | + "abstract": "...", |
| 25 | + "authors": ["..."], |
| 26 | + "url": "https://arxiv.org/abs/2604.12345", |
| 27 | + "sources": [ |
| 28 | + { "source": "arxiv", "score": 8.4, "matchedKeywords": ["long-context", "transformer"] }, |
| 29 | + { "source": "huggingface", "score": 7.1, "matchedKeywords": ["long-context"] } |
| 30 | + ], |
| 31 | + "blendedScore": 7.95, |
| 32 | + "publishedDate": "2026-04-28", |
| 33 | + "engagement": { "stars": null, "likes": null, "upvotes": 12 } |
| 34 | + } |
| 35 | + ] |
| 36 | + } |
| 37 | + ] |
| 38 | +} |
| 39 | +``` |
| 40 | + |
| 41 | +**Canonical id rules** (in priority order): |
| 42 | + |
| 43 | +1. `arxiv:<id>` if any source attaches an arXiv ID |
| 44 | +2. `gh:<owner>/<repo>` if a GitHub repo URL |
| 45 | +3. `hf:<repo_id>` if a HuggingFace Hub repo |
| 46 | +4. `wechat:<account_route>:<title_hash>` for WeChat 公众号 articles |
| 47 | +5. `xhs:<note_id>` / `x:<tweet_id>` for social posts |
| 48 | +6. `title:<sha1(title)[:12]>` as the last-resort fallback |
| 49 | + |
| 50 | +**Cross-source merging**: when two items resolve to the same canonical id, the helper keeps the highest scored entry as the "primary" and stacks the other source(s) into `sources[]`. `blendedScore` is the simple max across sources today (Phase 3 / Option B2 will replace this with a proper blend). |
| 51 | + |
| 52 | +## 2. `seeds.json` — output (you produce this) |
| 53 | + |
| 54 | +```json |
| 55 | +{ |
| 56 | + "schemaVersion": "1.0", |
| 57 | + "generatedAt": "2026-04-30T16:14:00Z", |
| 58 | + "candidatesFile": "./.cache/news-idea-briefing/candidates.json", |
| 59 | + "researchBrief": ".pipeline/docs/research_brief.json", |
| 60 | + "clusters": [ |
| 61 | + { |
| 62 | + "name": "Sparse attention for million-token context", |
| 63 | + "summary": "...", |
| 64 | + "supportingItems": ["arxiv:2604.12345v1", "gh:foo/bar", "hf:org/model"] |
| 65 | + } |
| 66 | + ], |
| 67 | + "seeds": [ |
| 68 | + { |
| 69 | + "id": "seed-1", |
| 70 | + "cluster": "Sparse attention for million-token context", |
| 71 | + "title": "Block-sparse retrieval head ablation on long-form QA", |
| 72 | + "rationale": "Item [arxiv:2604.12345v1] introduces block-sparse attention but only evaluates on …. Item [gh:foo/bar] open-sources the head but never benchmarks against the standard …. The gap is measuring whether the sparsity gain holds when …", |
| 73 | + "first_experiment": "Reproduce [arxiv:2604.12345v1]'s table 3 baseline, then ablate the retrieval head on the long-form QA subset of …", |
| 74 | + "risk": "If the gain in [arxiv:2604.12345v1] is dataset-specific, the ablation will null-result and tell us nothing new.", |
| 75 | + "confidence": "medium", |
| 76 | + "noveltyCheck": { "ran": true, "duplicates": [] } |
| 77 | + } |
| 78 | + ] |
| 79 | +} |
| 80 | +``` |
| 81 | + |
| 82 | +**Required fields per seed**: `id`, `cluster`, `title`, `rationale`, `first_experiment`, `risk`, `confidence`. Optional: `noveltyCheck`. |
| 83 | + |
| 84 | +**`confidence` policy**: |
| 85 | + |
| 86 | +- `high` — concrete gap clearly identified, ≥ 2 grounded citations, novelty check clean |
| 87 | +- `medium` — plausible gap, ≥ 1 grounded citation, no novelty check or clean check |
| 88 | +- `low` — speculative; or novelty check found a near-duplicate |
| 89 | + |
| 90 | +## 3. `idea_briefing.md` — human-readable narrative |
| 91 | + |
| 92 | +Structure (in this order, with these exact heading levels): |
| 93 | + |
| 94 | +```markdown |
| 95 | +# Idea Briefing — <YYYY-MM-DD> |
| 96 | + |
| 97 | +> Generated from <N> unique items across <M> sources. <K> clusters surfaced; <S> idea seeds proposed. |
| 98 | +
|
| 99 | +## Today at a glance |
| 100 | +- 1-paragraph overview citing 3-5 of the most striking items by canonical id |
| 101 | + |
| 102 | +## Clusters |
| 103 | +### <Cluster name> |
| 104 | +**What's new** — <1-2 sentences, cited> |
| 105 | +**Key items** |
| 106 | +- [<canonical id>] <title> — <one-line "why this matters"> |
| 107 | +- ... |
| 108 | +**Idea seeds** |
| 109 | +- **<seed title>** (<confidence>) — <rationale> <first experiment> <risk> |
| 110 | + |
| 111 | +(Repeat per cluster) |
| 112 | + |
| 113 | +## Lateral pick (optional) |
| 114 | +<If a cluster from an adjacent domain is included, briefly explain why.> |
| 115 | + |
| 116 | +## Methodology note |
| 117 | +- <how candidates were aggregated, what was excluded, what limitations to keep in mind> |
| 118 | +``` |
| 119 | + |
| 120 | +The Markdown is the artifact a human reads. The JSON is what Option C's "Promote to Auto Research" button consumes. |
| 121 | + |
| 122 | +## Where files land |
| 123 | + |
| 124 | +``` |
| 125 | +<project>/ |
| 126 | +├── .cache/news-idea-briefing/ |
| 127 | +│ └── candidates.json # ephemeral; safe to delete |
| 128 | +└── Ideation/ |
| 129 | + └── proactive/ |
| 130 | + └── 2026-04-30/ |
| 131 | + ├── idea_briefing.md |
| 132 | + └── seeds.json |
| 133 | + └── 2026-04-30-2/ # if a second run happens same day |
| 134 | + ├── ... |
| 135 | +``` |
| 136 | + |
| 137 | +Briefings are **append-only** history. Never overwrite an existing date folder; suffix with `-2`, `-3`, etc. |
0 commit comments