Skip to content

fix: 每日免费抓取与旧日志索引修复验收 - #76

Merged
yi-ge merged 1 commit into
mainfrom
codex/docsearch-daily-and-changelog
Oct 15, 2026
Merged

yi-ge merged 1 commit into
mainfrom
codex/docsearch-daily-and-changelog

Conversation

@yi-ge

@yi-ge yi-ge commented Oct 15, 2026

Copy link
Copy Markdown
Member

变更说明 / Description

按用户要求,在确认免费额度后将既有DocSearch crawler改为每日抓取,修复中英文v10.0.180更新日志的超大记录错误,并完成重抓与搜索验收。

  • 当前账单与现行DocSearch条款确认每月5,000,000抓取资源免费;按738URL×31天估算22,878,约0.46%。未升级套餐。
  • 云端配置v5已发布 every 1 day。仅两个精确日志路径在提取DOM按10条完整li分组并关闭该页聚合;保留781正文项、标题/id、HTML及顺序,其它页原样提取。
  • 原123KB单record超过100,000B限制;URLTester修复后双语各89records成功,英文Webhook对照110不变。
  • 正式v5抓取736success、2内容类型ignored、0failed;索引21,572→21,750。
  • 30搜索请求均200,28目标验收通过。保留2条未命中的带引号严格短语诊断;独立补查返回完整中部内容,未将诊断伪造成通过。
  • 双语真实搜索弹窗均能查到并打开对应v180章节。公开参考配置不含密钥,网站部署不会自动应用该管理端配置。

The existing free DocSearch allowance supports the requested daily crawl at the current site size. Published cloud configuration v5 changes the schedule to daily and groups complete list items only for the two affected v10.0.180 pages. It preserves content and the DocSearch helper's metadata/IDs. The real crawl completed with zero failures; both formerly failing pages now index successfully, and API regression/search checks passed. Two strict quoted-query diagnostic misses are retained separately from successful content-retrieval checks.

验证 / Validation

  • Actual cloud URLTester: 89 records per changelog locale; API control 110 unchanged.
  • Actual crawl: 736 success / 2 ignored / 0 failed; 21,750 live records.
  • API16 + endpoints4 + controls2 + changelog titles2 + body targets4 passed; real bilingual browser search/navigation passed.
  • Actual scoped extractor with Cheerio: 781 Markdown li → 80 groups; 3 breadcrumb li and all heading IDs untouched; complete HTML/text/order preserved.
  • node --check api/algolia-crawler.config.js, pnpm typecheck, pnpm api:check -- --build, JSON/sensitive-field checks, git diff --check passed.
  • No website page, static asset, runtime configuration, dependency or build script change; published changelog text is unchanged.

@vercel

vercel Bot commented Oct 15, 2026 •

Copy link
Copy Markdown

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated
docs Ready Ready Preview Oct 15, 2026 1:07pm UTC

Request Review

@yi-ge
yi-ge merged commit 3e0e3de into main Oct 15, 2026
6 checks passed
@yi-ge
yi-ge deleted the codex/docsearch-daily-and-changelog branch October 15, 2026 13:10

This branch was successfully deployed

1 active deployment
Preview — 3348af0d Deployed Oct 15, 2026 by vercel[bot]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant