Skip to content

fix: 让 TIGER 示例可运行并新增 MovieLens 脚本 - #245

Merged
1985312383 merged 3 commits into
datawhalechina:mainfrom
TyndaleLym:fix/tiger-example-workflow
Jun 8, 2026
Merged

fix: 让 TIGER 示例可运行并新增 MovieLens 脚本#245
1985312383 merged 3 commits into
datawhalechina:mainfrom
TyndaleLym:fix/tiger-example-workflow

Conversation

@TyndaleLym

Copy link
Copy Markdown
Contributor

背景

PR 3(TIGER Runnable Example)。原 run_tiger_amazon_books.py 存在一组导致无法稳定运行的问题,且缺少 MovieLens 脚本、测试与文档。本 PR 修复这些问题并补齐 MovieLens 支持。

主要改动

TIGER 示例修复(run_tiger_amazon_books.py

  • 运行模式拆分为 generate-toy-data / train / test / all--mode
  • 统一生成与读取的数据路径,避免"生成文件名 ≠ 读取文件名"
  • 移除写死的本机绝对 ckpt_path 默认值,回退到 --output_dir
  • 训练前 add_tokens 语义 ID 并 resize_token_embeddings(原代码漏掉,导致 <a_1> 被切成子词)
  • 测试时从 ckpt_path 加载 tokenizer/config/model,并把输入张量搬到对应 device

模型兼容性(torch_rechub/models/generative/tiger.py

  • 适配 transformers>=5:补 model_parallel/device_map(DataParallel 守卫),Trainertokenizerprocessing_class 按签名自动适配

新增 MovieLens 脚本(run_tiger_movielens.py

  • 与 HSTU/HLLM 一致的"一数据集一脚本"风格
  • 真实数据的 RQ-VAE → TIGER 流水线在 docstring/文档中写明

测试(tests/test_tiger_example.py

  • TigerSeqDataset 三种模式切分、get_new_tokens/get_all_items、collate 的 PAD→-100 屏蔽、Trie 前缀约束
  • toy 数据构建 + prepare-data + 一个真实 T5 的端到端 smoke(离线自动 skip)

文档

  • docs/{zh,en}/models/generative.md 新增 TIGERModel 小节
  • 新增 docs/{zh,en}/blog/tiger_reproduction.md 工作流文档并注册 VitePress 侧边栏

测试结果

  • isort / yapf / flake8(与 CI 同 ignore 列表)均通过
  • pytest tests/(排除 slow 与缺失可选依赖 pyarrow/onnx 的用例):49 passed / 41 skipped / 0 failed
  • 实测 toy --mode all:MovieLens 与 Amazon 均端到端跑通并输出 hit@k / ndcg@k

注意事项

  • 真实数据的 semantic_ids.json 仍需用 RQ-VAE 生成,且 item id 必须与 inter.json 对齐
  • 离线环境请用 --base_model google-t5/t5-small(规范仓库名)

TyndaleLym and others added 2 commits June 2, 2026 16:20
- split run modes into generate-toy-data/train/test/all (+ prepare-data for MovieLens)
- align generated and read data paths; drop hardcoded absolute ckpt_path default
- add semantic-id tokens and resize embeddings before training
- load tokenizer/config/model from ckpt_path at test time, move inputs to device
- fix TIGERModel for transformers>=5 (model_parallel guard, Trainer processing_class)
- add run_tiger_movielens.py with toy data and a ratings.dat prepare-data mode
- add TigerSeqDataset/Trie/collate unit tests and a toy end-to-end smoke test
- document TIGER in generative.md (zh/en) and add tiger_reproduction.md (zh/en)

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@github-actions github-actions Bot added documentation Improvements or additions to documentation | 文档更新 enhancement New feature or request | 新功能 model New model or model improvement | 模型相关 bug Something isn't working | Bug 修复 labels Jun 2, 2026
@1985312383

Copy link
Copy Markdown
Collaborator
  1. train() 没有从 --base_model 加载预训练 T5 权重,而是 TIGERModel(config) 随机初始化。PR 说 fine-tune T5,这和代码不一致,也会严重影响训练成本和效果。

  2. test() 里“checkpoint tokenizer 缺 semantic-id token 时自动补 token”的 fallback 会因 embedding size mismatch 在 from_pretrained 阶段失败,代码到不了后面的 resize。

…kpt load

Addresses review on datawhalechina#245:

- prepare-data: add --vocab_path to reuse the shared HSTU/HLLM vocab.pkl so
  inter.json item ids are the same token ids as the item embeddings and RQ-VAE
  semantic ids (TIGER/HSTU/HLLM share one id space). Falls back to the standalone
  movie_id->1..N mapping when omitted.
- test(): load the model with the checkpoint's own config and resize only on a
  tokenizer mismatch; previously a grown vocab_size was passed to from_pretrained,
  raising an embedding-size mismatch before the resize could run.
- Keep training from scratch (random init) per the TIGER paper; correct the
  misleading "fine-tune T5" wording in docstrings and the zh/en reproduction docs.
- Make the MovieLens toy data a deterministic +1 walk so the toy run shows
  non-trivial accuracy once training works.
- test(): set use_cache=True to speed up constrained beam search.
- tests: add a vocab-aligned prepare-data test.
@TyndaleLym

Copy link
Copy Markdown
Contributor Author

1、关于 "fine-tune T5" 的措辞

TIGER 原论文里 T5 编码器-解码器就是从头随机初始化训练的(语义 ID 词表不是自然语言,预训练的 NL 权重没有意义),官方参考实现 NonameUntitled/tiger 也是 T5ForConditionalGeneration(config) 随机初始化。所以 train() 用 TIGERModel(config) 从 --base_model 的 config 构建架构、随机初始化,是符合论文的;--base_model 只提供架构/config 与 tokenizer,不加载预训练权重。

因此我保留了训练代码,只把 PR 描述和文档里误导性的 "fine-tune T5" 改成“从头训练(随机初始化,不加载预训练权重)”,并在 train() docstring 和复现 blog 里写明了这一点,避免后续读者误解。

2、关于 test() 的 embedding size mismatch

这是真 bug,已修复。原因正如你所说:之前把已经增长过的 vocab_size 传进 from_pretrained,会在加载阶段就因 embedding size 不匹配报错,根本到不了后面的 resize。

现在改为:用 checkpoint 自带的 config 加载模型(TIGERModel.from_pretrained(ckpt),不再传入外部 config),embedding 尺寸与保存权重天然一致;之后只在 tokenizer 与 embedding 行数确实不一致时(例如喂进一个原始 T5 checkpoint 的兜底情形)才 resize_token_embeddings。这样既修了崩溃,也保留了原来的兜底逻辑。

@1985312383
1985312383 merged commit a1447a7 into datawhalechina:main Jun 8, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working | Bug 修复 documentation Improvements or additions to documentation | 文档更新 enhancement New feature or request | 新功能 model New model or model improvement | 模型相关

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants