Skip to content

Add evaluation guide, dataset and model cards; check committed annotations in CI - #26

Merged
professor3333 merged 1 commit into
mainfrom
docs/evaluation-cards
Nov 4, 2026
Merged

professor3333 merged 1 commit into
mainfrom
docs/evaluation-cards

Conversation

@professor3333

Copy link
Copy Markdown
Owner
  • docs/evaluation.md, docs/dataset_card.md, docs/model_card.md (measured counts only; no release model yet).
  • New CI test over every committed annotation record: schema, inventory, ids, overlaps, groups, status, text hash vs sample, snapshot commit, preprocessing version. Test-split records are checked for provenance only. Adds text-free validate_structure.
  • docs/test-access-log.md created (no entries) so existing links resolve.
  • README links the new docs and summarises the review policy.

…tions in CI

- docs/evaluation.md: correctness, metric definitions, intervals, paired
  comparison, split access and gold-origin rules, review policy.
- docs/dataset_card.md and docs/model_card.md with measured counts only;
  no release model exists yet.
- tests/test_committed_annotations.py: every committed record parses,
  matches its sample's text hash and the pinned snapshot, and passes the
  new text-free validate_structure (test-split records: provenance only).
- docs/test-access-log.md created with no entries; README links the new
  documents and summarises the review policy.
@professor3333
professor3333 merged commit f136a1c into main Nov 4, 2026
2 checks passed
@professor3333
professor3333 deleted the docs/evaluation-cards branch November 4, 2026 04:00
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant