Skip to content

feat(scan): classifier threshold, filtering, CycloneDX verdict and summary - #129

Merged
mscasso-scanoss merged 3 commits into
feat/snippet-classifierfrom
feat/snippet-classifier-options
Oct 29, 2026
Merged

mscasso-scanoss merged 3 commits into
feat/snippet-classifierfrom
feat/snippet-classifier-options

Conversation

@matiasdaloia

Copy link
Copy Markdown
Contributor

Stacked on #98 (base feat/snippet-classifier). Merge #98 first; this PR then retargets to main.

What

The options the snippet classifier needs to ship (scanoss/roadmap#14):

  • --classify-threshold <float>: P(false positive) at or above which a match is proposed as a false positive. Default 0.7, strictly between 0 and 1, only valid with --classify-snippets.
  • --classify-filter: removes snippet matches proposed as false positives, and components left with no evidence.
    • Advisories that only pointed at removed components go too, so no output keeps a dangling reference.
    • File matches, dependency evidence, unscored snippets and matches claimed by a bom.identify rule are never removed.
    • Filtering runs before enrichment, so removed components never reach the vulnerability or license services.
  • Every output format. --classify-snippets is no longer raw-only.
    • CycloneDX: occurrences have no properties, so the verdict follows the ranges in additionalContext: input lines 19-405; oss lines 19-405; classifier real p=0.00 model=lr-d51f2e100211.
    • SPDX-lite: it has no per-file data, so it carries no verdict. Filtering still applies.
  • A summary on stderr, e.g. Snippet classifier (model lr-d51f2e100211, threshold 0.7): scored 54 of 54 snippet matches, 3 proposed as false positives, 3 removed. It warns when matches could not be scored, and says so when file contents answer 401/403. Before this, those failures were only logged.
  • The file-contents id is the scan's file_hash, a 16-hex CRC64 today, not an MD5. Only the wording changes.
  • Docs: CHANGELOG [Unreleased] (covering --classify-snippets too, which feat(scan): classify snippet matches as false positives (opt-in) #98 had no entry for), README and CLIENT_HELP.

Why the filtering guidance says 0.9

On 8 public application repos (876 snippet matches, blind stratified review of 109) the released model gives FP precision 0.71 and recall 0.90 at 0.7; at 0.9 it gives 0.79 and 0.83. Details are in the comment on #98. 0.7 is fine for proposing; filtering drops some real matches, so the docs advise a higher threshold and say filtering never replaces review.

Testing

  • make fmt-check, make vet, make test: pass. go test -race on snippetmatch, scanpipeline, sbom and cmd: pass.
  • make lint was not run locally: the installed golangci-lint (built with go1.25) refuses the go1.27 toolchain. CI runs it.
  • New tests cover:
    • threshold validation and flag dependencies;
    • filtering: evidence removed, empty components removed, advisories pruned, file, unscored and identified matches untouched;
    • the CycloneDX context text and the summary counts.
  • Each filtering guard was broken on purpose to confirm a test fails.
  • Live against api.scanoss.com, on TheAlgorithms/Java (54 snippet matches):
    • --classify-filter: 3 proposed and 3 removed, no false-positive verdict left in the raw output.
    • -f cyclonedx: verdict present in additionalContext.
    • --classify-threshold without --classify-snippets, and a threshold of 1.5, are both rejected before upload.

/v3/file-contents/{id} takes whatever file_hash the scan reported for the
matched OSS file, today a 16-hex CRC64. The comments and parameter names
said MD5.
…oneDX

Inventory.RemoveFalsePositiveSnippets drops the snippet matches the
classifier proposed as false positives and any component left with no
evidence. It never touches file matches, declared or transitive evidence,
unscored or real snippets, or a match a bom.identify rule claimed, so the
dependency graph keeps every component it names. An advisory loses the
PURLs that named only removed components, and goes when none is left, so
CycloneDX never emits a vulnerability that affects nothing.

CycloneDX occurrences have no properties, so a scored snippet appends
'classifier <verdict> p=<0.00> model=<version>' to additionalContext,
after the ranges.
…summary

--classify-snippets now works with every output format. --classify-threshold
(default 0.7, strictly between 0 and 1) and --classify-filter need
--classify-snippets, and a bad value is refused before any upload.
Classification runs before enrichment, so a component the filter removes
is never sent to the decoration services.

After the scan, stderr gets one line saying how many snippet matches were
scored, proposed as false positives, removed and left unscored, and when
the API refused file contents with 401/403, so a run that scored nothing
no longer looks like a run that found nothing.

Docs: CHANGELOG (covering --classify-snippets too), CLIENT_HELP and README.
@mscasso-scanoss mscasso-scanoss self-assigned this Oct 29, 2026
@mscasso-scanoss
mscasso-scanoss merged commit 8ec29c6 into feat/snippet-classifier Oct 29, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants