Repository navigation
feat(scan): classifier threshold, filtering, CycloneDX verdict and summary - #129
Merged
mscasso-scanoss merged 3 commits intoOct 29, 2026
Merged
Conversation
/v3/file-contents/{id} takes whatever file_hash the scan reported for the
matched OSS file, today a 16-hex CRC64. The comments and parameter names
said MD5.
…oneDX Inventory.RemoveFalsePositiveSnippets drops the snippet matches the classifier proposed as false positives and any component left with no evidence. It never touches file matches, declared or transitive evidence, unscored or real snippets, or a match a bom.identify rule claimed, so the dependency graph keeps every component it names. An advisory loses the PURLs that named only removed components, and goes when none is left, so CycloneDX never emits a vulnerability that affects nothing. CycloneDX occurrences have no properties, so a scored snippet appends 'classifier <verdict> p=<0.00> model=<version>' to additionalContext, after the ranges.
…summary --classify-snippets now works with every output format. --classify-threshold (default 0.7, strictly between 0 and 1) and --classify-filter need --classify-snippets, and a bad value is refused before any upload. Classification runs before enrichment, so a component the filter removes is never sent to the decoration services. After the scan, stderr gets one line saying how many snippet matches were scored, proposed as false positives, removed and left unscored, and when the API refused file contents with 401/403, so a run that scored nothing no longer looks like a run that found nothing. Docs: CHANGELOG (covering --classify-snippets too), CLIENT_HELP and README.
Merged
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stacked on #98 (base
feat/snippet-classifier). Merge #98 first; this PR then retargets tomain.What
The options the snippet classifier needs to ship (scanoss/roadmap#14):
--classify-threshold <float>: P(false positive) at or above which a match is proposed as a false positive. Default 0.7, strictly between 0 and 1, only valid with--classify-snippets.--classify-filter: removes snippet matches proposed as false positives, and components left with no evidence.bom.identifyrule are never removed.--classify-snippetsis no longer raw-only.additionalContext:input lines 19-405; oss lines 19-405; classifier real p=0.00 model=lr-d51f2e100211.Snippet classifier (model lr-d51f2e100211, threshold 0.7): scored 54 of 54 snippet matches, 3 proposed as false positives, 3 removed. It warns when matches could not be scored, and says so when file contents answer 401/403. Before this, those failures were only logged.file_hash, a 16-hex CRC64 today, not an MD5. Only the wording changes.[Unreleased](covering--classify-snippetstoo, which feat(scan): classify snippet matches as false positives (opt-in) #98 had no entry for), README and CLIENT_HELP.Why the filtering guidance says 0.9
On 8 public application repos (876 snippet matches, blind stratified review of 109) the released model gives FP precision 0.71 and recall 0.90 at 0.7; at 0.9 it gives 0.79 and 0.83. Details are in the comment on #98. 0.7 is fine for proposing; filtering drops some real matches, so the docs advise a higher threshold and say filtering never replaces review.
Testing
make fmt-check,make vet,make test: pass.go test -raceon snippetmatch, scanpipeline, sbom and cmd: pass.make lintwas not run locally: the installed golangci-lint (built with go1.25) refuses the go1.27 toolchain. CI runs it.--classify-filter: 3 proposed and 3 removed, no false-positive verdict left in the raw output.-f cyclonedx: verdict present inadditionalContext.--classify-thresholdwithout--classify-snippets, and a threshold of 1.5, are both rejected before upload.