Repository navigation
Image series ingestion - #138
miriyalamani wants to merge 23 commits into
Conversation
One row per unique file in files (keyed by sha256) holds md5, PDQ, semantic and private embeddings, and the essay text. archives holds investigation labels and archive_files links archives to files.
Add Shared essay, privacy and hash modules for safe matching, importable by plugins without depending on other plugins. Pins presidio-analyzer, presidio-anonymizer and the spaCy model so nothing is downloaded at runtime.
redact_text replaces names, emails, phone numbers and places etc. The spaCy model loads on first use.
create_essay describes an image with Ollama (gemma3:4b) in at most 55 words, without quoting visible text or naming people or places. create_redacted_essay chains it with PII redaction
- Cap redacted essays at 350 chars (~75 CLIP tokens) on a word boundary; files.essay stays VARCHAR(600) as headroom - Remove detected PII outright (Presidio redact) and collapse whitespace instead of inserting <PERSON>-style tags - Log each redacted essay in the backend during ingestion; the UI result still shows only counts
…e CRUD. dd a second-step task picker for Image Ingestion (ingest, delete, rename), fix delete archive foreign-key ordering, return a clear "Archive not found" response instead of 500, shorten the essay hash UI copy, and switch essay generation to qwen3-vl:4b-instruct.
… ingest persistence and essay column naming after review.
- Essay: generate one plain paragraph (people, objects, scene) instead of JSON facts; keep whole sentences within 60 words / 350 chars and cut an overlong first sentence so the essay is never empty. JSON took 22-44 s per image and dropped scene details; a paragraph takes ~2.5 s. - Ingest only JPEG, PNG, BMP and WebP; skip GIF and TIFF (Ollama rejects TIFF for the essay). Scan the folder in image_ingestion instead of importing image_similarity's private _collect_image_paths. - Allow renaming an archive to its own label. - backend.spec: bundle Presidio and spaCy en_core_web_lg for the desktop build. - Tests: archive delete with and without retained hashes (in-memory SQLite), same-label rename, supported-format filter, paragraph essays. - Docs: restructure the plugin README (tasks, requirements, demo image source, smoke test, UI walkthrough, unit tests); update app-info format list and essay description.
If it's not already, can we please make this maximum configurable? |
| fusepy = "^3.0.1" | ||
|
|
||
| rb-lib = { path = "src/rb-lib", develop = true } | ||
| rb-safe-matching-common = { path = "src/rb-safe-matching-common", develop = true } |
There was a problem hiding this comment.
why add new rb-safe-matching-common modules to rescuebox , its needed only for image_ingestion plugin ?
There was a problem hiding this comment.
Good point, it doesn't strictly need to be there. Image-ingestion already declares it, so Poetry installs it either way. I added it to match how rb-lib is listed, but I'm happy to drop it from the root to keep that list minimal.
Not yet. I can add a max essay words field on Ingest form with default 60. |
|
The following command froze my machine after thrashing for about 5-10m.
Claude is telling me that the problem is that my Ollama settings were defaulting to a higher context than was necessary for the essay. On one hand, manually changing the default in Ollama did the trick for me. But it would be better to set the context length in the code since we know that we don't need too much. Something like the following. ESSAY_NUM_CTX: Final[int] = 8192 # image is ≤ ~4.1k tokens + ~150 prompt + ≤ ~100 output
response = ollama.generate(
model=model, prompt=ESSAY_PROMPT, images=[image_path],
options={"num_ctx": ESSAY_NUM_CTX, "num_predict": 256},
) |
…eration. Route plugin and subtask Run clicks through _handle_select so Menu mode select_tool() works without a callback. Pass num_ctx and num_predict to ollama.generate for essays (PR review).
|
With Semantic set to Yes, ingest fails with |
- image-similarity README: add the curl command for clipseg-rd64-refined.onnx and note that the image-ingestion Semantic hash loads these models from the same onnx_models/ folder. - image-ingestion README: link the Semantic requirement to the image-similarity installation section.
Rename the image-ingestion plugin to image-matching and fold the rb-safe-matching-common package into it, so ingestion, hashing and the upcoming query live in one plugin with one shared database - Layout: image_matching/main.py registers the tasks; image_ingestion/ holds ingest, delete and rename; hashing/ holds hashers, essay and PII redaction; archive_store.py stays shared for the future query/. - Routes move from /image_ingestion/* to /image_matching/*; the plugin registry, chatbot menu, backend.spec, pyproject, pytest.ini and the demo images folder are updated to match. - Stop writing redacted essays to the backend log and drop the rb-safe-matching-common __main__ demo. ESSAY=1 smoke_test.sh now prints essays from the database. - README and app-info describe ingestion as implemented and query and upload through the Safe Match intermediary as not yet implemented.
- Name the plugin Image Ingestion in the chatbot menu, plugin metadata and plugin registry. - Move app-info.md into image_ingestion/ and load it from there in main.py and backend.spec. - Restructure app-info.md and the README as Image Matching, then its Ingestion and Query tasks, then Ingestion's subtasks; mark each as implemented or not yet implemented. - Describe what the plugin does instead of referring to the design document.
Add a "Max essay words" number input (default 60) to Ingest Images. The value sets the essay prompt and word limit for newly generated essays; the 350-character cap still applies. Stored essays are not regenerated when the value changes. Update the CLI parser, smoke test payloads, README, app-info, and unit tests.
Purpose and change
This PR adds the Image Matching plugin (
image-matching) and implements Ingestion as its first task (shown in the RescueBox UI as tool Image Ingestion). Image Match (local hash search and UMass essay compare) is described in the README but not implemented in this PR.files,archives,archive_files). Each distinct image is stored once by SHA-256; re-ingest only computes hashes that are still missing.image-similarity, and a redacted essay from Ollamaqwen3-vl:4b-instructwith Presidio (en_core_web_lg), limited by Max essay words (default 60) and 350 characters. Ollama requests use boundednum_ctx/num_predictto avoid excessive context allocation.src-tauri/demo/image-matching/from image-series-dataset (Wikimedia Commons), shared table definitions inrb.api.database(ESSAY_COLUMN_CHARS), a CLIPSeg download command in theimage-similarityREADME, and Presidio/spaCy bundling inbackend.specfor the desktop build.How to try it
Full walkthrough:
src/image-matching/README.md.poetry install, then./run_backend_server. MD5 and PDQ need no extra models. Semantic needs SigLIP2/CLIPSeg ONNX underimage-similarity(download commands in that README). Essay needs Ollama and the Presidio spaCy model (installed withimage-matching)../run_ui_server, open http://localhost:8080/chatbot, choose Menu → 11. Image Ingestion → Open.src-tauri/demo/image-matching/series_0013_Alexandria_Ocasio-Cortez_2019with Semantic and Essay set to No. ExpectArchive 'series_0013_Alexandria_Ocasio-Cortez_2019': 3 files.Run the same ingest again and expectLabel already in use.image_matchingtag (/image_matching/ingest_images,delete_archive,rename_archive)../src/image-matching/smoke_test.shshould end withPASS: ingest, duplicate label, rename, delete. For essays (Ollama + Docker forpsql), useESSAY=1 ./src/image-matching/smoke_test.sh.cd startup && docker compose downwhen you are done.Validation and review
a8ff568,poetry run pytest src/image-matching/testsreported 22 passed (local).ruff check src/image-matchingandblack --check src/image-matchingwere clean (local).ESSAY_COLUMN_CHARSforfiles.essay; per-file ingest commits; delete-with-retain and rename edge-case tests; removal of temporary essay logging (essays are not written to the backend log);backend.specbundling; demo licensing notes; explicit CLIPSegcurlinimage-similarityREADME.Documentation and figures
README.mdadds a sample analysis line and mentions image hash archives in the storage section.src/image-matching/README.mddocuments tasks, requirements, demo images, smoke test (ESSAY=1), UI scenarios, unit tests, and a Mermaid diagram for ingestion vs future Safe Match flow.image_ingestion/app-info.mdandsmoke_test.shsupport in-app copy and automation.Security and privacy audit
image_matching/hashing/essay.py,privacy.py,image_ingestion/tasks.py,archive_store.py,rb/api/database.py(archive_files.path,files.essay).archive_files.path.filesrows that no other archive still references.Compatibility and limitations
archive_files,archives, andfiles(and any dependent data you rely on).filesrows without a new archive until ingest is retried.en_core_web_lgand pullingqwen3-vl:4b-instruct(about 3 GB); the Windows PyInstaller build was not verified in this PR.pyproject.tomlandpoetry.lockmay overlap with branch 137.|and,, so labels or descriptions containing those characters cannot be passed reliably through the CLI.