Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,13 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
`127.0.0.1:41721` (port configurable), optional LAN access (binds 0.0.0.0, password
required — browse your desktop library from a phone), and a read-only MCP endpoint
at `/mcp` (Streamable HTTP, bearer token with copy/regenerate in Settings).
- **Web links that survive re-indexing.** With the Local API on (or in the NAS web UI), Copy Web
Link / Copy Markdown with Thumbnail gives `http://<host>:<port>/open?hash=…&t=…`: it opens the
item in any browser on the network — phones and web-based notes included — and video jumps to
the timestamp; the Markdown form embeds a thumbnail (`/media/thumbh/<hash>`) so notes show a
preview. Like deep links they are addressed by content hash. Opening one while signed out goes
to the login page and back. MCP results now carry this page link as `httpUrl` (raw file:
`mediaUrl`).
- **MCP for AI clients.** Read-only tools `search_media` / `get_media` / `find_similar`
(results carry scores, matched video time ranges, `vixel://` deep links and http media
links). Available three ways: desktop Local API `/mcp`, server edition `/mcp`
Expand Down
2 changes: 1 addition & 1 deletion README.en.md
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,7 @@ Local AI search for photos, videos and audio. Fully offline, no account, nothing
- **Clean up similar photos.** Finds burst shots, re-exports and lightly edited copies, suggests which one to keep, and hides the rest in Vixel or moves them to the Trash after you confirm.
- **Offline map, similar photos, de-duplication, HEIC / RAW, quick look, dark mode.**
- **CLI for scripts and agents:** `vixel search "sunset at the beach" --json`.
- **Links for your notes.** Right-click → Copy Link / Copy Markdown Link gives a `vixel://` deep link to paste into Obsidian, Notes or any editor; clicking it opens Vixel at that item (video/audio jumps to the timestamp). Links are addressed by content hash, so they survive re-indexing.
- **Links for your notes.** Right-click → Copy Link / Copy Markdown Link gives a `vixel://` deep link to paste into Obsidian, Notes or any editor; clicking it opens Vixel at that item (video/audio jumps to the timestamp). Links are addressed by content hash, so they survive re-indexing. With the Local API on you can also Copy Web Link / Copy Markdown with Thumbnail: an `http://…/open?…` link that opens on phones and in web-based notes, with an inline preview in Markdown (the NAS web UI offers the same).
- **Local API (off by default).** Settings → General → Local API serves a web UI and HTTP API for your desktop library — enable LAN access (password required) to browse it from your phone — plus a read-only MCP endpoint at `/mcp` for AI clients (see below).

<img src="docs/media/video.jpg" width="860" alt="Video segment search" />
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,7 @@ SCRFD 人脸检测 + MobileFaceNet 512 维特征 + sqlite-vec 近邻索引。
- **HEIC / RAW** —— macOS 上通过系统 `sips` 解码 CR2 / CR3 / NEF / ARW / DNG / RAF / ORF / RW2,保留 EXIF。
- **快速查看** —— 空格预览,⌘1–5 切换 全部 / 图片 / 视频 / 音频 / 地图。
- **命令行** —— `vixel search "海边" --json`,方便脚本和 Agent 调用(见下文);结果里的 `matchedBy` 说明是画面、图中文字还是文件名命中。
- **链接进笔记** —— 右键「拷贝链接 / 拷贝 Markdown 链接」得到 `vixel://` 深链接,贴进 Obsidian、备忘录等任意笔记,点击回到 Vixel 定位到那一项(视频 / 音频直接跳到时间点)。链接按内容哈希定位,清库重建、移动文件后仍然有效。
- **链接进笔记** —— 右键「拷贝链接 / 拷贝 Markdown 链接」得到 `vixel://` 深链接,贴进 Obsidian、备忘录等任意笔记,点击回到 Vixel 定位到那一项(视频 / 音频直接跳到时间点)。链接按内容哈希定位,清库重建、移动文件后仍然有效。打开了「本地接口」时还能「拷贝网页链接 / 拷贝 Markdown(带缩略图)」:`http://…/open?…` 在手机、网页笔记等装不了 Vixel 的地方也能点开,Markdown 里直接显示缩略图;NAS 版网页里同样能拷贝。
- **本地接口(默认关闭)** —— 设置 → 通用 → 本地接口:在本机开出网页界面和 HTTP API,手机浏览器也能访问桌面的资料库(局域网访问必须设口令);同一服务的 `/mcp` 是给 AI 客户端的只读 MCP 接口(见下文)。
- **中英文界面** —— 跟随系统语言,也可在 设置 → 通用 里切换。
- **诊断** —— `vixel doctor` / `vixel bench` 体检和测速,`VIXEL_PROFILE=1` 打印慢操作,见 [docs/debugging.md](docs/debugging.md)。
Expand Down
4 changes: 2 additions & 2 deletions docs/server/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -125,7 +125,7 @@ volumes:

设置 `VIXEL_MCP_TOKEN` 后,服务器在 `/mcp` 提供一个 [MCP](https://modelcontextprotocol.io)
端点(Streamable HTTP),AI 客户端 / Agent 可以用自然语言查询媒体库。工具**全部只读**:
`search_media`(语义 + 图内文字 + 文件名搜索,命中视频时间段,带 vixel:// 深链接和 http 媒体链接)、
`search_media`(语义 + 图内文字 + 文件名搜索,命中视频时间段,带 vixel:// 深链接、网页查看链接 httpUrl(/open?…,按内容哈希定位)和原文件链接 mediaUrl)、
`get_media`(单项详情:OCR 文字、标签、人物等)、`find_similar`(找相似)。

```yaml
Expand Down Expand Up @@ -283,7 +283,7 @@ search instant. Lower `VIXEL_THREADS` (e.g. `2`) for a gentler, slower index.
With `VIXEL_MCP_TOKEN` set, the server exposes an [MCP](https://modelcontextprotocol.io)
endpoint (Streamable HTTP) at `/mcp`, so AI clients and agents can query the library in
natural language. All tools are **read-only**: `search_media` (semantic + text-in-image +
filename search, with matched video time ranges, a vixel:// deep link and an http media link),
filename search, with matched video time ranges, a vixel:// deep link, a web page link `httpUrl` (/open?…, addressed by content hash) and the raw file `mediaUrl`),
`get_media` (details of one item: OCR text, tags, people, …) and `find_similar`.

```yaml
Expand Down
6 changes: 4 additions & 2 deletions src/core/mcp/server.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -79,13 +79,15 @@ describe('MCP server core', () => {
expect(items[0]).toMatchObject({
id: 1, name: 'beach.jpg', type: 'image',
link: 'vixel://open?hash=aa11bb22cc33dd44&id=1',
httpUrl: 'http://127.0.0.1:41721/media/image/1',
httpUrl: 'http://127.0.0.1:41721/open?hash=aa11bb22cc33dd44&id=1',
mediaUrl: 'http://127.0.0.1:41721/media/image/1',
})
expect(items[1]).toMatchObject({
videoId: 7,
timeRange: { startSec: 32, endSec: 64 },
link: 'vixel://open?hash=deadbeef00112233&video=7&t=32',
httpUrl: 'http://127.0.0.1:41721/media/media/7',
httpUrl: 'http://127.0.0.1:41721/open?hash=deadbeef00112233&video=7&t=32',
mediaUrl: 'http://127.0.0.1:41721/media/media/7',
})
})

Expand Down
22 changes: 15 additions & 7 deletions src/core/mcp/tools.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
import type { DatabaseInstance } from '../db'
import type { SearchEngine } from '../search'
import { TagStore } from '../tagStore'
import { deepLinkForItem } from '../../shared/deeplink'
import { deepLinkForItem, httpLinkForItem } from '../../shared/deeplink'
import type { MediaKind, Photo, SearchResult } from '../../shared/types'

export interface McpToolContext {
Expand Down Expand Up @@ -93,11 +93,19 @@ const posInt = (v: unknown): number | undefined => {
return Number.isInteger(n) && n > 0 ? n : undefined
}

/** http 媒体链接:图片用原图路由,音视频用源文件路由(支持 Range),浏览器里打开仍需登录 */
function httpUrlFor(photo: Photo, base: string | null | undefined): string | undefined {
if (!base) return undefined
/**
* http 链接(有本地接口 / 服务器地址时才带):
* httpUrl —— 网页查看页 /open?…(按内容 hash 定位,活过重建索引;音视频跳到时间点),给人点、贴进笔记
* mediaUrl —— 原文件(图片原图 / 音视频源文件,支持 Range),给要下载 / 处理文件的 Agent
* 浏览器里打开仍需登录(有口令时)
*/
function httpLinks(photo: Photo, base: string | null | undefined, timeMs?: number | null): Record<string, string> {
if (!base) return {}
const root = base.replace(/\/$/, '')
return photo.videoId != null ? `${root}/media/media/${photo.videoId}` : `${root}/media/image/${photo.id}`
return {
httpUrl: httpLinkForItem(root, photo, timeMs),
mediaUrl: photo.videoId != null ? `${root}/media/media/${photo.videoId}` : `${root}/media/image/${photo.id}`,
}
}

function resultItem(ctx: McpToolContext, r: SearchResult, extra: McpCallExtra): Record<string, unknown> {
Expand All @@ -117,7 +125,7 @@ function resultItem(ctx: McpToolContext, r: SearchResult, extra: McpCallExtra):
...(r.segment ? { timeRange: { startSec: r.segment.startMs / 1000, endSec: r.segment.endMs / 1000 } } : {}),
...(p.takenAt ? { takenAt: p.takenAt } : {}),
link: deepLinkForItem(p, r.segment?.startMs),
...(httpUrlFor(p, extra.httpBase) ? { httpUrl: httpUrlFor(p, extra.httpBase) } : {}),
...httpLinks(p, extra.httpBase, r.segment?.startMs),
}
}

Expand Down Expand Up @@ -176,7 +184,7 @@ function getMedia(ctx: McpToolContext, args: Record<string, unknown>, extra: Mcp
favorite: !!photo.favorite,
...(media ? { segments: media.segments.map((s) => ({ startSec: s.startMs / 1000, endSec: s.endMs / 1000 })) } : {}),
link: deepLinkForItem(photo),
...(httpUrlFor(photo, extra.httpBase) ? { httpUrl: httpUrlFor(photo, extra.httpBase) } : {}),
...httpLinks(photo, extra.httpBase),
}
}

Expand Down
12 changes: 6 additions & 6 deletions src/main/deeplink.ts
Original file line number Diff line number Diff line change
Expand Up @@ -20,7 +20,7 @@ import { t } from '../shared/i18n'

export interface DeepLinkDeps {
/** 按 target 查库拼出结果(含命中片段);找不到返回 null */
resolve(target: DeepLinkTarget): SearchResult | null
resolve(target: DeepLinkTarget): Promise<SearchResult | null>
/** 在主窗口打开(quickPanel.openResult:窗口不存在时创建,加载完前暂存) */
open(result: SearchResult): void
/** 链接指向的内容不在库里:把主窗口带到前台再说明 */
Expand All @@ -34,7 +34,7 @@ function isDeepLink(arg: string): boolean {
return typeof arg === 'string' && arg.startsWith('vixel://')
}

function handle(url: string): void {
async function handle(url: string): Promise<void> {
if (!deps) {
pendingUrls.push(url)
return
Expand All @@ -44,7 +44,7 @@ function handle(url: string): void {
console.warn(`[deeplink] ignoring malformed link: ${url}`)
return
}
const result = deps.resolve(target)
const result = await deps.resolve(target)
if (result) {
console.log(`[deeplink] opening ${url}`)
deps.open(result)
Expand All @@ -64,15 +64,15 @@ export function initDeepLinks(): boolean {
if (process.platform === 'darwin') {
app.on('open-url', (event, url) => {
event.preventDefault()
if (isDeepLink(url)) handle(url)
if (isDeepLink(url)) void handle(url)
})
return true
}
// Windows / Linux:深链接到达的是一个新实例的 argv
if (!app.requestSingleInstanceLock()) return false
app.on('second-instance', (_event, argv) => {
const url = argv.find(isDeepLink)
if (url) handle(url)
if (url) void handle(url)
else deps?.ensureMainWindow()
})
return true
Expand All @@ -94,5 +94,5 @@ export function activateDeepLinks(d: DeepLinkDeps): void {
if (argvUrl && !pendingUrls.includes(argvUrl)) pendingUrls.push(argvUrl)
const queued = pendingUrls
pendingUrls = []
for (const url of queued) handle(url)
for (const url of queued) void handle(url)
}
31 changes: 5 additions & 26 deletions src/main/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -34,10 +34,10 @@ import { setInferenceTransport } from '../core/inference/transport'
import { initDeepLinks, activateDeepLinks } from './deeplink'
import { LocalApiServer } from './localApi'
import { LOCALAPI_CHANNELS, type LocalApiPatch } from '../shared/localapi'
import type { DeepLinkTarget } from '../shared/deeplink'
import { createServices, META_KEY_INDEX_ON_BATTERY, type VixelServices } from '../services/api'
import { createServiceEvents } from '../services/events'
import { IPC_CHANNELS, type MediaFilters, type ThemeMode, type IndexProgress, type EmbeddingConfig, type ImageQueryInput, type SearchResult } from '../shared/types'
import type { DeepLinkTarget } from '../shared/deeplink'
import { IPC_CHANNELS, type MediaFilters, type ThemeMode, type IndexProgress, type EmbeddingConfig, type ImageQueryInput } from '../shared/types'
import { TAG_CHANNELS } from '../shared/tags'
import { AUTOTAG_CHANNELS } from '../shared/autotags'
import { CLEANUP_CHANNELS } from '../shared/cleanup'
Expand Down Expand Up @@ -203,28 +203,6 @@ function ensureMainWindow(): BrowserWindow {
return win
}

/**
* 深链接 → 结果:hash(内容哈希,清库重建 / 移动文件后仍有效)优先,video / photo id 兜底。
* 时间点尽量落到包含它的已索引片段(详情页时间轴整段高亮),不在任何片段里就按原时间点 seek。
*/
function resolveDeepLinkTarget(target: DeepLinkTarget): SearchResult | null {
let photo = target.hash ? db.getRepresentativeByHash(target.hash) : undefined
if (!photo && target.videoId != null) {
const video = db.getVideoById(target.videoId)
if (video) photo = db.getRepresentativeByHash(video.fileHash)
}
if (!photo && target.photoId != null) photo = db.getPhoto(target.photoId)
if (!photo) return null
const result: SearchResult = { photo, score: 1 }
if (target.timeMs != null && photo.videoId != null) {
const ms = target.timeMs
// 正好是某个已索引片段的起点("拷贝链接"产出的就是)→ 用整段,详情页时间轴能整段高亮;
// 任意时间点 → 起点保持精确(播放器按 startMs seek),终点借所在片段的
const seg = db.getMediaDetail(photo.videoId)?.segments.find((s) => ms >= s.startMs && ms < s.endMs)
result.segment = seg && seg.startMs === ms ? seg : { startMs: ms, endMs: Math.max(ms, seg?.endMs ?? ms) }
}
return result
}

/**
* Windows / Linux:最后一个"真窗口"(主窗口 / 设置)关掉后,若启用了快速搜索快捷键,
Expand Down Expand Up @@ -533,6 +511,7 @@ function registerIpcHandlers(): void {
handle(IPC_CHANNELS.CLEAR_FAILED_TASKS, async () => api.clearFailedTasks())

handle(IPC_CHANNELS.GET_MEDIA_DETAIL, async (_event, videoId: number) => api.getMediaDetail(videoId))
handle(IPC_CHANNELS.RESOLVE_LINK, async (_event, target: DeepLinkTarget) => api.resolveLink(target))
handle(IPC_CHANNELS.GET_PHOTO_DETAIL, async (_event, photoId: number) => api.getPhotoDetail(photoId))
handle(IPC_CHANNELS.GET_THUMBNAIL, async (_event, photoId: number) => api.getThumbnail(photoId))
handle(IPC_CHANNELS.GET_PHOTO_LOCATIONS, async (_event, photoId: number) => api.getPhotoLocations(photoId))
Expand All @@ -549,7 +528,7 @@ function registerIpcHandlers(): void {
return win ? popupSmartAlbumMenu(win, String(name ?? '')) : null
})

handle(IPC_CHANNELS.SHOW_ITEM_MENU, async (event, opts: { count: number; isMedia: boolean; favorite: boolean; canImageSearch: boolean }) => {
handle(IPC_CHANNELS.SHOW_ITEM_MENU, async (event, opts: { count: number; isMedia: boolean; favorite: boolean; canImageSearch: boolean; webLinks?: boolean }) => {
const win = BrowserWindow.fromWebContents(event.sender)
return win ? popupItemMenu(win, opts) : null
})
Expand Down Expand Up @@ -822,7 +801,7 @@ app.whenReady().then(async () => {
createWindow()
// 深链接:服务和窗口通道就绪,开始处理(启动期间收到的链接在 deeplink.ts 的队列里等到这一刻)
activateDeepLinks({
resolve: resolveDeepLinkTarget,
resolve: (target) => services.api.resolveLink(target),
open: (r) => quickPanel!.openResult(r),
ensureMainWindow,
})
Expand Down
7 changes: 6 additions & 1 deletion src/main/menu.ts
Original file line number Diff line number Diff line change
Expand Up @@ -129,7 +129,7 @@ export function installAppMenu(
/** 网格项右键菜单:动作由渲染进程执行(它知道选中了哪些项),这里只负责弹出并回传选择 */
export function popupItemMenu(
win: BrowserWindow,
{ count, isMedia, favorite, canImageSearch }: { count: number; isMedia: boolean; favorite: boolean; canImageSearch: boolean }
{ count, isMedia, favorite, canImageSearch, webLinks }: { count: number; isMedia: boolean; favorite: boolean; canImageSearch: boolean; webLinks?: boolean }
): Promise<ItemMenuAction | null> {
return new Promise((resolve) => {
const pick = (action: ItemMenuAction) => (): void => resolve(action)
Expand All @@ -146,6 +146,11 @@ export function popupItemMenu(
...(multi ? [] : [
{ label: t.menu.copyLink, click: pick('copy-link') },
{ label: t.menu.copyMarkdownLink, click: pick('copy-md-link') },
// 网页链接(http://…/open?…):本地接口开着时才有 —— 手机、网页笔记等装不了 Vixel 的地方也能点开
...(webLinks ? [
{ label: t.menu.copyWebLink, click: pick('copy-web-link') },
{ label: t.menu.copyWebMarkdown, click: pick('copy-web-md') },
] : []),
] as MenuItemConstructorOptions[]),
{ label: favorite ? t.organize.unfavorite : t.organize.favorite, accelerator: 'F', registerAccelerator: false, click: pick('toggle-favorite') },
...(multi ? [] : [
Expand Down
7 changes: 6 additions & 1 deletion src/preload/index.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
import { contextBridge, ipcRenderer, webUtils } from 'electron'
import { IPC_CHANNELS, type MediaFilters, type SmartAlbum, type SmartAlbumMenuAction, type SearchResult, type WatchedFolder, type PhotoDetail, type PhotoLocation, type IndexProgress, type ModelStatus, type EmbeddingConfig, type Person, type FaceRecord, type BackupStatus, type MediaKind, type TaskOverview, type MediaDetail, type LibraryCounts, type MenuCommand, type ItemMenuAction, type ThemeMode, type PersonSuggestion, type PersonFace, type ImageQueryInput, type ImageSearchResponse, type SkippedFolder, type VoiceStatus, type VoiceTranscribeResult } from '../shared/types'
import type { Locale, LocaleSetting } from '../shared/i18n'
import type { DeepLinkTarget } from '../shared/deeplink'
import { CLEANUP_CHANNELS, type DupScanResult, type DupScanProgress, type HiddenItem, type TrashResult } from '../shared/cleanup'
import { COLLECT_CHANNELS, type CollectResult, type CollectServerStatus, type InboxInfo, type ItemSource } from '../shared/collect'
import { TAG_CHANNELS, type Tag, type TagMenuAction, type TagsForSelection } from '../shared/tags'
Expand Down Expand Up @@ -76,7 +77,7 @@ const api = {
}
},
/** 弹出原生右键菜单,resolve 为选中的动作;菜单关闭未选则为 null */
showItemMenu: (opts: { count: number; isMedia: boolean; favorite: boolean; canImageSearch: boolean }): Promise<ItemMenuAction | null> => {
showItemMenu: (opts: { count: number; isMedia: boolean; favorite: boolean; canImageSearch: boolean; webLinks?: boolean }): Promise<ItemMenuAction | null> => {
return ipcRenderer.invoke(IPC_CHANNELS.SHOW_ITEM_MENU, opts)
},
/** 收藏 / 取消收藏(按内容 hash,同内容的副本一起变) */
Expand Down Expand Up @@ -191,6 +192,10 @@ const api = {
getMediaDetail: (videoId: number): Promise<MediaDetail | null> => {
return ipcRenderer.invoke(IPC_CHANNELS.GET_MEDIA_DETAIL, videoId)
},
/** 链接(vixel://open / 网页 /open 的定位参数)→ 结果;找不到返回 null */
resolveLink: (target: DeepLinkTarget): Promise<SearchResult | null> => {
return ipcRenderer.invoke(IPC_CHANNELS.RESOLVE_LINK, target)
},

// 系统 / 模型状态
getAppPath: (): Promise<string> => {
Expand Down
Loading
Loading