Skip to content
neoandmatrixPublic

About

Create and store persistent memories.

Resources

Stars

2 stars

Watchers

0 watching

Forks

Latest commit

 

History

43 Commits

Folders and files

Repository files navigation

AnyMem

Your second brain. Store, search, and learn from your memories.

AnyMem is a memory system that lets you save text, URLs, images, and tweets — then search them semantically, quiz yourself, and build learning paths. Built on Cloudflare Workers, D1, Vectorize, R2, and Workers AI.

Important: This project requires your own Cloudflare deployment. You must have a Cloudflare account with the R2 subscription enabled (free plan is sufficient). Images are stored in R2, and the API runs on Cloudflare Workers. Follow the setup steps below to deploy your own instance.

Architecture

┌─────────────────────────────────────────────────────────┐
│                     Clients                             │
│  ┌──────────┐  ┌──────────┐  ┌────────────────────────┐ │
│  │ Android  │  │   TUI    │  │   Chrome Extension     │ │
│  │ + Linux  │  │  (Go)    │  │  Bookmarks / YT / X    │ │
│  │ (Flutter)│  │          │  │  Anti-doomscroll        │ │
│  └────┬─────┘  └────┬─────┘  └──────────┬─────────────┘ │
└───────┼──────────────┼──────────────────┼───────────────┘
        │              │                  │
        └──────────────┼──────────────────┘
                       ▼
        ┌──────────────────────────────┐
        │     Cloudflare Workers API   │
        │  ┌────────┐  ┌───────────┐  │
        │  │ Memory  │  │  Learn    │  │
        │  │ CRUD    │  │  System   │  │
        │  │ Query   │  │  SM-2     │  │
        │  │ Ask     │  │  Quiz     │  │
        │  └────┬───┬┘  └───┬──────┘  │
        └───────┼───┼───────┼─────────┘
                │   │       │
    ┌───────────┼───┼───────┼──────────────┐
    │           ▼   ▼       ▼              │
    │  ┌────┐ ┌─────────┐ ┌────┐ ┌─────┐  │
    │  │ D1 │ │Vectorize│ │ R2 │ │ AI  │  │
    │  │SQL │ │ Vectors │ │Imgs│ │LLM  │  │
    │  └────┘ └─────────┘ └────┘ └─────┘  │
    │         Cloudflare Platform          │
    └──────────────────────────────────────┘

Project Structure

anymem/
├── apps/
│   ├── api/                  # Cloudflare Workers API
│   │   ├── src/
│   │   │   ├── routes/       # memory.ts, learn.ts, chat.ts, progress.ts
│   │   │   ├── memory/       # engine, url/image processors, quiz, SM-2
│   │   │   ├── db/           # D1 queries, FTS5, learn queries
│   │   │   └── embeddings/   # Workers AI embedding
│   │   ├── schema.sql        # D1 schema v1
│   │   ├── schema-v2.sql     # Learning system tables
│   │   ├── schema-v3.sql     # Learnable flag
│   │   ├── schema-v4.sql     # FTS5 full-text search
│   │   ├── schema-v5.sql     # Chat, progress notes, learning status
│   │   └── wrangler.jsonc    # Cloudflare config
│   │
│   ├── android/              # Flutter app (Android + Linux)
│   │   ├── lib/
│   │   │   ├── screens/      # Search, Memories, Add, Learn, Chat
│   │   │   ├── services/     # API client, providers, share receiver
│   │   │   ├── models/       # Data models
│   │   │   └── widgets/      # Shared UI components
│   │   └── linux/            # Linux desktop packaging (RPM)
│   │
│   ├── tui/                  # Go terminal UI (Bubble Tea)
│   │   └── internal/
│   │       ├── api/          # Go API client
│   │       └── ui/           # TUI model + views
│   │
│   └── chrome-extension/     # Chrome extension
│       ├── background.js     # Bookmark/YouTube/Twitter import
│       ├── content-redirect.js # Anti-doomscroll overlay
│       ├── page.html/js      # Full-tab import UI
│       └── manifest.json
│
├── packages/                 # Shared configs (TS, ESLint)
└── turbo.json

Prerequisites

Tool Version What for
Bun 1.0+ API dependencies, Turborepo
Node.js 18+ Wrangler CLI
Go 1.21+ TUI client
Flutter 3.x Android + Linux app
Cloudflare account - Workers, D1, Vectorize, R2, AI

Setup from scratch

1. Clone and install dependencies

git clone <repo-url>
cd anymem
bun install

2. Cloudflare account setup

Before creating resources, you need to set up your Cloudflare account:

2a. Enable R2 (required for image storage)

  1. Go to Cloudflare Dashboard → R2 Object Storage
  2. Click Purchase R2 or Subscribe to R2 (the free plan includes 10GB storage + 1M Class A ops/month)
  3. R2 must be enabled before you can create buckets

2b. Find your Account ID

  1. Go to Cloudflare Dashboard
  2. Click on any domain, or go to Workers & Pages
  3. Your Account ID is in the right sidebar
  4. Copy it — you'll need it for wrangler.jsonc and API token URLs

2c. Create a Cloudflare API Token (for Browser Rendering + Workers AI)

This token is used by the API to call Browser Rendering (for URL content extraction) and Workers AI vision model (for image description).

  1. Go to Cloudflare Dashboard → My Profile → API Tokens
  2. Click Create Token
  3. Select Create Custom Token
  4. Configure permissions:
    • Account → Workers AI → Edit
    • Account → Cloudflare Browser Rendering → Edit
  5. Under Account Resources, select Include → your account
  6. Click Continue to Summary → Create Token
  7. Copy the token immediately — it's only shown once
  8. This token will be set as the CF_API_TOKEN secret later

2d. Activate the Vision Model (required for image uploads)

The Workers AI vision model (@cf/meta/llama-3.2-11b-vision-instruct) needs to be activated before first use. Make a single request to initialize it:

curl -X POST 'https://api.cloudflare.com/client/v4/accounts/YOUR_ACCOUNT_ID/ai/run/@cf/meta/llama-3.2-11b-vision-instruct' \
  --header 'Authorization: Bearer YOUR_CF_API_TOKEN' \
  --header 'Content-Type: application/json' \
  --data '{"prompt": "agree"}'

Replace YOUR_ACCOUNT_ID with your Cloudflare Account ID and YOUR_CF_API_TOKEN with the token you created in step 2c. You should get a JSON response with a "response" field — this confirms the model is active.

3. Create Cloudflare resources

cd apps/api

# Create D1 database
bunx wrangler d1 create anymemory-db
# Copy the database_id into wrangler.jsonc

# Create Vectorize index (768 dimensions for bge-base)
bunx wrangler vectorize create anymemory-vectors \
  --dimensions=768 --metric=cosine
# Copy the index name into wrangler.jsonc

# Create R2 bucket for images
bunx wrangler r2 bucket create anymem-images

4. Set environment variables and secrets

Variables (set in wrangler.jsonc under vars):

Edit apps/api/wrangler.jsonc and set:

"vars": {
  "EMBED_MODEL": "@cf/baai/bge-base-en-v1.5",
  "VEC_DIM": "768",
  "MIN_SCORE": "0.3",
  "VERSION": "1.0.0",
  "CF_ACCOUNT_ID": "YOUR_ACCOUNT_ID"  // ← set this
}

Secrets (set via CLI — these are encrypted and not stored in code):

cd apps/api

# Your API authentication token (clients send this as Bearer token)
bunx wrangler secret put API_KEY
# Enter a secure token when prompted (e.g. "my-secret-key")

# The Cloudflare API token you created in step 2c
bunx wrangler secret put CF_API_TOKEN
# Paste the token from step 2c when prompted

5. Run database migrations

cd apps/api

# Initial schema (memories, waypoints, embed logs)
bunx wrangler d1 execute anymemory-db --file=schema.sql

# Learning system (reviews, daily_stats, spaced repetition columns)
bunx wrangler d1 execute anymemory-db --file=schema-v2.sql

# Learnable flag on memories
bunx wrangler d1 execute anymemory-db --file=schema-v3.sql

# FTS5 full-text search + backfill existing memories
bunx wrangler d1 execute anymemory-db --file=schema-v4.sql

# Chat system, progress notes, learning status
bunx wrangler d1 execute anymemory-db --file=schema-v5.sql

6. Deploy the API

cd apps/api
bunx wrangler deploy

Your API will be available at https://anymemory.YOUR_SUBDOMAIN.workers.dev.

7. Backfill FTS (if you have existing memories)

curl -X POST 'https://your-worker.workers.dev/admin/backfill-fts' \
  -H 'Authorization: Bearer YOUR_API_KEY'

Running each client

API (development)

bun run api:dev
# Runs at http://localhost:8787

TUI

cd apps/tui
ANYMEM_API_URL=https://your-worker.workers.dev \
ANYMEM_API_KEY=your-key \
go run .

Flutter (Android)

cd apps/android
flutter pub get
flutter run                 # Connected Android device/emulator

Flutter (Linux)

cd apps/android
flutter pub get
flutter run -d linux        # Run directly

# Or build RPM:
flutter build linux --release

# Run flutter_distributor to set up the RPM structure
# (it will fail at the rpmbuild step - this is expected)
flutter_distributor release --name=dev --jobs=release-dev-linux-rpm

# Fix with custom spec and build the RPM manually
DIST_DIR="dist/1.0.0+1/anymem-1.0.0+1-linux_rpm/rpmbuild"
cp linux/packaging/rpm/anymem.spec "$DIST_DIR/SPECS/anymem.spec"
cp linux/packaging/rpm/anymem.png "$DIST_DIR/BUILD/anymem.png"
rpmbuild --define "_topdir $(pwd)/$DIST_DIR" \
  -bb "$(pwd)/$DIST_DIR/SPECS/anymem.spec"

# The RPM is at:
ls $DIST_DIR/RPMS/x86_64/*.rpm

# Install:
sudo rpm -i $DIST_DIR/RPMS/x86_64/anymem-*.rpm

Note: flutter_distributor's auto-generated RPM spec has a path bug (cp -r 'anymem/*' fails because rpmbuild runs from a different working directory). The custom anymem.spec uses %{_topdir}/BUILD/%{name}/* which resolves correctly. Always copy the custom spec before running rpmbuild.

Prerequisites for RPM build: patchelf and rpmbuild must be installed (sudo dnf install patchelf rpm-build).

Chrome Extension

  1. Open chrome://extensions
  2. Enable "Developer mode"
  3. Click "Load unpacked" → select apps/chrome-extension/
  4. Click the extension icon to open

API Endpoints

Memory

Method Endpoint Description
POST /memory/add Add text memory
POST /memory/add-url Add URL (auto-extracts content, YouTube oEmbed)
POST /memory/add-image Add image (multipart, vision + OCR)
POST /memory/query Semantic vector search
POST /memory/ask Search + LLM answer synthesis
GET /memory/all List/search memories (?search=, ?tag=, ?limit=, ?offset=)
GET /memory/image/:key Serve image from R2
DELETE /memory/:id Delete a memory (D1 + Vectorize + FTS)

Chat

Method Endpoint Description
POST /memory/:id/chat Send a message, get AI response (cross-memory context)
GET /memory/:id/conversations List conversations for a memory
GET /conversation/:id/messages Get conversation history
DELETE /conversation/:id Delete a conversation

Progress

Method Endpoint Description
POST /memory/:id/progress Add a progress note
GET /memory/:id/progress Get progress notes + learning status
PUT /memory/:id/progress/:noteId Update a note
DELETE /memory/:id/progress/:noteId Delete a note
PUT /memory/:id/status Set learning status

Learn

Method Endpoint Description
GET /learn/digest Get memories due for review (SM-2)
GET /learn/session Smart daily learning session (continue/related/exercise/review/discover)
POST /learn/review Record a review
POST /learn/quiz Generate a quiz question
POST /learn/quiz/answer Evaluate answer
GET /learn/stats Streak, progress, knowledge score
POST /learn/recommend Learning path from memory graph

Admin

Method Endpoint Description
POST /admin/backfill-fts Rebuild FTS index from existing memories

Features

Memory ingestion

  • Text: Direct content with LLM classification into 5 cognitive sectors
  • URLs: Browser Rendering → content extraction → heuristic classification (zero LLM)
  • YouTube: oEmbed metadata (zero LLM, zero Browser Rendering)
  • Images: R2 storage + vision model description + OCR (1 LLM call)
  • Learnable flag: Mark memories for the learning system at ingestion time

Search

  • Semantic search: Multi-sector vector embeddings via Vectorize
  • Full-text search: FTS5 keyword/phrase search on the Memories tab
  • LLM mode: Search + answer synthesis via llama-3.2-3b-instruct

Chat

  • Per-memory chat: Chat about any memory with qwen3-30b-a3b-fp8
  • Cross-memory context: Related memories automatically fetched and included
  • Prompt caching: x-session-affinity header for cached prefix across messages
  • Persistent conversations: Full history stored in D1

Learning system

  • Spaced repetition: SM-2 algorithm (same as Anki)
  • Smart daily sessions: Prioritizes continue > related > exercise > review > discover
  • Progressive quiz: Recall → Specific → Application difficulty
  • Progress notes: Track your learning journey per memory
  • Streak tracking: Daily stats, weekly chart, knowledge score
  • Anti-doomscroll: Chrome content script shows a memory instead of blocking

Chrome extension

  • Bookmark import: Full Chrome bookmark tree with folder-based tags
  • YouTube import: Google Takeout CSV playlists
  • Twitter/X import: GraphQL API bookmark fetch (uses session cookies)
  • Rate-limited, pausable, tracks already-imported

Environment variables

API (wrangler.jsonc vars)

Variable Description
EMBED_MODEL Embedding model (default: @cf/baai/bge-base-en-v1.5)
VEC_DIM Vector dimensions (default: 768)
MIN_SCORE Minimum composite search score (default: 0.3)
VERSION App version string
CF_ACCOUNT_ID Your Cloudflare account ID (required for Browser Rendering)

API (secrets via bunx wrangler secret put)

Secret Description
API_KEY Bearer token for client authentication
CF_API_TOKEN Cloudflare API token (needs Workers AI + Browser Rendering Edit permissions)

TUI (environment)

Variable Description
ANYMEM_API_URL API endpoint (default: http://localhost:8787)
ANYMEM_API_KEY Bearer token

Cloudflare services used

Service Free tier What for
Workers 100K req/day API hosting
D1 5M rows read/day Memory storage, FTS5, reviews
Vectorize 30M vector dimensions Semantic search embeddings
R2 10GB + 1M ops/month Image storage
Workers AI Various per model Embeddings, classification, chat, vision, quiz
Browser Rendering 1000 req/day URL content extraction

License

MIT

About

Create and store persistent memories.

Resources

Stars

2 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages