Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
68 changes: 57 additions & 11 deletions ecosystem/msk-paste/api.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,23 @@ MSK Paste exposes a full JSON REST API. All features available in the web UI are

## Authentication

There is **no authentication**. All endpoints are public. Write operations (create / delete) are protected by rate limiting and, in the case of deletion, a one-time random delete token.
Creating a paste through the API needs an API key. Send it as a Bearer header:

```http
Authorization: Bearer mskp_...
```

Keys for MSK Paste start with `mskp_`. The operator issues them on the server, there is no sign-up and no self-service. The hosted instance at paste.msk-scripts.de does not issue keys to others; create pastes there through the website. On your own instance you create keys with the key script, see [Installation](installation.md#api-keys).

Without a key, `POST /api/pastes` only accepts requests from the site's own form and answers `401`:

```json
{ "error": "Creating pastes through the API requires an API key", "code": "api-key-required" }
```

A wrong or revoked key also gets `401`, with `"code": "api-key-invalid"`.

Everything else works without a key, as before: fetching a paste, unlocking it with `POST /api/pastes/:id/verify`, deleting it with its delete token, and `GET /api/stats`.

---

Expand All @@ -24,23 +40,25 @@ All errors share the same shape:
```json
{
"error": "Human-readable message",
"code": "api-key-required",
"details": {
"fieldName": ["validation error 1", "validation error 2"]
}
}
```

`details` is only present on `400 Bad Request` (validation failures).
`code` names the reason in a fixed, machine-readable form. It comes with the refusals of `POST /api/pastes` for the API key, the limits and the content checks, so scripts should check `code` rather than the message. `details` is only present on `400 Bad Request` (validation failures).

| Status | Meaning |
|---|---|
| `400` | Validation failed, see `details` |
| `401` | Wrong password (on `/verify`) |
| `401` | Missing or invalid API key (on create), wrong password (on `/verify`) |
| `403` | Password required (on `/api/pastes/:id`) |
| `404` | Paste does not exist, has expired, or was burned |
| `409` | Custom ID is already in use |
| `409` | Custom ID is already in use, or the same content was pasted within the last 24 hours (`code: "duplicate"`) |
| `413` | Content exceeds 1 MB |
| `429` | Rate limit exceeded, see `Retry-After` header |
| `422` | Content refused by the [content checks](#content-checks) |
| `429` | Rate limit exceeded, see [Rate limiting](#rate-limiting) |

---

Expand All @@ -50,6 +68,7 @@ All errors share the same shape:

```http
POST /api/pastes HTTP/1.1
Authorization: Bearer mskp_...
Content-Type: application/json

{
Expand Down Expand Up @@ -97,6 +116,7 @@ The `deleteToken` is the **only** way to delete the paste later. Save it. It is

```bash
curl -X POST https://paste.msk-scripts.de/api/pastes \
-H "Authorization: Bearer $MSK_PASTE_KEY" \
-H "Content-Type: application/json" \
-d '{
"content": "print(\"hello\")",
Expand All @@ -107,6 +127,16 @@ curl -X POST https://paste.msk-scripts.de/api/pastes \

---

## Content checks

Every new paste goes through the same checks, whether it comes from the web form or from an API key. A paste is refused when it

- contains an encoded file: an unbroken run of 16 KB of base64 or hex characters, or a paste of 64 KB or more made almost only of wrapped base64 (`422`, `code: "blob"`)
- is mostly links: 3 or more links that make up at least half of the text, or 3 or more BBCode `[url=` or HTML `<a href=` links (`422`, `code: "links"`)
- repeats content that was already pasted within the last 24 hours (`409`, `code: "duplicate"`)

---

## `GET /api/pastes/:id`: fetch a paste

### Behaviour
Expand Down Expand Up @@ -213,29 +243,39 @@ Returns anonymous aggregate numbers shown on the [/stats](https://paste.msk-scri

## Rate limiting

The create endpoint is limited to **10 requests per hour per IP hash** by default. When exceeded:
Requests with an API key and requests from the web form are limited separately:

| Sent by | Limit | `code` on `429` | Setting |
|---|---|---|---|
| API key | 120 pastes per hour per key | `key-limit` | `API_KEY_CREATE_PER_HOUR` |
| Wrong API keys | 10 per address in 10 minutes, then the address waits | `key-attempts` | fixed |
| Web form | 10 pastes per hour per IP hash | `rate-limit` | `RATE_LIMIT_CREATE_PER_HOUR` |
| Web form | 20 pastes and 5 MB in total per IP address and 24 hours | `daily-limit` | `DAILY_PASTES_PER_IP`, `DAILY_BYTES_PER_IP` |

Requests with a key are not held to the limits of the web form. When the hourly limit per IP hash is exceeded:

```http
HTTP/1.1 429 Too Many Requests
Retry-After: 1742
Content-Type: application/json

{ "error": "Rate limit exceeded. Please try again later." }
{ "error": "Rate limit exceeded", "code": "rate-limit" }
```

`Retry-After` is in seconds. Self-hosted instances can change the limit via `RATE_LIMIT_CREATE_PER_HOUR`.
`Retry-After` is in seconds. Self-hosted instances can change every limit with the settings in the table.

---

## Example: command-line uploader

A minimal Bash function to upload a file:
A minimal Bash function to upload a file. It reads your API key from `MSK_PASTE_KEY`:

```bash
mskpaste() {
local file="$1"
local lang="${2:-plaintext}"
curl -sS -X POST https://paste.msk-scripts.de/api/pastes \
-H "Authorization: Bearer $MSK_PASTE_KEY" \
-H "Content-Type: application/json" \
-d "$(jq -n \
--arg c "$(cat "$file")" \
Expand All @@ -244,7 +284,8 @@ mskpaste() {
| jq -r '.url'
}

# Usage:
# Usage (the key you got from the operator, e.g. set in ~/.bashrc):
export MSK_PASTE_KEY="mskp_..."
mskpaste script.lua lua
mskpaste server.log
```
Expand All @@ -256,7 +297,10 @@ mskpaste server.log
```js
const res = await fetch('https://paste.msk-scripts.de/api/pastes', {
method: 'POST',
headers: { 'Content-Type': 'application/json' },
headers: {
'Authorization': `Bearer ${process.env.MSK_PASTE_KEY}`,
'Content-Type': 'application/json',
},
body: JSON.stringify({
content: 'console.log("hi")',
language: 'javascript',
Expand All @@ -272,10 +316,12 @@ console.log(paste.url)
## Example: Python

```python
import os
import requests

r = requests.post(
"https://paste.msk-scripts.de/api/pastes",
headers={"Authorization": f"Bearer {os.environ['MSK_PASTE_KEY']}"},
json={
"content": "print('hi')",
"language": "python",
Expand Down
19 changes: 16 additions & 3 deletions ecosystem/msk-paste/faq.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ Yes. The hosted instance at [paste.msk-scripts.de](https://paste.msk-scripts.de)

### Do I need an account?

No. MSK Paste has no user system by design. The delete token you receive after creating a paste is what allows you to remove it later.
No. MSK Paste has no user system by design. The delete token you receive after creating a paste is what allows you to remove it later. Creating pastes through the API needs an API key, though, see below.

### Can I edit a paste after creating it?

Expand Down Expand Up @@ -45,11 +45,20 @@ No. Every paste is unlisted, knowing the ID is the only way to find a paste. The

### Where's my API key?

There is no API key. All endpoints are public. Write operations are protected by rate limiting (10 creates per hour per IP hash).
You get it from the operator. Creating pastes through the API needs a key starting with `mskp_`, and keys are only issued on the server, there is no sign-up or self-service. The hosted instance at paste.msk-scripts.de does not issue keys to others; create pastes there through the website. If you host MSK Paste yourself, create one with `sudo bash /opt/msk-paste/scripts/api-key.sh create "<label>"`, see [Installation](installation.md#api-keys). Fetching, unlocking and deleting with the delete token still work without a key.

### Why am I getting `429 Too Many Requests`?

You've hit the rate limit on `POST /api/pastes`. Check the `Retry-After` header for how many seconds to wait. If you're self-hosting and need a higher limit, change `RATE_LIMIT_CREATE_PER_HOUR` in `.env`.
You've hit a limit on `POST /api/pastes`. The `code` field in the response says which one, and the `Retry-After` header, where present, says how many seconds to wait:

- **With an API key:** 120 pastes per hour per key (`key-limit`)
- **From the web form:** 10 pastes per hour per IP hash (`rate-limit`), and 20 pastes and 5 MB in total per IP address and 24 hours (`daily-limit`)

If you're self-hosting and need higher limits, change `API_KEY_CREATE_PER_HOUR`, `RATE_LIMIT_CREATE_PER_HOUR`, `DAILY_PASTES_PER_IP` or `DAILY_BYTES_PER_IP` in `.env`.

### Why is my paste refused with `422` or `409`?

It failed a content check. Encoded files (long runs of base64 or hex), pastes that are mostly links, and the same content again within 24 hours are refused, from the form and from API keys alike. The `code` field says which check it was: `blob`, `links` or `duplicate`. Details on the [REST API page](api.md#content-checks).

### How do I delete a paste via the API?

Expand Down Expand Up @@ -118,6 +127,10 @@ openssl rand -hex 32

Paste it into `.env`, then restart the service.

### The form refuses to create pastes

In production the form needs Cloudflare Turnstile. Without `CAPTCHA_SITE_KEY` and `CAPTCHA_SECRET_KEY` in `.env` it creates nothing and answers `503` with `code: "captcha-unavailable"`. Create a Turnstile widget for your instance's hostname in the Cloudflare dashboard, put both keys into `.env` and restart the service. Cloudflare's test keys only work outside production. See [Installation](installation.md#environment-variables).

### Let's Encrypt rate limit hit

If you've been experimenting with certbot and hit a rate limit, use the **staging** environment for testing:
Expand Down
54 changes: 50 additions & 4 deletions ecosystem/msk-paste/installation.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@ This guide walks you through installing MSK Paste on your own Debian or Ubuntu s
| Component | Minimum Version |
|---|---|
| OS | Debian 11+ / Ubuntu 22.04+ |
| Node.js | 20.x or 22.x LTS |
| Node.js | 24 LTS or newer |
| MariaDB | 10.6+ |
| Apache | 2.4+ (with `mod_proxy`, `mod_proxy_http`, `mod_headers`, `mod_ssl`) |
| Domain | A subdomain pointing to your server (e.g. `paste.example.com`) |
Expand Down Expand Up @@ -54,10 +54,10 @@ When it finishes, the service is live at your domain over HTTPS.

If you prefer to do it step by step:

### 1. Install Node.js 22
### 1. Install Node.js 24

```bash
curl -fsSL https://deb.nodesource.com/setup_22.x | sudo -E bash -
curl -fsSL https://deb.nodesource.com/setup_24.x | sudo -E bash -
sudo apt install -y nodejs
```

Expand Down Expand Up @@ -147,9 +147,20 @@ DB_NAME=msk_paste
# Generate with: openssl rand -hex 32
IP_HASH_SECRET=change_me_with_openssl_rand_hex_32

# ─── Spam protection ─────────────────────────────────────────────────
# Cloudflare Turnstile for the create form, required in production
CAPTCHA_SITE_KEY=
CAPTCHA_SECRET_KEY=
# Pastes and bytes one IP address may create through the form per 24 hours
DAILY_PASTES_PER_IP=20
DAILY_BYTES_PER_IP=5242880
# Pastes one API key may create per hour
API_KEY_CREATE_PER_HOUR=120

# ─── Limits ──────────────────────────────────────────────────────────
RATE_LIMIT_CREATE_PER_HOUR=10
MAX_PASTE_SIZE_BYTES=1048576 # 1 MB
# 1 MB
MAX_PASTE_SIZE_BYTES=1048576
PASTE_ID_LENGTH=8
PASTE_ID_MIN_CUSTOM=4
PASTE_ID_MAX_CUSTOM=32
Expand All @@ -159,6 +170,10 @@ PASTE_ID_MAX_CUSTOM=32
The `IP_HASH_SECRET` must be at least 32 hex characters. Generate it with `openssl rand -hex 32`. **Never commit it to git.**
:::

:::info
The create form needs a Cloudflare Turnstile widget. Create one in the Cloudflare dashboard under **Turnstile** for the hostname of your instance (e.g. `paste.example.com`) and put its site key and secret key into `CAPTCHA_SITE_KEY` and `CAPTCHA_SECRET_KEY`. Both are required in production: without them the form refuses to create pastes. Cloudflare's test keys (site key `1x00000000000000000000AA`, secret `1x0000000000000000000000000000000AA`) always pass, but only outside production.
:::

---

## Scheduled tasks
Expand All @@ -179,6 +194,33 @@ sudo crontab -e

---

## API keys

Creating pastes through the [REST API](api.md#authentication) needs an API key. There is no self-service, you issue keys on the server with the key script. It needs read access to `/opt/msk-paste/.env`, so run it as root or as the service user:

```bash
sudo bash /opt/msk-paste/scripts/api-key.sh help
```

The commands:

- `list`: all keys with id, prefix, label, creation time, last use and state
- `create ["<label>"]`: issues a new key starting with `mskp_`. It is shown **once**, store it right away
- `revoke <key|prefix|id>`: blocks a key at once. Its row stays, with the time it was revoked
- `delete <key|prefix|id>`: removes a key entirely
- `help`: shows this overview

For example:

```bash
sudo bash /opt/msk-paste/scripts/api-key.sh create "CI pipeline"
sudo bash /opt/msk-paste/scripts/api-key.sh revoke 3
```

Only a scrypt hash of a key is stored, in the table `api_keys` that migration `003_api_keys.sql` adds, together with a short prefix so you can tell keys apart in `list`. A lost key cannot be shown again: revoke it and issue a new one. Each key may create 120 pastes per hour (`API_KEY_CREATE_PER_HOUR`).

---

## Updating

The repository includes an `update.sh` script for manual updates:
Expand All @@ -191,6 +233,10 @@ This pulls the latest commit, installs dependencies, runs migrations, rebuilds,

If you have configured GitHub Actions deployment, pushing to `main` does this automatically.

:::warning
Coming from a version without spam protection? Install Node.js 24 first and add the new variables to `.env` (see [Environment variables](#environment-variables)). Without `CAPTCHA_SITE_KEY` and `CAPTCHA_SECRET_KEY` the form stops creating pastes in production, and scripts that create pastes through the API need an [API key](#api-keys) from then on.
:::

---

## Verifying the installation
Expand Down
36 changes: 27 additions & 9 deletions ecosystem/msk-paste/privacy.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,19 +26,21 @@ For every paste created, the database row contains:
| `delete_token` | Random 64-char token, shown only to the creator |
| `size_bytes` | Size of `content` in bytes |
| `created_at` | Creation timestamp |
| `created_ip_hash` | **HMAC-SHA-256** of the creator's IP, used only for rate limiting |
| `created_ip_hash` | **HMAC-SHA-256** of the creator's IP, used only for rate limiting, including the daily limit of the web form. Deleted together with the paste |

That's it. There is no `views` table, no user table, no session table, no IP-address column in plain text.

API keys for the REST API live in a table of their own, `api_keys`. It holds only a scrypt hash of each key, a short prefix, a label, and when the key was created, last used and revoked. The key itself is never stored.

---

## What is NOT stored

- **No plain-text IP addresses.** IPs are hashed with HMAC-SHA-256 using a server-side secret (`IP_HASH_SECRET`). Without the secret, the hashes cannot be reversed.
- **No GeoIP / country lookups.** The application never queries any geolocation service.
- **No analytics.** No Google Analytics, no Plausible, no Fathom, no Matomo, nothing.
- **No tracking cookies.** The only cookie set by MSK Paste is `MSK_PASTE_LOCALE`, which stores your language preference (`de` or `en`).
- **No third-party scripts.** All assets are served from the same origin. Shiki and fonts are bundled at build time.
- **No tracking cookies.** The only cookie set by MSK Paste is `NEXT_LOCALE`, which stores your language preference (`de` or `en`).
- **One third-party script, and only on the create page.** The page with the create form loads Cloudflare Turnstile as spam protection, see [below](#spam-protection-cloudflare-turnstile). Pages that display a paste load nothing from third parties. Everything else is served from the same origin, Shiki and fonts are bundled at build time.
- **No referer logging at the app level.** Apache may write standard access logs (configurable by the host).
- **No view history.** The view counter increments, but the application has no way to know *who* viewed a paste.

Expand All @@ -48,13 +50,27 @@ That's it. There is no `views` table, no user table, no session table, no IP-add

Rate limiting needs to identify "the same client" without storing identifiable information. The flow is:

1. Take the requesting IP from `X-Forwarded-For` (set by Apache) or the socket address.
1. Take the client IP from `X-Real-IP`, which Apache sets from the connection and which replaces whatever a client sent. If it is missing, take the last entry of `X-Forwarded-For`, the one Apache appended. `CF-Connecting-IP` and the first `X-Forwarded-For` entry are never used, because the client writes them itself and could pick a new address for every request.
2. Compute `HMAC-SHA-256(ip, IP_HASH_SECRET)`.
3. Store the hash in an **in-memory** sliding window of recent requests.
3. Store the hash in an **in-memory** sliding window of recent requests. That is the hourly limit.

The daily limit of the web form (20 pastes and 5 MB per 24 hours) counts the pastes stored with the same `created_ip_hash`. The hash is deleted together with the paste.

Because the secret is generated per-installation (`openssl rand -hex 32`), even an attacker with full database access cannot reverse hashes back to IP addresses, there is no rainbow table they can compute without the secret.

The in-memory bucket resets on every restart, so even the rate-limit state is short-lived.
The in-memory bucket of the hourly limit resets on every restart, so that part of the rate-limit state is short-lived. Requests with an API key are counted per key, not per IP address.

---

## Spam protection (Cloudflare Turnstile)

The page with the create form loads Cloudflare Turnstile from `challenges.cloudflare.com`. Turnstile checks whether a person is sending the form. It is the only third-party script of MSK Paste, and it is not loaded on pages that display a paste.

When the create page loads, your browser connects to Cloudflare, which receives your IP address and information about your browser. When you send the form, MSK Paste passes the Turnstile token and your IP address to Cloudflare to confirm the check.

- **Provider:** Cloudflare, Inc., USA
- **Legal basis:** Art. 6 (1) (f) GDPR, the legitimate interest in keeping the service free of spam and abuse
- **Transfer to the USA:** Cloudflare is certified under the EU-US Data Privacy Framework. Standard Contractual Clauses apply in addition.

---

Expand Down Expand Up @@ -102,17 +118,19 @@ You are responsible for storing backups securely. Consider:

## CSP and security headers

MSK Paste ships with strict Content Security Policy and other headers via `next.config.ts`:
MSK Paste sets a strict Content Security Policy and other headers on every response, in `proxy.ts`:

```
Content-Security-Policy: default-src 'self'; ...
Content-Security-Policy: default-src 'none'; script-src 'self' 'nonce-...' 'strict-dynamic' https://challenges.cloudflare.com; ...
Strict-Transport-Security: max-age=63072000; includeSubDomains; preload
X-Content-Type-Options: nosniff
Referrer-Policy: strict-origin-when-cross-origin
X-Frame-Options: DENY
Permissions-Policy: camera=(), microphone=(), geolocation=()
Permissions-Policy: camera=(), microphone=(), geolocation=(), payment=(), usb=()
```

The only outside source the policy allows is `https://challenges.cloudflare.com`, for the Turnstile script, its frame and its requests.

The Apache vhost also enforces HTTPS, OCSP stapling, and modern TLS ciphers.

---
Expand Down
Loading
Loading