Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ MCP data tools and web search. User-facing docs are in `README.md`; this file is
`openrouter.py` + `zai.py` (the only provider-specific code), `runner.py` (THE tool loop), `policy.py` (THE retry ladder), `gate.py` (which tools a message needs).
- `prompts.py` all prompts. `ask.py` one entry point: route -> Request -> policy.ask -> usage -> NEEDS_TOOLS rerun.
- `telegram/`: `handlers.py` (trigger -> request -> reply), `send.py`, `draft.py` (typing + streaming
drafts), `commands.py` (/credits, /usage). `features/`: `holding_news.py`, `movers.py`, `dm_buttons.py`, `post.py`.
drafts), `commands.py` (/credits), `access.py` (who is an admin). `features/`: `holding_news.py`, `movers.py`, `dm_buttons.py`, `post.py`.
- `app.py` wiring/startup; `context.py` `Ctx` (settings, store, backend, registry, bot).
- `lib/` holds only what the bot needs to run. Developer helpers live in `tools/`: `report.py` (`make prompt-report`),
`capture.py` (real MCP output for fixtures); the bot never imports them.
Expand Down
16 changes: 12 additions & 4 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -160,8 +160,15 @@ Each is off by default. The holding-news DM and the movers reply are switched on
|---|---|---|
| `/start` | DM | Greeting (and the buttons if enabled) |
| `/holdingnews` | DM, if holding news is on | Run the holding-news check now |
| `/credits` | a user ID in `ADMIN_CHAT_IDS` | Provider balance (OpenRouter; xAI and z.ai have none to show) |
| `/usage` | a user ID in `ADMIN_CHAT_IDS` | Requests, tokens (and % cached) and cost per model, last 24 h and 7 days |
| `/credits` | an admin (see below), in a group or a DM | Provider balance (OpenRouter; xAI and z.ai have none to show), then requests, tokens (and % cached) and cost per model for the last 24 h and 7 days |

**Admins** are the user IDs in `ADMIN_CHAT_IDS` plus anyone who administers a group the bot is in (checked with
Telegram, cached for five minutes). The bot only knows a group once someone has written in it, or it was added, since
it started tracking groups. Admins who post anonymously (as the group) can't be recognised. With `ADMIN_ONLY` on (the
default), only admins can DM the bot, and Sharesight data is for admins only: anyone else asking about a portfolio
gets "Portfolio data is for admins only." Holding-news DMs, buttons and down-server messages the bot sends itself are
unaffected. With nobody in `ADMIN_CHAT_IDS` and no known group, nobody can DM the bot or reach Sharesight: set
`ADMIN_ONLY=off` to allow everyone.

## Caching and token saving
**Provider prompt caching** (xAI and OpenRouter keep a conversation on the server holding its cached
Expand Down Expand Up @@ -198,7 +205,7 @@ Each answer's log line shows the cached percentage.
symbol, tags and Yahoo URLs dropped), lines cut to 240 characters and the bot's own to 160.
- **Smaller prompts:** the system prompt is about half its previous size, the repeated instruction
tail lives in the (cached) system prompt, images go at low detail.
- **Measure it:** every request is recorded in the `usage` table (`/usage`), and
- **Measure it:** every request is recorded in the `usage` table (`/credits`), and
`make prompt-report` prints where a typical request's tokens go.

## Running several instances
Expand Down Expand Up @@ -237,7 +244,8 @@ Set `TELEGRAM_BOT_TOKEN` and exactly one of the three LLM keys.
### Advanced and optional
| Variable | Meaning |
|---|---|
| `ADMIN_CHAT_IDS` | Telegram IDs of the bot's admins (comma/space separated). Your user ID gets a message when an MCP server goes down and may use `/credits` and `/usage`; a group's chat ID gets the down-server messages only. Empty (default) = no alerts and no admin commands. |
| `ADMIN_CHAT_IDS` | Telegram IDs of the bot's admins (comma/space separated). Your user ID gets a message when an MCP server goes down and may use `/credits`; a group's chat ID gets the down-server messages only. Admins of the groups the bot is in count as admins too. |
| `ADMIN_ONLY` | on|off. Only admins may DM the bot or get Sharesight data (default on). |
| `DB_PATH` | SQLite file for chat history (default chat_log.db). Instances may share it. |
| `HISTORY_LIMIT` | Messages of chat history in the prompt; the window is HISTORY_LIMIT to 1.5x (default 20). |
| `MAX_TOKENS` | Reply cap in tokens, reasoning included (default 3000). |
Expand Down
8 changes: 3 additions & 5 deletions lib/app.py
Original file line number Diff line number Diff line change
Expand Up @@ -123,24 +123,22 @@ async def wrapper(update: Update, context: ContextTypes.DEFAULT_TYPE) -> None:
return wrapper

# Commands first: the first matching handler wins, and the catch-all would swallow them.
if st.admin_chats:
app.add_handler(CommandHandler("credits", command(commands.credits)))
app.add_handler(CommandHandler("usage", command(commands.usage)))
app.add_handler(CommandHandler("credits", command(commands.credits)))
app.add_handler(MessageHandler(
(filters.ChatType.GROUPS | filters.ChatType.PRIVATE)
& (filters.UpdateType.MESSAGE | filters.UpdateType.EDITED_MESSAGE)
& ~filters.StatusUpdate.ALL, handlers.on_message))
if st.holding_news: # the Unsubscribe / Undo buttons on holding-news messages
app.add_handler(CallbackQueryHandler(command(holding_news.on_button), pattern=r"^hn:"))
if st.post_to_groups: # keeps the list of groups the bot can post in
if st.post_to_groups or st.admin_only: # keeps the list of groups the bot is in
app.add_handler(ChatMemberHandler(command(post.on_membership), ChatMemberHandler.MY_CHAT_MEMBER))
app.add_error_handler(on_error)
return app


def allowed_updates(st: config.Settings) -> list[str]:
return (["message", "edited_message"] + (["callback_query"] if st.holding_news else [])
+ (["my_chat_member"] if st.post_to_groups else []))
+ (["my_chat_member"] if st.post_to_groups or st.admin_only else []))


def git_hash() -> str:
Expand Down
14 changes: 11 additions & 3 deletions lib/ask.py
Original file line number Diff line number Diff line change
Expand Up @@ -37,17 +37,25 @@ def forward(text: str) -> None:
return forward


def _public(servers: list) -> list:
"""The MCP servers anyone may use: not the ones gated on portfolio questions (Sharesight)."""
return [s for s in servers if s.cfg.get("gate") != "portfolio"]


async def ask(ctx: Ctx, parts: list[dict], route: Route, *, must_search: bool = False,
cache_key: str | None = None, on_text=None, kind: str = "chat",
chat_id: int = 0, extra_tools=()) -> Answer:
chat_id: int = 0, extra_tools=(), admin: bool = True) -> Answer:
"""Ask the configured provider and record the usage. `route` says which tools to offer;
must_search forces a search (and offers nothing else). A message the gate gave fewer tools
than exist that gets back NEEDS_TOOLS is asked again with everything on."""
than exist that gets back NEEDS_TOOLS is asked again with everything on. A non-admin never
gets the portfolio tools, in either request."""
st = ctx.st
if must_search:
if not st.search:
raise RuntimeError("search is disabled, but this request needs it")
route = Route([], True)
elif not admin:
route = dataclasses.replace(route, servers=_public(route.servers))
saver = st.token_saver
req = Request(
system=system_prompt(ctx.first_name, route, ctx.backend.search_what, saver=saver),
Expand All @@ -65,7 +73,7 @@ async def ask(ctx: Ctx, parts: list[dict], route: Route, *, must_search: bool =
await _record(ctx, req.model, kind, chat_id, answer)
if (route.simple or route.partial) and wants_tools(answer.text):
log.info("Gate offered fewer tools, the model asked for more: asking again with everything on")
full = Route(ctx.registry.up(), st.search)
full = Route(ctx.registry.up() if admin else _public(ctx.registry.up()), st.search)
req = dataclasses.replace(
req, system=system_prompt(ctx.first_name, full, ctx.backend.search_what, saver=saver),
model=st.model, tools=[t for s in full.servers for t in s.tools] + [t for t, _ in extra_tools],
Expand Down
5 changes: 4 additions & 1 deletion lib/config.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,8 @@
"REASONING": "Reasoning effort: low, medium or high; empty = the model's default (low on z.ai, which cannot turn thinking off).",
"SEARCH": "on|off. Web search (and X search on xAI; z.ai's search API on z.ai). Default on.",
# Advanced and optional
"ADMIN_CHAT_IDS": "Telegram IDs of the bot's admins (comma/space separated). Your user ID gets MCP-down alerts and may use /credits and /usage; a group's chat ID gets the alerts only. Empty = neither.",
"ADMIN_CHAT_IDS": "Telegram IDs of the bot's admins (comma/space separated). Your user ID gets MCP-down alerts and may use /credits; a group's chat ID gets the alerts only. Admins of the groups the bot is in count as admins too.",
"ADMIN_ONLY": "on|off. Only admins may DM the bot or get Sharesight data (default on).",
"DB_PATH": "SQLite file for chat history (default chat_log.db). Instances may share it.",
"HISTORY_LIMIT": "Messages of chat history in the prompt; the window is HISTORY_LIMIT to 1.5x (default 20).",
"MAX_TOKENS": "Reply cap in tokens, reasoning included (default 3000).",
Expand Down Expand Up @@ -73,6 +74,7 @@ class Settings:
tz: ZoneInfo
mcp_config: str
admin_chats: frozenset[int] # user IDs (alerts + admin commands) and group chat IDs (alerts only)
admin_only: bool
movers_bots: frozenset[str]
dm_buttons: bool
post_to_groups: bool
Expand Down Expand Up @@ -166,6 +168,7 @@ def load(env: Mapping[str, str] | None = None) -> Settings:
tz=ZoneInfo(env.get("BOT_TZ", "UTC")),
mcp_config=env.get("MCP_CONFIG", "mcp_servers.json"),
admin_chats=_ids(env.get("ADMIN_CHAT_IDS", "")),
admin_only=_flag(env, "ADMIN_ONLY", default=True),
movers_bots=frozenset(
u.lower().lstrip("@") for u in env.get("MOVERS_BOTS", "").replace(",", " ").split()
),
Expand Down
3 changes: 2 additions & 1 deletion lib/context.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,7 @@

from __future__ import annotations

from dataclasses import dataclass
from dataclasses import dataclass, field
from datetime import datetime
from typing import Any

Expand All @@ -19,6 +19,7 @@ class Ctx:
backend: Backend
registry: Registry
bot: Any = None # telegram.Bot, set at startup
group_admins: dict = field(default_factory=dict) # chat id -> (admin user IDs, expiry); see telegram/access.py

@property
def first_name(self) -> str:
Expand Down
38 changes: 38 additions & 0 deletions lib/telegram/access.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
"""Who counts as an admin: a user in ADMIN_CHAT_IDS, or an admin of a group the bot has seen."""

from __future__ import annotations

import asyncio
import logging
import time

from telegram.error import TelegramError

from ..context import Ctx

log = logging.getLogger("bot")
CACHE_SECONDS = 300 # a demoted admin loses access within this long


async def _group_admins(ctx: Ctx, chat_id: int) -> frozenset[int]:
cached = ctx.group_admins.get(chat_id)
if cached and cached[1] > time.monotonic():
return cached[0]
try:
admins = frozenset(a.user.id for a in await ctx.bot.get_chat_administrators(chat_id))
except TelegramError as e:
log.warning("Can't read the admins of chat %s: %s", chat_id, e)
admins = frozenset()
ctx.group_admins[chat_id] = (admins, time.monotonic() + CACHE_SECONDS)
return admins


async def is_admin(ctx: Ctx, user_id: int) -> bool:
"""True for a user in ADMIN_CHAT_IDS or an administrator of any group the bot knows. Anonymous
admins post as the group itself, so they can't be recognised."""
if user_id in ctx.st.admin_chats:
return True
for chat_id, _ in await asyncio.to_thread(ctx.store.chats):
if user_id in await _group_admins(ctx, chat_id):
return True
return False
34 changes: 16 additions & 18 deletions lib/telegram/commands.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,4 @@
"""Owner commands: /credits (provider balance, where there is one) and /usage (token ledger)."""
"""Admin command: /credits (provider balance, where there is one, and the token ledger)."""

from __future__ import annotations

Expand All @@ -10,35 +10,27 @@
from telegram.ext import ContextTypes

from ..context import Ctx
from .access import is_admin

log = logging.getLogger("bot")


async def _gate(ctx: Ctx, update: Update) -> bool:
"""Log the command message (these handlers take it before on_message would) and say
whether the sender is an admin (a user ID in ADMIN_CHAT_IDS)."""
whether the sender is an admin (see access.is_admin)."""
await asyncio.to_thread(ctx.store.save, update.effective_message)
user = update.effective_user
return bool(user and user.id in ctx.st.admin_chats)
if user and await is_admin(ctx, user.id):
return True
log.info("Ignored %s from user %s: not an admin", update.effective_message.text, user and user.id)
return False


async def _reply(ctx: Ctx, update: Update, text: str) -> None:
sent = await update.effective_message.reply_text(text, disable_notification=True)
await asyncio.to_thread(ctx.store.save, sent)


async def credits(ctx: Ctx, update: Update, context: ContextTypes.DEFAULT_TYPE) -> None:
"""/credits - admins only: how much provider credit is left (OpenRouter)."""
if not await _gate(ctx, update):
return
try:
text = await ctx.backend.credits() or "This provider has no credit balance to show."
except Exception as e:
log.exception("Credit check failed")
text = f"Couldn't fetch credits: {e}"
await _reply(ctx, update, text)


def usage_text(rows: list[tuple], label: str) -> str:
if not rows:
return f"No usage recorded in the {label}."
Expand All @@ -50,12 +42,18 @@ def usage_text(rows: list[tuple], label: str) -> str:
return "\n".join(lines)


async def usage(ctx: Ctx, update: Update, context: ContextTypes.DEFAULT_TYPE) -> None:
"""/usage - admins only: tokens and cost per model for the last 24 hours and 7 days."""
async def credits(ctx: Ctx, update: Update, context: ContextTypes.DEFAULT_TYPE) -> None:
"""/credits - admins only: the provider balance (OpenRouter has one), then tokens and cost per
model for the last 24 hours and 7 days."""
if not await _gate(ctx, update):
return
try:
balance = await ctx.backend.credits() or "This provider has no credit balance to show."
except Exception as e:
log.exception("Credit check failed")
balance = f"Couldn't fetch credits: {e}"
now = int(time.time())
parts = []
parts = [balance]
for label, secs in (("last 24 hours", 86_400), ("last 7 days", 7 * 86_400)):
rows = await asyncio.to_thread(ctx.store.usage_summary, now - secs)
parts.append(usage_text(rows, label))
Expand Down
19 changes: 16 additions & 3 deletions lib/telegram/handlers.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,10 +23,12 @@
from ..store import is_dm
from ..textfmt import md_to_html, strip_disclaimer
from ..tickers import link_tickers
from .access import is_admin
from .draft import Draft, start_typing
from .send import reply_chunks, reply_html

log = logging.getLogger("bot")
PORTFOLIO_ADMINS_ONLY = "Portfolio data is for admins only."
READ_IMAGE_WORDS = re.compile(r"\b(read|text|says?|chart|table|number|screenshot|ocr|zoom|detail|label)\b", re.I)


Expand Down Expand Up @@ -77,7 +79,7 @@ def _record(self, msg: Message) -> None:
self.ctx.store.remember_user(msg)
self.ctx.store.save(msg)
chat = getattr(msg, "chat", None)
if self.ctx.st.post_to_groups and chat and chat.type in post.GROUP_TYPES and chat.title:
if chat and chat.type in post.GROUP_TYPES and chat.title:
self.ctx.store.remember_chat(chat.id, chat.title)

async def on_message(self, update: Update, context: ContextTypes.DEFAULT_TYPE) -> None:
Expand All @@ -90,10 +92,16 @@ async def on_message(self, update: Update, context: ContextTypes.DEFAULT_TYPE) -
trig = self._trigger(msg)
if trig is None:
return
if trig.private and self.ctx.st.admin_only and not await self._is_admin(msg):
log.info("Ignored a DM from user %s: not an admin", msg.from_user and msg.from_user.id)
return
if await self._command(msg, trig):
return
await self._answer(msg, trig)

async def _is_admin(self, msg: Message) -> bool:
return bool(msg.from_user and await is_admin(self.ctx, msg.from_user.id))

def _trigger(self, msg: Message) -> Trigger | None:
"""Is this message for the bot? Other bots are logged but never answered (avoids bot
loops), except movers lists, which are answered once."""
Expand All @@ -111,7 +119,7 @@ def _trigger(self, msg: Message) -> Trigger | None:
return Trigger(text, private, is_movers, reply_target, replied_to_bot)

async def _command(self, msg: Message, trig: Trigger) -> bool:
"""Private-chat commands handled here (/credits and /usage have their own handlers)."""
"""Private-chat commands handled here (/credits has its own handler)."""
if not trig.private or not trig.text.startswith("/"):
return False
command = trig.text.split()[0].split("@")[0].lower()
Expand Down Expand Up @@ -216,6 +224,10 @@ async def _generate(self, msg: Message, trig: Trigger, draft: Draft | None) -> s
else:
context = trig.text + " " + (describe(trig.reply_target) if trig.reply_target else "")
chosen = route(context, st, ctx.registry)
admin = not st.admin_only or await self._is_admin(msg)
if not admin and is_portfolio_question(context, st) and any(
s.cfg.get("gate") == "portfolio" for s in ctx.registry.up()):
return PORTFOLIO_ADMINS_ONLY
extra = []
if st.post_to_groups and trig.private and post.wants_post(trig.text):
extra = [post.tool(ctx, msg.from_user.id, msg.from_user.full_name or "")]
Expand All @@ -225,7 +237,8 @@ async def _generate(self, msg: Message, trig: Trigger, draft: Draft | None) -> s
parts = [{"type": "text", "text": await self._chat_text(msg, trig, fresh=fresh)}]
parts += await self._images(msg, trig)
answer = await ask(ctx, parts, chosen,
cache_key=f"chat-{chat_id}", on_text=on_text, chat_id=chat_id, extra_tools=extra)
cache_key=f"chat-{chat_id}", on_text=on_text, chat_id=chat_id, extra_tools=extra,
admin=admin)
return format_answer(answer.text, movers_bots=st.movers_bots, summary=trig.movers)


Expand Down
2 changes: 1 addition & 1 deletion tests/conftest.py
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
from lib import config
from lib.store import Store

BASE_ENV = {"TELEGRAM_BOT_TOKEN": "1:x", "OPENROUTER_API_KEY": "key"}
BASE_ENV = {"TELEGRAM_BOT_TOKEN": "1:x", "OPENROUTER_API_KEY": "key", "ADMIN_ONLY": "off"} # admin-only has its own tests


@pytest.fixture
Expand Down
Loading
Loading