Axophyte

Axophyte (Axo) is one agent across two surfaces: a multi-server Discord bot running on Bun, and Toph's private workspace agent on Pi Web (running the Pi Coding Agent SDK on Node).

Both surfaces share Axophyte's voice, media decoders, SVG artist, PDF form tools, and web search in src/agent, while keeping separate turn loops, tool boundaries, and memory stores in src/discord and src/pi.

Capabilities & tools

Shared (src/agent)

Capability	Tools / Pipeline	Description
Drawing	draw	Plans and writes SVG in an isolated art session, renders PNG locally with resvg (DejaVu fonts), and self-reviews with vision (revise: true edits the channel or chat's last drawing)
PDF forms	read_pdf_form, fill_pdf_form	Lists AcroForm fields or detects drawn blanks (underlines, boxes, checkboxes) from a 150 dpi render, then fills fields in black Helvetica and renders page previews for visual verification
Web	web_search, open_url	Searches the web and extracts page text through Parallel and Tavily without fetching external pages directly from the host
Media decoding	Automatic on attachments	Converts videos and animated GIFs into 2×2 stills grids (ffmpeg), transcribes voice notes and audio/video tracks locally (whisper.cpp), and extracts text from PDFs (poppler) and docx/odt/rtf/epub (pandoc)

PDF form limits: Rotated pages, flat scans without drawn lines/boxes, comb fields or grid cells that share edges, non-Latin-1 text, and signatures are not handled automatically.

Discord (src/discord)

On Discord, Axophyte participates in each server's configured forum channel, in active threads (threads started from an Axophyte message or where it was mentioned), and anywhere else when @Axophyte is mentioned or replied to. It coalesces message bursts and typing indicators before answering, tracks per-server custom emoji meanings, and compacts older thread history into SQLite summaries when a conversation outgrows the model's context window.

Tool	Description
react	Reacts to a tagged message (m1, m2, …) with a server or Unicode emoji; can stand alone (reply: false) or accompany a text reply
remember, revise	Saves or updates per-server notes about the speaker (learned only in channels readable by every server member)
search_server, read_conversation	Searches and reads server history across channels visible to everyone in the current channel (disabled in private threads)
self_info	Returns runtime facts on demand: active model/provider, privacy boundary, vision support, memory rules, and commands

Slash commands

All slash command replies are ephemeral and never enter conversation history.

Command	Access	Effect
/model [name]	Server admin	View or switch the active model bot-wide (local, anthropic/…, openai-codex/…, deepseek/…, google/…); resets to local on restart
/codex-login	App owner + server admin	Link a ChatGPT subscription via OpenAI device-code login for openai-codex/ models
/memory show	Everyone	Show Axophyte's saved notes about you in this server
/memory forget <item|all>	Everyone	Delete one or all of your saved notes in this server
/emoji <emoji> <use>	Server admin	Set or override what a custom or application emoji means in this server
/retry	Server admin	Re-answer the channel's latest human message

Pi Web (src/pi)

The pi package builds an axophyte-pi Pi package (extension.js), which loads into Pi Web alongside Pi's coding tools (read, bash, edit, write, grep, find, ls, codemode, tool_search) and Pi Web's subagent tools (Agent, get_subagent_result, steer_subagent), replacing the default coding-agent persona with Axophyte's voice and OMP's engineering rules.

• Long-term memory: Hindsight-backed (axo-toph bank). Main sessions get one stable working-with-toph mental model in the system prompt, holding patterns about how Toph works that his written guidelines miss; nothing else is recalled automatically. recall and reflect query past sessions on demand, and retain saves explicit facts. Each session's active-branch transcript is retained as one replace-in-place document every 3 completed user turns and at session end when turns remain pending. Navigating to a different branch supersedes the abandoned branch at the next retain. Subagents get no mental model and are never retained.
• Attachments & PDFs: Chat uploads preserve original files outside the workspace, with inline raster images and folded SVG previews, transcripts, and extracted document text. Filled PDFs are saved to forms/<name>-filled.pdf; drawings appear in chat and are saved under drawings/.
• Session command-line tools: Call nix_shell with nixpkgs package attributes, for example {"packages":["imagemagick","potrace","resvg"]}. Packages become available to later bash calls without wrapping each command in nix shell. Use package attributes rather than executable names (imagemagick, not magick); {"packages":[]} lists the active loadout. Additions are cumulative, failed or cancelled loads preserve the previous environment, and other sessions are unaffected.

Codemode uses the same session environment:

await tools.nix_shell({ packages: ["imagemagick", "potrace", "resvg"] });
return await tools.bash({
  command: "magick -version && potrace --version && resvg --version",
});

The loadout is held in memory for the running session, not installed into the user's profile or restored from chat history. Session shutdown, extension reload, or a service restart clears it. Only command lookup (PATH) is shared; shell variables and working-directory changes do not persist between commands. Pi's shellCommandPrefix and shellPath settings still apply to each bash call. Internal media-helper processes are separate from the agent's bash tool and retain their bundled dependencies.

Setup

Discord (nix run .#discord)

AXOPHYTE_DISCORD_TOKEN=... \
AXOPHYTE_TAVILY_KEY=... \
AXOPHYTE_SERVERS='{"<guild_id>":"<forum_id_or_null>"}' \
AXOPHYTE_LLAMA_URL=http://127.0.0.1:8080 \
AXOPHYTE_MODEL=qwen3.8-27b-abliterated \
  nix run path:.#discord

The Nix wrapper bundles ffmpeg, poppler-utils, pandoc, whisper-cpp (with large-v3-turbo-q5_0), resvg, and DejaVu fonts automatically.

Variable	Required	Meaning
AXOPHYTE_DISCORD_TOKEN	Yes	Discord bot token
AXOPHYTE_TAVILY_KEY	No	Tavily API key for primary web search & extraction (falls back to Parallel)
AXOPHYTE_SERVERS	Yes	JSON map of guildId → forumChannelId | null (null = mentions and replies only)
AXOPHYTE_LLAMA_URL	Yes	Base URL of the default local llama.cpp server
AXOPHYTE_MODEL	Yes	Model ID used for local inference at startup
ANTHROPIC_API_KEY, DEEPSEEK_API_KEY, GEMINI_API_KEY	No	Unlock corresponding cloud providers in /model autocomplete
AXOPHYTE_AUTH_FILE	No	Path to Codex OAuth credentials (defaults to ./auth.json)
AXOPHYTE_WHISPER_MODEL, AXOPHYTE_RESVG, AXOPHYTE_SVG_FONTS	No	Override bundled Nix store paths when running outside the wrapper

Pi Web (nix build .#pi)

Build the Pi package and add its store path to $PI_CODING_AGENT_DIR/settings.json under "packages":

nix build path:.#pi -o result-pi

Variable	Required	Meaning
AXO_HINDSIGHT_URL	Yes	Hindsight server URL
AXO_HINDSIGHT_BANK	No	Hindsight bank (defaults to axo-toph; override for throwaway smoke tests)
HINDSIGHT_API_KEY	No	Bearer token if the Hindsight server requires authentication
TAVILY_API_KEY	No	Optional Tavily fallback key for web search (defaults to keyless Parallel)

For Discord Developer Portal setup, /codex-login, and dot.nix / Pangolin host deployment on Zebes and Nimbus, see docs/deployment.md.

Development

nix develop path:.
bun install --frozen-lockfile
bun test
bun run typecheck
bun2nix -o bun.nix          # after dependency changes
nix flake check path:.      # builds .#discord and .#pi