Architecture

This page is for the curious and for agent authors: how Agent City is built, what runs where, and how the pieces talk to each other. You don’t need any of it to use the app.

The big picture

┌──────────────────────────── Agent City.app ───────────────────────────┐
│                                                                       │
│  Tauri shell (Rust)  ──starts──▶  Embedded Python 3.12                │
│  · window (WKWebView)              └─ server.py (FastAPI + uvicorn)   │
│  · menu bar, Open at Login              · HTTP + WebSocket /ws        │
│  · native notifications                 · agents, coordinator, loop   │
│  · signed auto-updates                  · SQLite in the data folder   │
│        │                                   │                          │
│        └──── loads http://127.0.0.1:47800 ─┘                          │
│                                            │ runs                     │
│  Swift helpers: agent-city-computer ◀──────┤ (Accessibility)          │
│                 agent-city-meetings ◀──────┘ (mic + system audio)     │
└───────────────────────────────────────────────────────────────────────┘
          │ OpenAI-compatible chat completions
          ▼
  Ollama on this Mac · OpenAI · Anthropic · LM Studio · vLLM · OpenRouter…

Agent City is a local web app in a native shell. The UI is a web page, the brains are a Python server, and the shell’s job is to start that server, show its page, and make it behave like a Mac app.

The shell: Tauri

The app is a small Tauri shell written in Rust. On launch it:

1. Starts the server with the Python inside the bundle, passing:

   • HIVE_HOME=~/Library/Application Support/Agent City (where your data goes),
   • a token generated fresh at each launch, and
   • HIVE_EXIT_WITH_PARENT, so the server stops itself if the app dies unexpectedly.

2. Waits for the ready line. The server prints exactly one line when it’s listening:

   AGENT_CITY_READY {"url": "http://127.0.0.1:47800", "port": 47800, "token": true, "home": "…"}

   Until then the window shows “Starting Agent City”. If the server can’t start, the window shows why, with the end of its log.

3. Opens the page at http://127.0.0.1:<port>/?token=…. That first request swaps the token for an HttpOnly cookie, and from then on the cookie is enough.

It prefers port 47800 and picks a free one if that’s taken.

The shell also:

• lives in the menu bar. Closing the window doesn’t stop the server, so automations and reminders keep running. A single-instance lock means opening the app again focuses the existing window.
• opens external links and sign-ins in your browser. Google refuses OAuth inside embedded web views, so sign-ins go to your default browser and come back to a 127.0.0.1 callback.
• bridges notifications. WKWebView has no browser Notification API, so the shell injects one backed by Tauri’s notification plugin. A capability file lets only the server’s own page use it.
• handles updates. It reads a signed latest.json from GitHub Releases 20 seconds after launch and every 6 hours, and refuses any update not signed with Agent City’s updater key. Paired phones can ask the Mac to update: the shell and server talk over the server’s stdin and stdout (AGENT_CITY_ASK update_check, AGENT_CITY_UPDATE {…}).

The embedded Python

Agent City ships its own Python, so there’s nothing for you to install.

• Runtime: a relocatable CPython 3.12 from python-build-standalone, the install_only_stripped build for the Mac’s architecture. The build script checks it against the release’s SHA256SUMS.
• Packages: requirements.txt is installed into it with uv at build time: openai, fastapi, uvicorn[standard], pypdf, reportlab, python-docx, openpyxl, pillow, segno, cryptography and telethon.
• Trimmed: the test suite, idlelib, tkinter/Tcl/Tk, ensurepip, headers, caches and pip’s launchers are removed. pip itself stays, as python -m pip, for optional downloads.
• Read-only: the bundle is never written to. Optional extras like the built-in Whisper (faster-whisper, about 200 MB of packages) and Kokoro text-to-speech are installed into your data folder with the bundled pip, alongside their models.
• Signed: every binary in the bundled Python is signed with the hardened runtime (the interpreter with its own entitlements), then the app is signed, notarized by Apple and stapled.

The app’s code (server.py, core/, the shipped agents and the prebuilt web page) is copied into the bundle from git, so only tracked files ship.

Code and data are kept apart

Location	Holds	Writable
Agent City.app/Contents/Resources/	Python, the server code, shipped agents, the web page, Swift helpers	No
HIVE_HOME/memory/	Every database, settings, keys, and agents installed from sources (memory/agents/)	Yes
HIVE_HOME/agents/	Agents you build with the Agent builder	Yes
HIVE_HOME/.removed-agents/	Agents you removed, so they can be restored	Yes
HIVE_HOME/logs/server.log	The server’s log	Yes

Shipped agents can’t be edited or removed. If you make an agent with the same name, yours wins.

The server

server.py is a FastAPI app run by uvicorn. It serves:

• the web UI: a Preact + TypeScript app built with Vite into static files ahead of time. Routing uses preact-iso and state uses signals. There’s no Node at runtime.
• one WebSocket, /ws: everything the UI does (send a message, stop, save an automation, approve, rename) is a JSON message up, and every change (tokens streaming, tool calls, agent asks, views, state) is a JSON message down. Reloading the page or opening a second window picks up the same live state, because conversations live in the server, not the page.
• a few HTTP endpoints: /api/events, /api/health, file uploads, voice, speech, workspace downloads and the OAuth callback.

Request guard

Agent City has no login but holds tokens that can read your mail, so a web page you visit must not be able to reach it. Every request passes three checks:

Check	Stops
Host: the Host header must be localhost, 127.0.0.1 or ::1	DNS rebinding
Origin: state-changing requests and WebSocket upgrades from another site’s page are refused	CSRF
Token: the per-launch token, as a cookie	Anything else on your Mac

/api/events (which uses event keys), /api/health and /oauth/callback (tied to an unguessable state) are exempt from the token.

The phone listener

With phone access on, a second listener opens on 0.0.0.0:47801. Every request there needs a paired phone’s key (stored as a hash), except pairing itself, /api/events and /api/health. The main listener stays on 127.0.0.1. For HTTPS, Agent City runs tailscale serve --bg --https=8443 in front of it.

Agents

An agent is a folder with a SOUL.md and a tools.py. At startup the server loads every agent folder from three places: starters and shipped agents in the bundle, agents installed from sources, and agents you built. See Building agents for the files.

• SOUL.md becomes the agent’s system prompt, minus its YAML front matter. The description line is what the coordinator routes by.
• tools.py is imported into the server process. TOOLS are OpenAI function-calling schemas, and HANDLERS maps each name to a Python function.

Agents run in-process, in threads, not in a sandbox. That’s why agents from sources are reviewed before they install.

The tool-calling loop

Each agent runs the standard loop against any OpenAI-compatible chat completions endpoint, using the openai SDK:

1. Send the system prompt (soul + shared memory + its own memory + context + lessons), the conversation and the tool schemas.
2. If the model calls tools, run them (several at once if it asks for several), append the results, and go again.
3. Stop when the model answers in text, or when the step limit is reached. The default is 10 in a chat; on the last step it’s told to answer with what it has.

Older tool results are cut to 2,000 characters, and when the model’s server reports its context size (Ollama does), the oldest messages are dropped before the context overflows. Shared memory is never trimmed.

The coordinator and ask_agent

The coordinator (My team) is an agent with no tools except a built-in ask_agent(agent, question). Its prompt lists every agent’s one-line description. Each ask_agent call starts a fresh, one-off conversation with that agent, so the coordinator writes complete questions.

Questions go at most two hops: coordinator → agent → agent. An agent answering another agent can’t ask a third, so there are no loops. When the coordinator asks several agents in one turn, they run in parallel threads.

Some built-in tools are only offered when a message calls for them (set_reminder when you mention reminding, make_chart when you ask for a chart, start_background_task for long work). Small models pick the wrong tool less often when they have fewer to choose from.

Keys are scoped per agent

Saved keys never enter the process’s real environment, so programs Agent City starts (MCP servers, Claude Code, signal-cli) don’t inherit them. os.environ is replaced with a view that, while an agent’s tool runs, shows only the keys that agent declares in NEEDS, OPTIONAL and CONNECTS, and hides anything else that looks like a secret. Saved values are also redacted from tool results, chats and the audit log. This stops accidental leaks between agents. It isn’t a sandbox against hostile code.

Concurrency

Every chat, automation, event and background task runs in its own thread, with per-run state in contextvars. Access to the model is queued per server:

• a model on your Mac or LAN runs one run at a time, so the others wait their turn,
• a cloud model takes up to 4 requests at once (1 to 8 in Settings), and
• the Computer agent takes one run at a time, since there’s one screen.

A run waiting for your approval releases the model while it waits.

Native helpers

Two things only macOS itself can do are handled by small Swift command-line helpers in the bundle, which the server runs as subprocesses that answer in JSON:

Helper	Used by	What it does
agent-city-computer	Computer	Reads windows through the Accessibility API as numbered elements, performs clicks and keystrokes, takes screenshots, and sets AXManualAccessibility so Electron apps expose their contents.
agent-city-meetings	Meetings	Detects when a call app takes the microphone, records the mic (with echo cancellation) and system audio, cuts them at pauses, and separates speakers with FluidAudio.

Storage

Everything is in SQLite files under HIVE_HOME/memory/. There’s no external database.

File	Holds
memory.db	Shared memory, each agent’s own memory and context
history.db	Chats
files.db	Text extracted from shared files, with an FTS5 index
workflows.db	Automations and their last 30 runs
reminders.db	Reminders and timers
events.db	Incoming events (last 500) and hashed event keys
audit.db	Every tool call
keys.db	API keys and OAuth tokens
oauth.db	Which sign-ins each agent was granted
settings.db	Settings
meetings.db	Meeting transcripts and notes
mcp.json, repos.json	MCP server configs and agent sources

Platform support

The Tauri shell is cross-platform, but the build script, the bundled Python layout, the Swift helpers and the Agent builder’s launchers are Mac-only today. Builds are per-architecture: the published download is arm64 (Apple silicon). Windows and Linux versions are planned.