MacAssist

A minimal, macOS-native personal assistant that runs in your menu bar.

MacAssist is a small background process you forget about and evoke whenever you need it. It quietly checks your email (strictly read-only) and pings you with a native notification when something is actually noteworthy. A hotkey summons a chat; another hotkey screenshots your screen and hands the image to the model before you finish typing your question. Behind it sits any OpenAI-compatible model endpoint (llama.cpp, MLX, vLLM, …) — the design assumes a weak local model and is forgiving of its garbage.

It is not a coding agent and not a general-purpose agent framework. It has 8 tools — email list/read/search, web search, web fetch, screenshot, open (app or URL), notify — and that is the whole world. No shell, no file editing, no subagents, no workflow engine.

The design spirit comes from Sentdex's minion.py (one OpenAI-compatible client, a lean tool loop, memory as markdown). The architecture — a Rust core (all machinery) plus a thin Swift/AppKit app (all visible UI), JSON-lines IPC over stdio — is documented in docs/PLAN.md.

Features

• Chat — menu-bar popover (or ⌥Space), chat-styled transcript, live streaming, tool activity lines, new-chat button for a fresh session, dim "ctx" counter showing the last request's context size (from the endpoint's usage stats).

• Email triage — every hour (and at startup) unseen Apple Mail is read in a batch and summarized by one model call; noteworthy mail raises a native notification. Everything is read-only, enforced at the script level and tested. State (seen store, daily triage log) is local.

• Tools — the model can: list/read/search mail (subject, sender, bodies, plus the local triage archive for digging), web search (SearXNG), fetch a page as markdown, capture the screen, open a URL in your default browser or launch a macOS app by name, and send you a notification.

• Hotkeys (no accessibility permission needed — Carbon, not event taps):

  Key	Action
  ⌥Space	open/close chat (input focused)
  ⌥⇧Space	capture the screen now, attach it to your next message
  ⌥⌘Space	audio input (stub — next milestone)

• Screenshots for a small model — captures are downscaled to ~1440px before the model sees them (an 8MP image would cost thousands of vision tokens on a 9–12B model).

Requirements

• macOS 14+ (developed and tested on macOS 15.6.1, Apple Silicon)
• Xcode Command Line Tools (full Xcode is not required — the Swift UI compiles with plain swiftc)
• Rust (rustup or xcodebuild-bundled; cargo on PATH)
• An OpenAI-compatible chat-completions endpoint (e.g. a local llama.cpp llama-server or MLX server). For image input, use a multimodal model.

Install

git clone https://git.kokoham.com/sleepy/mac-assist.git
cd mac-assist
./scripts/build.sh        # cargo release + swiftc + bundle + sign (~1 min)
open dist/MacAssist.app

That's it — the sparkle icon appears in your menu bar. First launch:

0. Setup wizard — offers to clone & build llama.cpp into ~/.llama and pick a model preset (writes ~/.llama/serve.sh + points the config at http://127.0.0.1:17777/v1), or point MacAssist at a server you already run. Re-open it any time: right-click the sparkle → "Setup Wizard…". "Configure Server…" re-tunes model/context/KV-quant, saves a fresh serve.sh, and starts/stops the server.
1. Notification prompt — allow it (email triage pings). Requires the Apple Development signing identity in your keychain (free Apple dev certificate; build.sh picks it up automatically). Without it the build falls back to ad-hoc signing and notifications are disabled on macOS 15 (see docs/PLAN.md, decision D9) — everything else still works.
2. Mail automation prompt — appears on the first triage (within a minute); allow "MacAssist wants to control Mail".
3. Screen Recording prompt — appears on the first screenshot; allow it.

Tip: ad-hoc/development signatures can invalidate TCC grants when the binary is rebuilt. If a permission "disappears" after a rebuild, re-allow it in System Settings → Privacy & Security.

Usage

Chat. Click the sparkle (or press ⌥Space) and type. Examples that use the tools:

• "What were my last 5 emails?"
• "Read email #3 in detail"
• "Did I ever get an email about the studio lease? Dig through older mail."
• "Check today's Hacker News" → opens it in your default browser
• "Open Steam, I want to check discounts" → launches the app (resolved from /Applications, /System/Applications, ~/Applications — case-insensitive, tolerant of small typos)
• "Look at my screen and tell me what's broken" (or press ⌥⇧Space first, then type your question — the shot is already attached)

Screenshot hotkey. ⌥⇧Space captures immediately (background), shows a "📎 Screenshot attached" line, and the image ships with your next message — the model starts on it the moment you hit Enter.

New chat. The pencil button (top-right) starts a fresh core session with an empty model context. Use it when the conversation gets long — the dim "ctx 4.2k"-style counter in the header shows how much context the last request sent.

Right-click the sparkle → "Settings…" (reveals the data directory in Finder) or "Quit MacAssist".

Email triage. Runs at startup and hourly. It only notifies when the model verdicts mail as noteworthy ("N noteworthy emails" + top line); silence is the default. Every triage run appends to a local daily log and a seen-store, which also powers email_search's "archive digging".

Configuration

Everything lives in one data directory:

~/Library/Application Support/MacAssist/
  config.toml        # the config (created from defaults on first launch)
  prompts/           # system.md, tools.md, triage.md — edit freely
  memory/            # triage logs, (later: profile/facts)
  state/             # seen-emails.jsonl

Override the location with MACASSIST_DATA_DIR. All files are plain text; edit config.toml / the prompts and restart the app (quit via the right-click menu, open dist/MacAssist.app again).

A fully commented default is in config.example.toml:

[model]
base_url = "https://chat.kokoham.com/v1"   # any OpenAI-compatible endpoint
model = "bazzite"                          # your model name
api_key = "your-api-key"                     # whatever your server wants
tool_protocol = "auto"                     # "auto" | "native" | "text"
max_tool_iterations = 8

[email]
enabled = true
triage_interval_hours = 1
triage_on_startup = true
max_unseen_per_run = 40
body_chars = 1500

[web]
search_url = "https://search.kokoham.com"  # SearXNG with JSON API
search_path = "/search"
kuri_fetch = "~/.local/bin/kuri-fetch"     # JS-capable fetcher ("" disables)
fetch_chars = 20000
timeout_secs = 25

Notes:

• The defaults in config.example.toml point at a development endpoint — point [model] at your own server.
• tool_protocol = "auto" sends native tool calls and falls back to a text-tag protocol if the server rejects them (weak servers do).
• [web].kuri_fetch: any binary that reads a URL from argv and prints markdown; the built-in HTTP+tag-stripping fetcher is the fallback.
• [ui] is reserved (hotkeys are fixed in the app for now).

Project layout

mac-assist/
  core/                    # Rust binary macassist-core — all machinery
  ui/                      # Swift/AppKit menu-bar app — all visible UI
  prompts/                 # bundled default prompts
  config.example.toml      # the one config file
  scripts/                 # build.sh, run.sh (repl mode, --selftest)
  docs/                    # VISION, PLAN (architecture/IPC/milestones), ONBOARDING
  dist/                    # built MacAssist.app (not committed)

Development

./scripts/build.sh                  # full rebuild (core + UI + bundle + sign)
cargo test -p macassist-core         # core unit tests (parsers, protocols, guards)
scripts/run.sh --selftest            # headless end-to-end check (no UI, no model needed)
target/release/macassist-core repl   # terminal REPL against the same core (no UI)

Rebuilding the Swift UI alone (fast path):

swiftc -O -o dist/MacAssist.app/Contents/MacAssist ui/Sources/MacAssist/*.swift
codesign --force --sign "Apple Development: …" dist/MacAssist.app

For agents: start at docs/ONBOARDING.md (hard guardrails, build/test commands, TCC debugging notes). Architecture, IPC wire format, tool spec, and every decision (D1–D12) are in docs/PLAN.md.

Status & roadmap

• Done (M0–M2 + UI): hybrid process model, dual tool protocol, streaming chat, email triage end-to-end, notifications (UN, dev-certificate signed), email tools, screenshot tool, open tool, hotkeys, chat-styled popover.
• Next (M3): memory tools + session persistence + consolidation.
• Then (M4): audio input (⌥⌘Space), notification-click context.
• Later (M5): PDF drag-drop, larger-mailbox (Gmail/IMAP) hook — email_search is where server-side search will slot in.