sleepy_agent_ios/README.md

Sleepy Agent iOS

A Swift port of Sleepy Agent - a fully local AI assistant for iOS powered by Google's Gemma 4 models.

Status: Developer Build (Work in Progress)

Architecture complete - waiting on LiteRT-LM integration.

Quick Facts

• 19 Swift files (~4,500 LOC)
• 3 Objective-C++ bridge files for LiteRT-LM integration
• 5 documentation files
• iOS 15.0+ target
• SwiftUI for UI

What's Working

✅ Complete SwiftUI interface (chat, settings, navigation)
✅ Audio recording & playback
✅ Text-to-Speech (TTS)
✅ Web search (SearXNG)
✅ Model download manager
✅ Conversation persistence
✅ Image picker
⚠️ LLM inference (stubbed - needs LiteRT-LM binaries)

The LiteRT-LM Situation

Based on research of Google's repositories:

Option	Status	Gemma 4	Recommendation
MediaPipe Tasks Swift	Deprecated	❌	Short-term only
C++ Bridge ⭐	Available	✅	Recommended
Native Swift APIs	Coming soon	✅	Wait if possible

Google's Approach

The Google AI Edge Gallery iOS app uses a C++ bridge (confirmed by source code analysis):

• litert-samples shows ObjC++ bridge pattern
• MediaPipe has working Swift API but it's deprecated
• Issue #420 confirms iOS source isn't public

Our Implementation

Objective-C++ Bridge Files (included):

• SleepyAgent/Inference/Bridge/LlmEngineBridge.h - Header
• SleepyAgent/Inference/Bridge/LlmEngineBridge.mm - Implementation
• SleepyAgent/Inference/Bridge/SleepyAgent-Bridging-Header.h - Swift bridge

See LITERT_INTEGRATION.md for details.

Building

Prerequisites

• macOS with Xcode 15+
• iOS 15.0+ device or simulator
• CocoaPods (sudo gem install cocoapods)

Quick Start

cd sleepy_agent_ios

# Check status
make check

# Setup project
make setup  # Installs CocoaPods

# Open in Xcode
open SleepyAgent.xcworkspace  # or Package.swift

To Enable LLM Inference

You need to obtain LiteRT-LM binaries:

1. Build from source: https://github.com/google-ai-edge/LiteRT-LM
2. Download releases: Check GitHub releases page
3. Contact Google: For prebuilt iOS binaries

Then:

1. Add LiteRT-LM headers to Xcode project
2. Link liblitert_lm.a static library
3. Configure bridging header in Build Settings
4. Uncomment C++ code in LlmEngineBridge.mm

See BUILD.md and LITERT_INTEGRATION.md for full instructions.

Project Structure

SleepyAgent/
├── App/                        # Entry point, app lifecycle
├── Core/                       # Models, DI container
│   └── Models/Message.swift    # Data models
├── Inference/                  # LLM engine
│   ├── Bridge/                 # ⭐ ObjC++ LiteRT-LM bridge
│   ├── Agent.swift             # Tool calling, chat logic
│   ├── LlmEngine.swift         # Swift engine interface
│   └── ...
├── Services/                   # Audio, network, storage
│   ├── AudioRecorder.swift
│   ├── TtsService.swift
│   ├── WebSearchService.swift
│   └── ...
└── UI/                         # SwiftUI views
    ├── Views/                  # MainView, SettingsView, etc.
    └── ViewModels/             # State management

Architecture

┌─────────────────────────────────────────┐
│           SwiftUI Views                 │
│     (MainView, SettingsView, ...)       │
└─────────────────┬───────────────────────┘
                  │
┌─────────────────▼───────────────────────┐
│       Swift ViewModels                  │
│   (MainViewModel, SettingsViewModel)    │
└─────────────────┬───────────────────────┘
                  │
┌─────────────────▼───────────────────────┐
│      Swift LlmEngine (Actor)            │
│     (async/await, thread-safe)          │
└─────────────────┬───────────────────────┘
                  │
┌─────────────────▼───────────────────────┐
│   Objective-C++ LlmEngineBridge         │
│   (ObjC++ wrapper for LiteRT-LM)        │
└─────────────────┬───────────────────────┘
                  │
┌─────────────────▼───────────────────────┐
│      C++ LiteRT-LM Library              │
│   (Google's on-device LLM engine)       │
└─────────────────────────────────────────┘

Documentation

• BUILD.md - Build instructions, troubleshooting
• LITERT_INTEGRATION.md - LiteRT-LM integration details
• LITERT_IOS_STATUS.md - Current state of LiteRT on iOS
• PROJECT_STATUS.md - Overall project status
• FILES.md - Complete file listing

Key Differences from Android

Feature	Android	iOS
UI	Jetpack Compose	SwiftUI
Async	Kotlin Coroutines	Swift Concurrency (async/await)
DI	Manual	Manual
Storage	DataStore + JSON	UserDefaults + FileManager
LLM Engine	LiteRT-LM Kotlin	LiteRT-LM C++ via bridge
Audio	AudioRecord	AVAudioEngine
Background	WorkManager	URLSession (limited)

Testing Without LLM

The app works in "stub mode" for UI testing:

1. Build and run
2. Type any message
3. See placeholder response explaining LiteRT-LM integration

Next Steps

To complete the iOS port:

1. Obtain LiteRT-LM iOS binaries (build or download)
2. Add to Xcode project
3. Configure build settings for C++
4. Uncomment C++ code in bridge files
5. Test with Gemma 4 E2B model

Need Help?

• Open an issue: https://github.com/sleepyeldrazi/sleepy-agent/issues
• Check LITERT_INTEGRATION.md for troubleshooting
• See Google's litert-samples for reference

License

MIT License