Final README with accurate LiteRT-LM integration guidance

- Documents the C++ bridge approach based on Google's actual implementation - Explains MediaPipe (deprecated) vs C++ bridge vs Swift APIs (coming soon) - Clear instructions for obtaining LiteRT-LM binaries - Project structure and architecture diagram
2026-04-06 14:55:20 +02:00
parent 45d43f2645
commit 08f1aac758
1 changed files with 191 additions and 0 deletions
@@ -0,0 +1,191 @@
+# Sleepy Agent iOS
+
+A Swift port of Sleepy Agent - a fully local AI assistant for iOS powered by Google's Gemma 4 models.
+
+## Status: Developer Build (Work in Progress)
+
+**Architecture complete** - waiting on LiteRT-LM integration.
+
+## Quick Facts
+
+- **19 Swift files** (~4,500 LOC)
+- **3 Objective-C++ bridge files** for LiteRT-LM integration
+- **5 documentation files**
+- **iOS 15.0+** target
+- **SwiftUI** for UI
+
+## What's Working
+
+✅ Complete SwiftUI interface (chat, settings, navigation)  
+✅ Audio recording & playback  
+✅ Text-to-Speech (TTS)  
+✅ Web search (SearXNG)  
+✅ Model download manager  
+✅ Conversation persistence  
+✅ Image picker  
+⚠️ LLM inference (stubbed - needs LiteRT-LM binaries)  
+
+## The LiteRT-LM Situation
+
+Based on research of Google's repositories:
+
+| Option | Status | Gemma 4 | Recommendation |
+|--------|--------|---------|----------------|
+| MediaPipe Tasks Swift | Deprecated | ❌ | Short-term only |
+| **C++ Bridge** ⭐ | **Available** | ✅ | **Recommended** |
+| Native Swift APIs | Coming soon | ✅ | Wait if possible |
+
+### Google's Approach
+
+The [Google AI Edge Gallery](https://apps.apple.com/us/app/google-ai-edge-gallery/id6749645337) iOS app uses a **C++ bridge** (confirmed by source code analysis):
+
+- [litert-samples](https://github.com/google-ai-edge/litert-samples) shows ObjC++ bridge pattern
+- MediaPipe has [working Swift API](https://github.com/google-ai-edge/mediapipe-samples/tree/main/examples/llm_inference/ios) but it's deprecated
+- [Issue #420](https://github.com/google-ai-edge/gallery/issues/420) confirms iOS source isn't public
+
+### Our Implementation
+
+**Objective-C++ Bridge Files** (included):
+- `SleepyAgent/Inference/Bridge/LlmEngineBridge.h` - Header
+- `SleepyAgent/Inference/Bridge/LlmEngineBridge.mm` - Implementation
+- `SleepyAgent/Inference/Bridge/SleepyAgent-Bridging-Header.h` - Swift bridge
+
+See [LITERT_INTEGRATION.md](LITERT_INTEGRATION.md) for details.
+
+## Building
+
+### Prerequisites
+
+- macOS with Xcode 15+
+- iOS 15.0+ device or simulator
+- CocoaPods (`sudo gem install cocoapods`)
+
+### Quick Start
+
+```bash
+cd sleepy_agent_ios
+
+# Check status
+make check
+
+# Setup project
+make setup  # Installs CocoaPods
+
+# Open in Xcode
+open SleepyAgent.xcworkspace  # or Package.swift
+```
+
+### To Enable LLM Inference
+
+You need to obtain LiteRT-LM binaries:
+
+1. **Build from source**: https://github.com/google-ai-edge/LiteRT-LM
+2. **Download releases**: Check GitHub releases page
+3. **Contact Google**: For prebuilt iOS binaries
+
+Then:
+1. Add LiteRT-LM headers to Xcode project
+2. Link `liblitert_lm.a` static library
+3. Configure bridging header in Build Settings
+4. Uncomment C++ code in `LlmEngineBridge.mm`
+
+See [BUILD.md](BUILD.md) and [LITERT_INTEGRATION.md](LITERT_INTEGRATION.md) for full instructions.
+
+## Project Structure
+
+```
+SleepyAgent/
+├── App/                        # Entry point, app lifecycle
+├── Core/                       # Models, DI container
+│   └── Models/Message.swift    # Data models
+├── Inference/                  # LLM engine
+│   ├── Bridge/                 # ⭐ ObjC++ LiteRT-LM bridge
+│   ├── Agent.swift             # Tool calling, chat logic
+│   ├── LlmEngine.swift         # Swift engine interface
+│   └── ...
+├── Services/                   # Audio, network, storage
+│   ├── AudioRecorder.swift
+│   ├── TtsService.swift
+│   ├── WebSearchService.swift
+│   └── ...
+└── UI/                         # SwiftUI views
+    ├── Views/                  # MainView, SettingsView, etc.
+    └── ViewModels/             # State management
+```
+
+## Architecture
+
+```
+┌─────────────────────────────────────────┐
+│           SwiftUI Views                 │
+│     (MainView, SettingsView, ...)       │
+└─────────────────┬───────────────────────┘
+                  │
+┌─────────────────▼───────────────────────┐
+│       Swift ViewModels                  │
+│   (MainViewModel, SettingsViewModel)    │
+└─────────────────┬───────────────────────┘
+                  │
+┌─────────────────▼───────────────────────┐
+│      Swift LlmEngine (Actor)            │
+│     (async/await, thread-safe)          │
+└─────────────────┬───────────────────────┘
+                  │
+┌─────────────────▼───────────────────────┐
+│   Objective-C++ LlmEngineBridge         │
+│   (ObjC++ wrapper for LiteRT-LM)        │
+└─────────────────┬───────────────────────┘
+                  │
+┌─────────────────▼───────────────────────┐
+│      C++ LiteRT-LM Library              │
+│   (Google's on-device LLM engine)       │
+└─────────────────────────────────────────┘
+```
+
+## Documentation
+
+- **[BUILD.md](BUILD.md)** - Build instructions, troubleshooting
+- **[LITERT_INTEGRATION.md](LITERT_INTEGRATION.md)** - LiteRT-LM integration details
+- **[LITERT_IOS_STATUS.md](LITERT_IOS_STATUS.md)** - Current state of LiteRT on iOS
+- **[PROJECT_STATUS.md](PROJECT_STATUS.md)** - Overall project status
+- **[FILES.md](FILES.md)** - Complete file listing
+
+## Key Differences from Android
+
+| Feature | Android | iOS |
+|---------|---------|-----|
+| UI | Jetpack Compose | SwiftUI |
+| Async | Kotlin Coroutines | Swift Concurrency (async/await) |
+| DI | Manual | Manual |
+| Storage | DataStore + JSON | UserDefaults + FileManager |
+| LLM Engine | LiteRT-LM Kotlin | LiteRT-LM C++ via bridge |
+| Audio | AudioRecord | AVAudioEngine |
+| Background | WorkManager | URLSession (limited) |
+
+## Testing Without LLM
+
+The app works in "stub mode" for UI testing:
+
+1. Build and run
+2. Type any message
+3. See placeholder response explaining LiteRT-LM integration
+
+## Next Steps
+
+To complete the iOS port:
+
+1. Obtain LiteRT-LM iOS binaries (build or download)
+2. Add to Xcode project
+3. Configure build settings for C++
+4. Uncomment C++ code in bridge files
+5. Test with Gemma 4 E2B model
+
+## Need Help?
+
+- Open an issue: https://github.com/sleepyeldrazi/sleepy-agent/issues
+- Check [LITERT_INTEGRATION.md](LITERT_INTEGRATION.md) for troubleshooting
+- See Google's [litert-samples](https://github.com/google-ai-edge/litert-samples) for reference
+
+## License
+
+MIT License