sleepy_agent_ios/README.md

# Sleepy Agent iOS

A Swift port of Sleepy Agent - a fully local AI assistant for iOS powered by Google's Gemma 4 models.

## Status: Developer Build (Work in Progress)

**Architecture complete** - waiting on LiteRT-LM integration.

## Quick Facts

- **19 Swift files** (~4,500 LOC)
- **3 Objective-C++ bridge files** for LiteRT-LM integration
- **5 documentation files**
- **iOS 15.0+** target
- **SwiftUI** for UI

## What's Working

✅ Complete SwiftUI interface (chat, settings, navigation)  
✅ Audio recording & playback  
✅ Text-to-Speech (TTS)  
✅ Web search (SearXNG)  
✅ Model download manager  
✅ Conversation persistence  
✅ Image picker  
⚠️ LLM inference (stubbed - needs LiteRT-LM binaries)  

## The LiteRT-LM Situation

Based on research of Google's repositories:

| Option | Status | Gemma 4 | Recommendation |
|--------|--------|---------|----------------|
| MediaPipe Tasks Swift | Deprecated | ❌ | Short-term only |
| **C++ Bridge** ⭐ | **Available** | ✅ | **Recommended** |
| Native Swift APIs | Coming soon | ✅ | Wait if possible |

### Google's Approach

The [Google AI Edge Gallery](https://apps.apple.com/us/app/google-ai-edge-gallery/id6749645337) iOS app uses a **C++ bridge** (confirmed by source code analysis):

- [litert-samples](https://github.com/google-ai-edge/litert-samples) shows ObjC++ bridge pattern
- MediaPipe has [working Swift API](https://github.com/google-ai-edge/mediapipe-samples/tree/main/examples/llm_inference/ios) but it's deprecated
- [Issue #420](https://github.com/google-ai-edge/gallery/issues/420) confirms iOS source isn't public

### Our Implementation

**Objective-C++ Bridge Files** (included):
- `SleepyAgent/Inference/Bridge/LlmEngineBridge.h` - Header
- `SleepyAgent/Inference/Bridge/LlmEngineBridge.mm` - Implementation
- `SleepyAgent/Inference/Bridge/SleepyAgent-Bridging-Header.h` - Swift bridge

See [LITERT_INTEGRATION.md](LITERT_INTEGRATION.md) for details.

## Building

### Prerequisites

- macOS with Xcode 15+
- iOS 15.0+ device or simulator
- CocoaPods (`sudo gem install cocoapods`)

### Quick Start

```bash
cd sleepy_agent_ios

# Check status
make check

# Setup project
make setup  # Installs CocoaPods

# Open in Xcode
open SleepyAgent.xcworkspace  # or Package.swift
```

### To Enable LLM Inference

You need to obtain LiteRT-LM binaries:

1. **Build from source**: https://github.com/google-ai-edge/LiteRT-LM
2. **Download releases**: Check GitHub releases page
3. **Contact Google**: For prebuilt iOS binaries

Then:
1. Add LiteRT-LM headers to Xcode project
2. Link `liblitert_lm.a` static library
3. Configure bridging header in Build Settings
4. Uncomment C++ code in `LlmEngineBridge.mm`

See [BUILD.md](BUILD.md) and [LITERT_INTEGRATION.md](LITERT_INTEGRATION.md) for full instructions.

## Project Structure

```
SleepyAgent/
├── App/                        # Entry point, app lifecycle
├── Core/                       # Models, DI container
│   └── Models/Message.swift    # Data models
├── Inference/                  # LLM engine
│   ├── Bridge/                 # ⭐ ObjC++ LiteRT-LM bridge
│   ├── Agent.swift             # Tool calling, chat logic
│   ├── LlmEngine.swift         # Swift engine interface
│   └── ...
├── Services/                   # Audio, network, storage
│   ├── AudioRecorder.swift
│   ├── TtsService.swift
│   ├── WebSearchService.swift
│   └── ...
└── UI/                         # SwiftUI views
    ├── Views/                  # MainView, SettingsView, etc.
    └── ViewModels/             # State management
```

## Architecture

```
┌─────────────────────────────────────────┐
│           SwiftUI Views                 │
│     (MainView, SettingsView, ...)       │
└─────────────────┬───────────────────────┘
                  │
┌─────────────────▼───────────────────────┐
│       Swift ViewModels                  │
│   (MainViewModel, SettingsViewModel)    │
└─────────────────┬───────────────────────┘
                  │
┌─────────────────▼───────────────────────┐
│      Swift LlmEngine (Actor)            │
│     (async/await, thread-safe)          │
└─────────────────┬───────────────────────┘
                  │
┌─────────────────▼───────────────────────┐
│   Objective-C++ LlmEngineBridge         │
│   (ObjC++ wrapper for LiteRT-LM)        │
└─────────────────┬───────────────────────┘
                  │
┌─────────────────▼───────────────────────┐
│      C++ LiteRT-LM Library              │
│   (Google's on-device LLM engine)       │
└─────────────────────────────────────────┘
```

## Documentation

- **[BUILD.md](BUILD.md)** - Build instructions, troubleshooting
- **[LITERT_INTEGRATION.md](LITERT_INTEGRATION.md)** - LiteRT-LM integration details
- **[LITERT_IOS_STATUS.md](LITERT_IOS_STATUS.md)** - Current state of LiteRT on iOS
- **[PROJECT_STATUS.md](PROJECT_STATUS.md)** - Overall project status
- **[FILES.md](FILES.md)** - Complete file listing

## Key Differences from Android

| Feature | Android | iOS |
|---------|---------|-----|
| UI | Jetpack Compose | SwiftUI |
| Async | Kotlin Coroutines | Swift Concurrency (async/await) |
| DI | Manual | Manual |
| Storage | DataStore + JSON | UserDefaults + FileManager |
| LLM Engine | LiteRT-LM Kotlin | LiteRT-LM C++ via bridge |
| Audio | AudioRecord | AVAudioEngine |
| Background | WorkManager | URLSession (limited) |

## Testing Without LLM

The app works in "stub mode" for UI testing:

1. Build and run
2. Type any message
3. See placeholder response explaining LiteRT-LM integration

## Next Steps

To complete the iOS port:

1. Obtain LiteRT-LM iOS binaries (build or download)
2. Add to Xcode project
3. Configure build settings for C++
4. Uncomment C++ code in bridge files
5. Test with Gemma 4 E2B model

## Need Help?

- Open an issue: https://github.com/sleepyeldrazi/sleepy-agent/issues
- Check [LITERT_INTEGRATION.md](LITERT_INTEGRATION.md) for troubleshooting
- See Google's [litert-samples](https://github.com/google-ai-edge/litert-samples) for reference

## License

MIT License