Initial iOS port - Complete source code and build system
- 19 Swift source files (~4900 lines) - Complete UI with SwiftUI (MainView, SettingsView, MessageBubble, InputBar) - Inference layer (LlmEngine, Agent, ToolCalling, ConversationContext) - Services (Audio, TTS, WebSearch, ModelDownload, Storage) - Build system: Makefile, Package.swift, Podfile - Documentation: BUILD.md, plan.md, PROJECT_STATUS.md - Ready for Xcode build - just need LiteRT dependency added
This commit is contained in:
@@ -0,0 +1,197 @@
|
||||
# Building Sleepy Agent iOS
|
||||
|
||||
## Prerequisites
|
||||
|
||||
1. **macOS with Apple Silicon (M1/M2/M3/M4)**
|
||||
2. **Xcode 15 or later** (from App Store)
|
||||
3. **iOS 15.0+ device** or simulator
|
||||
4. **CocoaPods** (`sudo gem install cocoapods`)
|
||||
|
||||
## Quick Start
|
||||
|
||||
### Option 1: Xcode GUI (Easiest)
|
||||
|
||||
1. **Open the Swift Package in Xcode:**
|
||||
```bash
|
||||
cd sleepy_agent_ios
|
||||
open Package.swift
|
||||
```
|
||||
|
||||
2. **Xcode will resolve the package** and create a project
|
||||
|
||||
3. **Add LiteRT dependency:**
|
||||
- File → Add Package Dependencies
|
||||
- Search for TensorFlow Lite or LiteRT
|
||||
- Or manually download the framework
|
||||
|
||||
4. **Configure signing:**
|
||||
- Select the project
|
||||
- Signing & Capabilities → Team → Select your Apple ID
|
||||
- Bundle Identifier: `com.sleepy.agent` (or your own)
|
||||
|
||||
5. **Build and run:**
|
||||
- Select target device (your iPhone or a simulator)
|
||||
- Press Cmd+R
|
||||
|
||||
### Option 2: CocoaPods + xcodebuild (Command Line)
|
||||
|
||||
1. **Install CocoaPods dependencies:**
|
||||
```bash
|
||||
cd sleepy_agent_ios
|
||||
pod install
|
||||
```
|
||||
|
||||
2. **Open the workspace in Xcode:**
|
||||
```bash
|
||||
open SleepyAgent.xcworkspace
|
||||
```
|
||||
|
||||
3. **Or build from command line:**
|
||||
```bash
|
||||
make build # Debug build
|
||||
make archive # Release archive
|
||||
```
|
||||
|
||||
### Option 3: Create Fresh Xcode Project
|
||||
|
||||
If the above doesn't work, create a new project:
|
||||
|
||||
```bash
|
||||
# Make the script executable
|
||||
chmod +x create-project.sh
|
||||
|
||||
# Run the setup script
|
||||
./create-project.sh
|
||||
```
|
||||
|
||||
Then manually copy the Swift files from `SleepyAgent/` into your new project.
|
||||
|
||||
## Creating an .ipa File
|
||||
|
||||
### For Development/Debugging:
|
||||
|
||||
1. **Archive the app:**
|
||||
```bash
|
||||
make archive
|
||||
```
|
||||
|
||||
2. **Open the archive in Xcode Organizer:**
|
||||
```bash
|
||||
open build/SleepyAgent.xcarchive
|
||||
```
|
||||
|
||||
3. **Export as IPA:**
|
||||
- In Organizer, select the archive
|
||||
- Click "Distribute App"
|
||||
- Choose "Development"
|
||||
- Follow prompts to export
|
||||
|
||||
### For Sideloading (No Developer Account):
|
||||
|
||||
Use [AltStore](https://altstore.io) or similar tools to install the .ipa without a paid developer account.
|
||||
|
||||
### For Jailbroken Devices:
|
||||
|
||||
```bash
|
||||
# Build and package without signing
|
||||
make build-release
|
||||
cd build/Release-iphoneos
|
||||
mkdir -p Payload/SleepyAgent.app
|
||||
cp -r SleepyAgent.app/* Payload/SleepyAgent.app/
|
||||
zip -r SleepyAgent.ipa Payload
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### "No such module 'TensorFlowLite'"
|
||||
|
||||
LiteRT-LM needs to be added manually. Options:
|
||||
|
||||
1. **Use CocoaPods:** Edit `Podfile` and add the correct pod
|
||||
2. **Use Swift Package Manager:** Add TensorFlow Lite as a dependency
|
||||
3. **Manual framework:** Download from [TensorFlow releases](https://github.com/tensorflow/tensorflow/releases)
|
||||
|
||||
### Signing Issues
|
||||
|
||||
```bash
|
||||
# Check available signing identities
|
||||
security find-identity -v -p codesigning
|
||||
|
||||
# For development builds, use automatic signing in Xcode
|
||||
# For distribution, you need a paid Apple Developer account ($99/year)
|
||||
```
|
||||
|
||||
### "Could not find iOS SDK"
|
||||
|
||||
```bash
|
||||
# Select Xcode command line tools
|
||||
sudo xcode-select --switch /Applications/Xcode.app/Contents/Developer
|
||||
|
||||
# Accept license
|
||||
sudo xcodebuild -license accept
|
||||
```
|
||||
|
||||
### Model Download Issues
|
||||
|
||||
The app downloads models from HuggingFace. If downloads fail:
|
||||
|
||||
1. Manually download from: https://huggingface.co/litert-community
|
||||
2. Place in: Files app → On My iPhone → Sleepy Agent → models/
|
||||
|
||||
## File Structure
|
||||
|
||||
```
|
||||
sleepy_agent_ios/
|
||||
├── SleepyAgent/ # Source code
|
||||
│ ├── App/ # Entry point
|
||||
│ ├── Core/ # Models, DI
|
||||
│ ├── Inference/ # LLM engine
|
||||
│ ├── Services/ # Audio, network, etc.
|
||||
│ ├── UI/ # SwiftUI views
|
||||
│ ├── Info.plist # App config
|
||||
│ └── SleepyAgent.entitlements # Capabilities
|
||||
├── Package.swift # Swift Package Manager
|
||||
├── Podfile # CocoaPods config
|
||||
├── Makefile # Build automation
|
||||
└── BUILD.md # This file
|
||||
```
|
||||
|
||||
## Build Commands
|
||||
|
||||
```bash
|
||||
# Setup
|
||||
make setup # Install CocoaPods
|
||||
make check # Check environment
|
||||
|
||||
# Building
|
||||
make build # Debug build (simulator)
|
||||
make build-release # Release build (device)
|
||||
make archive # Create .xcarchive
|
||||
make ipa # Create .ipa
|
||||
|
||||
# Installation
|
||||
make install # Install to connected device
|
||||
|
||||
# Cleanup
|
||||
make clean # Clean build artifacts
|
||||
make clean-all # Full clean including Pods
|
||||
```
|
||||
|
||||
## Debugging on Device
|
||||
|
||||
1. Connect iPhone via USB
|
||||
2. Trust this computer on iPhone
|
||||
3. In Xcode: Window → Devices and Simulators
|
||||
4. Select your device
|
||||
5. Build and run (Cmd+R)
|
||||
|
||||
## Notes
|
||||
|
||||
- **LiteRT-LM iOS models**: https://huggingface.co/collections/litert-community/ios-models
|
||||
- **Physical device recommended** - Simulator can't test audio/camera well
|
||||
- **First launch** - Model loading may take 10-30 seconds depending on size
|
||||
- **RAM requirements** - E2B needs ~4GB free, E4B needs ~6GB free
|
||||
|
||||
## Need Help?
|
||||
|
||||
Open an issue at: https://github.com/sleepyeldrazi/sleepy-agent/issues
|
||||
@@ -0,0 +1,54 @@
|
||||
# Sleepy Agent iOS - File Structure
|
||||
|
||||
## Complete File List (19 Swift files)
|
||||
|
||||
### App (1 file)
|
||||
- `SleepyAgentApp.swift` - App entry point, AppDelegate, AudioSessionManager
|
||||
|
||||
### Core (2 files)
|
||||
- `Core/Models/Message.swift` - Message, ToolCall, ModelVariant models
|
||||
- `Core/DI/AppContainer.swift` - Manual dependency injection container
|
||||
|
||||
### Inference (4 files)
|
||||
- `Inference/LlmEngine.swift` - LiteRT-LM wrapper
|
||||
- `Inference/Agent.swift` - High-level agent with tool calling
|
||||
- `Inference/ConversationContext.swift` - Chat history management
|
||||
- `Inference/ToolCalling.swift` - Tool definitions and parsing
|
||||
|
||||
### Services (5 files)
|
||||
- `Services/AudioRecorder.swift` - Voice recording with VAD
|
||||
- `Services/TtsService.swift` - Text-to-speech (AVSpeechSynthesizer)
|
||||
- `Services/WebSearchService.swift` - SearXNG client
|
||||
- `Services/ModelDownloadService.swift` - Model download with progress
|
||||
- `Services/ConversationStorage.swift` - JSON persistence
|
||||
|
||||
### UI Views (4 files)
|
||||
- `UI/Views/MainView.swift` - Main chat interface with sidebar
|
||||
- `UI/Views/SettingsView.swift` - Settings screen
|
||||
- `UI/Views/MessageBubble.swift` - Chat message component
|
||||
- `UI/Views/InputBar.swift` - Text input with buttons
|
||||
|
||||
### UI ViewModels (2 files)
|
||||
- `UI/ViewModels/MainViewModel.swift` - Main screen logic
|
||||
- `UI/ViewModels/SettingsViewModel.swift` - Settings logic
|
||||
|
||||
### Documentation
|
||||
- `README.md` - Project overview
|
||||
- `plan.md` - Development plan with lessons learned
|
||||
- `FILES.md` - This file
|
||||
|
||||
## Missing Components
|
||||
|
||||
The following still need implementation:
|
||||
1. **Podfile** - For LiteRT dependency
|
||||
2. **Info.plist** - App permissions (camera, microphone, photo library)
|
||||
3. **Assets** - App icons, colors
|
||||
4. **LiteRT Integration** - The actual LiteRT-LM Swift bindings need proper integration
|
||||
|
||||
## Next Steps to Build
|
||||
|
||||
1. Create Xcode project or use Swift Package Manager
|
||||
2. Add LiteRT dependency via CocoaPods
|
||||
3. Configure signing and bundle ID
|
||||
4. Add required permissions to Info.plist
|
||||
5. Test on physical device
|
||||
@@ -0,0 +1,179 @@
|
||||
# Sleepy Agent iOS Build Makefile
|
||||
# Requires: Xcode 15+, iOS 15.0+ SDK
|
||||
|
||||
# Configuration
|
||||
APP_NAME = SleepyAgent
|
||||
BUNDLE_ID = com.sleepy.agent
|
||||
SCHEME = SleepyAgent
|
||||
WORKSPACE = $(APP_NAME).xcworkspace
|
||||
PROJECT = $(APP_NAME).xcodeproj
|
||||
|
||||
# Build directories
|
||||
BUILD_DIR = build
|
||||
DERIVED_DATA = $(BUILD_DIR)/DerivedData
|
||||
ARCHIVE_PATH = $(BUILD_DIR)/SleepyAgent.xcarchive
|
||||
IPA_PATH = $(BUILD_DIR)/SleepyAgent.ipa
|
||||
|
||||
# Default: Show help
|
||||
.PHONY: help
|
||||
help:
|
||||
@echo "Sleepy Agent iOS Build System"
|
||||
@echo ""
|
||||
@echo "Available targets:"
|
||||
@echo " setup - Install dependencies (CocoaPods)"
|
||||
@echo " project - Generate Xcode project"
|
||||
@echo " build - Build debug version"
|
||||
@echo " build-release - Build release version"
|
||||
@echo " archive - Create archive for distribution"
|
||||
@echo " ipa - Create .ipa file (requires signing)"
|
||||
@echo " install - Install to connected device"
|
||||
@echo " clean - Clean build artifacts"
|
||||
@echo " all - Setup + build"
|
||||
@echo ""
|
||||
@echo "Prerequisites:"
|
||||
@echo " - Xcode 15+ installed"
|
||||
@echo " - iOS 15.0+ SDK"
|
||||
@echo " - CocoaPods installed: sudo gem install cocoapods"
|
||||
@echo " - Valid signing certificate for device/Archive builds"
|
||||
|
||||
# Setup dependencies
|
||||
.PHONY: setup
|
||||
setup:
|
||||
@echo "Installing CocoaPods dependencies..."
|
||||
pod install
|
||||
|
||||
# Generate project (if using a project generator)
|
||||
.PHONY: project
|
||||
project:
|
||||
@echo "Opening project in Xcode..."
|
||||
@echo "Note: If $(WORKSPACE) doesn't exist, run 'make setup' first"
|
||||
@if [ -f "$(WORKSPACE)" ]; then \
|
||||
open "$(WORKSPACE)"; \
|
||||
else \
|
||||
open "$(PROJECT)"; \
|
||||
fi
|
||||
|
||||
# Build debug for simulator
|
||||
.PHONY: build
|
||||
build:
|
||||
@echo "Building Sleepy Agent (Debug)..."
|
||||
xcodebuild \
|
||||
-workspace "$(WORKSPACE)" \
|
||||
-scheme "$(SCHEME)" \
|
||||
-configuration Debug \
|
||||
-sdk iphonesimulator \
|
||||
-derivedDataPath "$(DERIVED_DATA)" \
|
||||
build
|
||||
|
||||
# Build release
|
||||
.PHONY: build-release
|
||||
build-release:
|
||||
@echo "Building Sleepy Agent (Release)..."
|
||||
xcodebuild \
|
||||
-workspace "$(WORKSPACE)" \
|
||||
-scheme "$(SCHEME)" \
|
||||
-configuration Release \
|
||||
-sdk iphoneos \
|
||||
-derivedDataPath "$(DERIVED_DATA)" \
|
||||
build
|
||||
|
||||
# Create archive
|
||||
.PHONY: archive
|
||||
archive:
|
||||
@echo "Creating archive..."
|
||||
@mkdir -p $(BUILD_DIR)
|
||||
xcodebuild \
|
||||
-workspace "$(WORKSPACE)" \
|
||||
-scheme "$(SCHEME)" \
|
||||
-configuration Release \
|
||||
-sdk iphoneos \
|
||||
-archivePath "$(ARCHIVE_PATH)" \
|
||||
archive
|
||||
@echo "Archive created at: $(ARCHIVE_PATH)"
|
||||
|
||||
# Create IPA (requires proper signing)
|
||||
.PHONY: ipa
|
||||
ipa: archive
|
||||
@echo "Creating IPA..."
|
||||
@echo "NOTE: This requires a valid signing identity and provisioning profile"
|
||||
@echo "For developer-only builds, use 'make archive' and install via Xcode"
|
||||
|
||||
# Method 1: Using xcodebuild -exportArchive
|
||||
xcodebuild -exportArchive \
|
||||
-archivePath "$(ARCHIVE_PATH)" \
|
||||
-exportPath "$(BUILD_DIR)/Export" \
|
||||
-exportOptionsPlist exportOptions.plist \
|
||||
|| echo "Export failed. Check your signing configuration."
|
||||
|
||||
@echo ""
|
||||
@echo "IPA export attempted. Check $(BUILD_DIR)/Export/"
|
||||
|
||||
# Install to connected device (requires device to be connected and trusted)
|
||||
.PHONY: install
|
||||
install:
|
||||
@echo "Installing to connected device..."
|
||||
@if [ -z "$$(xcrun devicectl list devices 2>/dev/null | grep -i 'iPhone\|iPad')" ]; then \
|
||||
echo "ERROR: No device detected. Connect your iPhone/iPad and trust this computer."; \
|
||||
exit 1; \
|
||||
fi
|
||||
|
||||
xcodebuild \
|
||||
-workspace "$(WORKSPACE)" \
|
||||
-scheme "$(SCHEME)" \
|
||||
-configuration Debug \
|
||||
-destination 'generic/platform=iOS' \
|
||||
-derivedDataPath "$(DERIVED_DATA)" \
|
||||
build
|
||||
|
||||
@echo "App built. Use Xcode to install to device, or use:"
|
||||
@echo " xcrun devicectl device install app --device <device-id> <path-to-app>"
|
||||
|
||||
# Clean build artifacts
|
||||
.PHONY: clean
|
||||
clean:
|
||||
@echo "Cleaning build artifacts..."
|
||||
rm -rf $(BUILD_DIR)
|
||||
xcodebuild clean -workspace "$(WORKSPACE)" -scheme "$(SCHEME)" 2>/dev/null || true
|
||||
|
||||
# Full clean including Pods
|
||||
.PHONY: clean-all
|
||||
clean-all: clean
|
||||
@echo "Removing Pods directory..."
|
||||
rm -rf Pods
|
||||
rm -rf "$(WORKSPACE)"
|
||||
rm -rf "$(PROJECT)/xcuserdata"
|
||||
rm -rf "$(PROJECT)/project.xcworkspace"
|
||||
|
||||
# Everything
|
||||
.PHONY: all
|
||||
all: setup build
|
||||
@echo "Build complete!"
|
||||
|
||||
# Check environment
|
||||
.PHONY: check
|
||||
check:
|
||||
@echo "Checking build environment..."
|
||||
@echo ""
|
||||
@echo "Xcode version:"
|
||||
xcodebuild -version 2>/dev/null || echo "ERROR: Xcode not found"
|
||||
@echo ""
|
||||
@echo "Swift version:"
|
||||
swift --version 2>/dev/null || echo "ERROR: Swift not found"
|
||||
@echo ""
|
||||
@echo "CocoaPods version:"
|
||||
pod --version 2>/dev/null || echo "WARNING: CocoaPods not installed"
|
||||
@echo ""
|
||||
@echo "Available iOS SDKs:"
|
||||
xcodebuild -showsdks 2>/dev/null | grep ios || echo "ERROR: No iOS SDK found"
|
||||
@echo ""
|
||||
@echo "Connected devices:"
|
||||
xcrun devicectl list devices 2>/dev/null | head -20 || echo "No devices or devicectl not available"
|
||||
|
||||
# Debug: Show build settings
|
||||
.PHONY: settings
|
||||
settings:
|
||||
xcodebuild \
|
||||
-workspace "$(WORKSPACE)" \
|
||||
-scheme "$(SCHEME)" \
|
||||
-configuration Debug \
|
||||
-showBuildSettings | grep -E 'PRODUCT_BUNDLE_IDENTIFIER|CODE_SIGN|ARCH|SUPPORTED_PLATFORMS|SWIFT_VERSION'
|
||||
@@ -0,0 +1,164 @@
|
||||
# Sleepy Agent iOS - Project Status
|
||||
|
||||
## ✅ What's Complete
|
||||
|
||||
### Source Code (19 Swift files)
|
||||
- **App Layer** - App entry point, lifecycle management, audio session config
|
||||
- **Core Layer** - Models (Message, ToolCall, ModelVariant), DI container
|
||||
- **Inference Layer** - LlmEngine, Agent, ConversationContext, ToolCalling
|
||||
- **Services Layer** - AudioRecorder, TtsService, WebSearchService, ModelDownloadService, ConversationStorage
|
||||
- **UI Layer** - MainView, SettingsView, MessageBubble, InputBar + ViewModels
|
||||
|
||||
### Build System
|
||||
- `Package.swift` - Swift Package Manager manifest
|
||||
- `Podfile` - CocoaPods configuration for LiteRT
|
||||
- `Makefile` - Build automation (build, archive, ipa, install)
|
||||
- `Info.plist` - App configuration and permissions
|
||||
- `SleepyAgent.entitlements` - Sandbox and capability declarations
|
||||
- `exportOptions.plist` - IPA export configuration
|
||||
|
||||
### Scripts
|
||||
- `create-project.sh` - Project setup helper
|
||||
|
||||
### Documentation
|
||||
- `README.md` - Project overview
|
||||
- `BUILD.md` - Detailed build instructions
|
||||
- `plan.md` - Architecture decisions and porting notes
|
||||
- `FILES.md` - File structure reference
|
||||
|
||||
## ⚠️ What's Missing (Needs Xcode)
|
||||
|
||||
### Critical (Must Have)
|
||||
1. **Xcode Project File** (`.xcodeproj` or `.xcworkspace`)
|
||||
- Can't create without Xcode installed
|
||||
- Could try `swift package generate-xcodeproj` if you have it
|
||||
|
||||
2. **LiteRT Framework Integration**
|
||||
- Need to add TensorFlow Lite / LiteRT dependency
|
||||
- Options:
|
||||
- CocoaPods: `pod 'TensorFlowLiteSwift'`
|
||||
- Swift Package Manager: Add TensorFlow repo
|
||||
- Manual: Download and link framework
|
||||
|
||||
3. **Code Signing Setup**
|
||||
- Need Apple Developer account or free signing for personal use
|
||||
- Team ID in project settings
|
||||
|
||||
### Nice to Have
|
||||
4. **App Icons** - Create in Assets.xcassets
|
||||
5. **Launch Screen** - Storyboard or SwiftUI
|
||||
6. **Unit Tests** - Test targets
|
||||
|
||||
## How to Build (Step by Step)
|
||||
|
||||
Since Xcode isn't installed on this machine, here's what YOU need to do on your Mac:
|
||||
|
||||
### 1. Install Prerequisites
|
||||
```bash
|
||||
# Install Xcode from App Store
|
||||
# Then install CocoaPods
|
||||
sudo gem install cocoapods
|
||||
```
|
||||
|
||||
### 2. Setup Project
|
||||
```bash
|
||||
cd sleepy_agent_ios
|
||||
|
||||
# Option A: Use Swift Package Manager (easiest)
|
||||
open Package.swift
|
||||
|
||||
# Option B: Use CocoaPods
|
||||
pod install
|
||||
open SleepyAgent.xcworkspace
|
||||
```
|
||||
|
||||
### 3. Add LiteRT
|
||||
In Xcode:
|
||||
- File → Add Package Dependencies
|
||||
- Add: `https://github.com/tensorflow/tensorflow`
|
||||
- Or use the CocoaPods version
|
||||
|
||||
### 4. Configure & Build
|
||||
```bash
|
||||
# Check environment
|
||||
make check
|
||||
|
||||
# Build for simulator
|
||||
make build
|
||||
|
||||
# Build for device (requires signing setup)
|
||||
make build-release
|
||||
|
||||
# Create archive
|
||||
make archive
|
||||
|
||||
# Create IPA (requires proper signing)
|
||||
make ipa
|
||||
```
|
||||
|
||||
## Expected Output
|
||||
|
||||
After successful build, you'll have:
|
||||
|
||||
```
|
||||
build/
|
||||
├── DerivedData/ # Build intermediates
|
||||
├── SleepyAgent.xcarchive # Archive for distribution
|
||||
└── Export/
|
||||
└── SleepyAgent.ipa # Final IPA file
|
||||
```
|
||||
|
||||
## Testing
|
||||
|
||||
### On Simulator
|
||||
```bash
|
||||
make build
|
||||
# Run in Xcode with iPhone 15 Pro simulator selected
|
||||
```
|
||||
|
||||
### On Device
|
||||
```bash
|
||||
# Connect iPhone via USB
|
||||
make install
|
||||
# Or use Xcode: Cmd+R with device selected
|
||||
```
|
||||
|
||||
### IPA Installation
|
||||
- **AltStore** (free, no dev account needed)
|
||||
- **Xcode** (with dev account or free personal team)
|
||||
- **TestFlight** (for distribution)
|
||||
|
||||
## Known Limitations
|
||||
|
||||
1. **No Xcode on build machine** - Couldn't create .xcodeproj or compile
|
||||
2. **LiteRT version** - May need to adjust based on latest release
|
||||
3. **iOS 15 minimum** - AttributedString markdown requires iOS 15+
|
||||
4. **Physical device recommended** - Simulator has limited audio/camera support
|
||||
|
||||
## Next Steps for You
|
||||
|
||||
1. ✅ Copy all files to your Mac
|
||||
2. ✅ Install Xcode
|
||||
3. ✅ Run `make setup` or `pod install`
|
||||
4. ✅ Open in Xcode
|
||||
5. ✅ Add LiteRT dependency
|
||||
6. ✅ Set signing team
|
||||
7. ✅ Build (Cmd+B)
|
||||
8. ✅ Run on device (Cmd+R)
|
||||
9. ✅ Create IPA (Product → Archive → Distribute)
|
||||
|
||||
## File Count
|
||||
|
||||
- 19 Swift source files
|
||||
- 6 Configuration files
|
||||
- 4 Documentation files
|
||||
- 2 Build scripts
|
||||
|
||||
**Total: 31 files ready to use**
|
||||
|
||||
## Help
|
||||
|
||||
If you get stuck:
|
||||
1. Check `BUILD.md` for detailed troubleshooting
|
||||
2. File an issue: https://github.com/sleepyeldrazi/sleepy-agent/issues
|
||||
3. Refer to Android version for logic reference
|
||||
@@ -0,0 +1,28 @@
|
||||
// swift-tools-version:5.9
|
||||
import PackageDescription
|
||||
|
||||
let package = Package(
|
||||
name: "SleepyAgent",
|
||||
platforms: [
|
||||
.iOS(.v15)
|
||||
],
|
||||
products: [
|
||||
.library(
|
||||
name: "SleepyAgent",
|
||||
targets: ["SleepyAgent"]),
|
||||
],
|
||||
dependencies: [
|
||||
// LiteRT will be added via CocoaPods or manually
|
||||
],
|
||||
targets: [
|
||||
.target(
|
||||
name: "SleepyAgent",
|
||||
dependencies: [],
|
||||
path: "SleepyAgent",
|
||||
exclude: ["Info.plist"],
|
||||
swiftSettings: [
|
||||
.enableExperimentalFeature("StrictConcurrency")
|
||||
]
|
||||
),
|
||||
]
|
||||
)
|
||||
@@ -0,0 +1,24 @@
|
||||
platform :ios, '15.0'
|
||||
|
||||
use_frameworks!
|
||||
|
||||
target 'SleepyAgent' do
|
||||
# TensorFlow Lite / LiteRT
|
||||
pod 'TensorFlowLiteSwift', '~> 2.16.0'
|
||||
|
||||
# If LiteRT-LM has a specific pod, use that instead:
|
||||
# pod 'LiteRT', '~> 0.10.0'
|
||||
|
||||
target 'SleepyAgentTests' do
|
||||
inherit! :search_paths
|
||||
# Pods for testing
|
||||
end
|
||||
end
|
||||
|
||||
post_install do |installer|
|
||||
installer.pods_project.targets.each do |target|
|
||||
target.build_configurations.each do |config|
|
||||
config.build_settings['IPHONEOS_DEPLOYMENT_TARGET'] = '15.0'
|
||||
end
|
||||
end
|
||||
end
|
||||
@@ -0,0 +1,39 @@
|
||||
import SwiftUI
|
||||
import AVFoundation
|
||||
|
||||
@main
|
||||
struct SleepyAgentApp: App {
|
||||
@UIApplicationDelegateAdaptor(AppDelegate.self) var appDelegate
|
||||
|
||||
let container = AppContainer()
|
||||
|
||||
var body: some Scene {
|
||||
WindowGroup {
|
||||
MainView()
|
||||
.environmentObject(container.mainViewModel)
|
||||
.environmentObject(container.settingsViewModel)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
class AppDelegate: NSObject, UIApplicationDelegate {
|
||||
func application(_ application: UIApplication, didFinishLaunchingWithOptions launchOptions: [UIApplication.LaunchOptionsKey: Any]?) -> Bool {
|
||||
AudioSessionManager.shared.configure()
|
||||
return true
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - Audio Session Manager
|
||||
class AudioSessionManager {
|
||||
static let shared = AudioSessionManager()
|
||||
|
||||
func configure() {
|
||||
let audioSession = AVAudioSession.sharedInstance()
|
||||
do {
|
||||
try audioSession.setCategory(.playAndRecord, mode: .default, options: [.defaultToSpeaker, .allowBluetooth])
|
||||
try audioSession.setActive(true)
|
||||
} catch {
|
||||
print("Failed to configure audio session: \(error)")
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,54 @@
|
||||
import Foundation
|
||||
|
||||
@MainActor
|
||||
final class AppContainer: ObservableObject {
|
||||
let audioRecorder = AudioRecorder()
|
||||
let ttsService = TtsService()
|
||||
let webSearchService = WebSearchService()
|
||||
let modelDownloadService = ModelDownloadService()
|
||||
let conversationStorage = ConversationStorage()
|
||||
let llmEngine = LlmEngine()
|
||||
let conversationContext = ConversationContext()
|
||||
|
||||
lazy var agent: Agent = {
|
||||
Agent(llmEngine: llmEngine, context: conversationContext, webSearch: webSearchService)
|
||||
}()
|
||||
|
||||
lazy var mainViewModel: MainViewModel = {
|
||||
MainViewModel(
|
||||
agent: agent,
|
||||
audioRecorder: audioRecorder,
|
||||
ttsService: ttsService,
|
||||
conversationStorage: conversationStorage,
|
||||
llmEngine: llmEngine
|
||||
)
|
||||
}()
|
||||
|
||||
lazy var settingsViewModel: SettingsViewModel = {
|
||||
SettingsViewModel(
|
||||
modelDownloadService: modelDownloadService,
|
||||
llmEngine: llmEngine
|
||||
)
|
||||
}()
|
||||
|
||||
init() {
|
||||
Task {
|
||||
await autoDetectModel()
|
||||
}
|
||||
}
|
||||
|
||||
private func autoDetectModel() async {
|
||||
let fileManager = FileManager.default
|
||||
guard let documentsPath = fileManager.urls(for: .documentDirectory, in: .userDomainMask).first?.path else { return }
|
||||
|
||||
let modelsPath = (documentsPath as NSString).appendingPathComponent("models")
|
||||
let e2bPath = (modelsPath as NSString).appendingPathComponent("gemma-4-E2B-it.litertlm")
|
||||
let e4bPath = (modelsPath as NSString).appendingPathComponent("gemma-4-E4B-it.litertlm")
|
||||
|
||||
if fileManager.fileExists(atPath: e2bPath) {
|
||||
_ = try? await llmEngine.loadModel(path: e2bPath)
|
||||
} else if fileManager.fileExists(atPath: e4bPath) {
|
||||
_ = try? await llmEngine.loadModel(path: e4bPath)
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,62 @@
|
||||
import Foundation
|
||||
|
||||
enum MessageRole: String, Codable {
|
||||
case user
|
||||
case assistant
|
||||
case tool
|
||||
case system
|
||||
}
|
||||
|
||||
struct Message: Identifiable, Codable, Equatable {
|
||||
let id: UUID
|
||||
var role: MessageRole
|
||||
var content: String
|
||||
var toolCalls: [ToolCall]?
|
||||
var toolCallId: String?
|
||||
let timestamp: Date
|
||||
|
||||
init(id: UUID = UUID(), role: MessageRole, content: String, toolCalls: [ToolCall]? = nil, toolCallId: String? = nil, timestamp: Date = Date()) {
|
||||
self.id = id
|
||||
self.role = role
|
||||
self.content = content
|
||||
self.toolCalls = toolCalls
|
||||
self.toolCallId = toolCallId
|
||||
self.timestamp = timestamp
|
||||
}
|
||||
|
||||
static func user(_ content: String) -> Message {
|
||||
Message(role: .user, content: content)
|
||||
}
|
||||
|
||||
static func assistant(_ content: String, toolCalls: [ToolCall]? = nil) -> Message {
|
||||
Message(role: .assistant, content: content, toolCalls: toolCalls)
|
||||
}
|
||||
}
|
||||
|
||||
struct ToolCall: Codable, Equatable {
|
||||
let id: String
|
||||
let name: String
|
||||
let arguments: [String: String]
|
||||
}
|
||||
|
||||
enum ModelVariant: String, Codable, CaseIterable {
|
||||
case e2b = "E2B"
|
||||
case e4b = "E4B"
|
||||
case custom = "Custom"
|
||||
|
||||
var displayName: String {
|
||||
switch self {
|
||||
case .e2b: return "Gemma 4 E2B (Fast)"
|
||||
case .e4b: return "Gemma 4 E4B (Quality)"
|
||||
case .custom: return "Custom Model"
|
||||
}
|
||||
}
|
||||
|
||||
var sizeDescription: String {
|
||||
switch self {
|
||||
case .e2b: return "~2.7 GB"
|
||||
case .e4b: return "~4.5 GB"
|
||||
case .custom: return "Varies"
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,359 @@
|
||||
import Foundation
|
||||
import UIKit
|
||||
|
||||
// MARK: - Agent Events
|
||||
|
||||
/// Events emitted by the Agent during processing
|
||||
public enum AgentEvent: Sendable {
|
||||
case token(String)
|
||||
case executingTool(toolName: String, arguments: [String: String])
|
||||
case toolResult(toolName: String, result: String)
|
||||
case complete(String)
|
||||
case error(String)
|
||||
}
|
||||
|
||||
// MARK: - Agent State
|
||||
|
||||
public enum AgentState: Sendable, Equatable {
|
||||
case idle
|
||||
case generating
|
||||
case executingTool
|
||||
case streaming
|
||||
case error
|
||||
}
|
||||
|
||||
// MARK: - Agent Errors
|
||||
|
||||
public enum AgentError: LocalizedError {
|
||||
case modelNotLoaded
|
||||
case maxIterationsReached
|
||||
case processingFailed(underlying: Error)
|
||||
|
||||
public var errorDescription: String? {
|
||||
switch self {
|
||||
case .modelNotLoaded:
|
||||
return "LLM model is not loaded"
|
||||
case .maxIterationsReached:
|
||||
return "Maximum tool calling iterations reached"
|
||||
case .processingFailed(let error):
|
||||
return "Processing failed: \(error.localizedDescription)"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - Agent
|
||||
|
||||
/// High-level agent that manages conversation with the LLM, including tool calling
|
||||
/// Supports multiple Gemma tool call formats
|
||||
@MainActor
|
||||
public final class Agent: ObservableObject {
|
||||
|
||||
// MARK: - Constants
|
||||
|
||||
private let maxIterations = 5
|
||||
|
||||
// MARK: - Dependencies
|
||||
|
||||
private let llmEngine: any LlmEngine
|
||||
private let context: ConversationContext
|
||||
private let toolRegistry: ToolRegistry
|
||||
private let toolParser = ToolParser()
|
||||
|
||||
// MARK: - State
|
||||
|
||||
@Published public private(set) var state: AgentState = .idle
|
||||
|
||||
private var conversation: Conversation?
|
||||
private let systemPrompt: String
|
||||
|
||||
// MARK: - Initialization
|
||||
|
||||
public init(
|
||||
llmEngine: any LlmEngine,
|
||||
context: ConversationContext,
|
||||
toolRegistry: ToolRegistry,
|
||||
customSystemPrompt: String? = nil
|
||||
) {
|
||||
self.llmEngine = llmEngine
|
||||
self.context = context
|
||||
self.toolRegistry = toolRegistry
|
||||
|
||||
// Build system prompt with tools description
|
||||
var basePrompt = customSystemPrompt ?? Self.defaultSystemPrompt
|
||||
|
||||
// We'll append tool descriptions asynchronously
|
||||
self.systemPrompt = basePrompt
|
||||
}
|
||||
|
||||
private static let defaultSystemPrompt = """
|
||||
You are a helpful AI assistant with access to tools.
|
||||
|
||||
When you need to use a tool, you MUST output EXACTLY in this format:
|
||||
<|tool_call>call:tool_name{"name": "tool_name", "arguments": {"param": "value"}}<tool_call|>
|
||||
|
||||
IMPORTANT:
|
||||
- Replace 'tool_name' with the actual tool name you want to use
|
||||
- Do NOT use 'tool_name' as a placeholder - use the real tool name
|
||||
- For web_search, use: {"name": "web_search", "arguments": {"query": "..."}}
|
||||
|
||||
After receiving tool results, provide a helpful response to the user.
|
||||
"""
|
||||
|
||||
// MARK: - Conversation Management
|
||||
|
||||
private func ensureConversation() async throws -> Conversation {
|
||||
if conversation?.isAlive != true {
|
||||
// Build complete system prompt with current tools
|
||||
let toolsDescription = await toolRegistry.buildToolsDescription()
|
||||
let completePrompt = systemPrompt + toolsDescription
|
||||
|
||||
conversation = try llmEngine.createConversation(systemPrompt: completePrompt)
|
||||
}
|
||||
return conversation!
|
||||
}
|
||||
|
||||
/// Pre-warms the KV cache with system prompt
|
||||
public func prewarmCache() async {
|
||||
do {
|
||||
_ = try await ensureConversation()
|
||||
} catch {
|
||||
print("Failed to pre-warm cache: \(error)")
|
||||
}
|
||||
}
|
||||
|
||||
/// Resets the conversation and clears context
|
||||
public func reset() async {
|
||||
conversation?.close()
|
||||
conversation = nil
|
||||
await context.clear()
|
||||
state = .idle
|
||||
}
|
||||
|
||||
// MARK: - Processing
|
||||
|
||||
/// Process user input with optional multimodal data
|
||||
/// - Parameters:
|
||||
/// - input: The user's text input
|
||||
/// - audioData: Optional audio data (WAV or PCM)
|
||||
/// - images: Optional images
|
||||
/// - Returns: AsyncStream of AgentEvents
|
||||
public func processInput(
|
||||
input: String,
|
||||
audioData: Data? = nil,
|
||||
images: [UIImage]? = nil
|
||||
) -> AsyncStream<AgentEvent> {
|
||||
AsyncStream { continuation in
|
||||
Task {
|
||||
do {
|
||||
try await processInputInternal(
|
||||
input: input,
|
||||
audioData: audioData,
|
||||
images: images,
|
||||
continuation: continuation
|
||||
)
|
||||
} catch {
|
||||
continuation.yield(.error(error.localizedDescription))
|
||||
await MainActor.run { state = .error }
|
||||
}
|
||||
continuation.finish()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private func processInputInternal(
|
||||
input: String,
|
||||
audioData: Data?,
|
||||
images: [UIImage]?,
|
||||
continuation: AsyncStream<AgentEvent>.Continuation
|
||||
) async throws {
|
||||
// Add user message to context
|
||||
let _ = await context.addMessage(.user(content: input))
|
||||
|
||||
// Ensure conversation is ready
|
||||
let conv = try await ensureConversation()
|
||||
|
||||
var iteration = 0
|
||||
var currentAudio = audioData
|
||||
var currentImages = images
|
||||
|
||||
// Main conversation loop
|
||||
while iteration < maxIterations {
|
||||
iteration += 1
|
||||
|
||||
await MainActor.run { state = .generating }
|
||||
|
||||
// Build prompt from context (for subsequent iterations)
|
||||
let prompt = iteration == 1 ? input : await context.buildPrompt()
|
||||
|
||||
// Generate response
|
||||
var fullResponse = ""
|
||||
|
||||
let stream = llmEngine.generateStream(
|
||||
conversation: conv,
|
||||
prompt: prompt,
|
||||
audioData: currentAudio,
|
||||
images: currentImages
|
||||
)
|
||||
|
||||
for try await token in stream {
|
||||
fullResponse.append(token)
|
||||
}
|
||||
|
||||
// Clear multimodal data after first iteration
|
||||
currentAudio = nil
|
||||
currentImages = nil
|
||||
|
||||
// Parse for tool calls
|
||||
let toolCalls = toolParser.parseToolCalls(from: fullResponse)
|
||||
|
||||
if toolCalls.isEmpty {
|
||||
// No tool calls - stream response to user
|
||||
await MainActor.run { state = .streaming }
|
||||
|
||||
// Yield tokens with slight delay for streaming effect
|
||||
for char in fullResponse {
|
||||
continuation.yield(.token(String(char)))
|
||||
try? await Task.sleep(nanoseconds: 5_000_000) // 5ms
|
||||
}
|
||||
|
||||
// Add to context
|
||||
let _ = await context.addMessage(.assistant(content: fullResponse, toolCalls: nil))
|
||||
|
||||
continuation.yield(.complete(fullResponse))
|
||||
await MainActor.run { state = .idle }
|
||||
return
|
||||
}
|
||||
|
||||
// Tool calls detected
|
||||
await MainActor.run { state = .executingTool }
|
||||
|
||||
let contentBeforeTools = toolParser.extractContentBeforeTools(from: fullResponse)
|
||||
|
||||
// Notify about tool execution
|
||||
for toolCall in toolCalls {
|
||||
let tool = await toolRegistry.getTool(named: toolCall.name)
|
||||
let displayName = tool?.displayName ?? toolCall.name
|
||||
continuation.yield(.executingTool(toolName: displayName, arguments: toolCall.arguments))
|
||||
}
|
||||
|
||||
// Execute tools
|
||||
let toolResults = await toolRegistry.executeToolCalls(toolCalls)
|
||||
|
||||
// Add assistant message with tool calls to context
|
||||
let _ = await context.addMessage(.assistant(content: contentBeforeTools, toolCalls: toolCalls))
|
||||
|
||||
// Add tool results to context and notify
|
||||
for result in toolResults {
|
||||
let _ = await context.addMessage(.toolResult(
|
||||
toolCallId: result.id,
|
||||
toolName: result.name,
|
||||
result: result.result
|
||||
))
|
||||
continuation.yield(.toolResult(toolName: result.name, result: result.result))
|
||||
}
|
||||
}
|
||||
|
||||
// Max iterations reached - generate final response
|
||||
await MainActor.run { state = .generating }
|
||||
|
||||
let finalPrompt = await context.buildPrompt()
|
||||
|
||||
let finalStream = llmEngine.generateStream(
|
||||
conversation: conv,
|
||||
prompt: finalPrompt,
|
||||
audioData: nil,
|
||||
images: nil
|
||||
)
|
||||
|
||||
var finalResponse = ""
|
||||
for try await token in finalStream {
|
||||
finalResponse.append(token)
|
||||
}
|
||||
|
||||
await MainActor.run { state = .streaming }
|
||||
|
||||
for char in finalResponse {
|
||||
continuation.yield(.token(String(char)))
|
||||
try? await Task.sleep(nanoseconds: 5_000_000)
|
||||
}
|
||||
|
||||
let _ = await context.addMessage(.assistant(content: finalResponse, toolCalls: nil))
|
||||
|
||||
continuation.yield(.complete(finalResponse))
|
||||
await MainActor.run { state = .idle }
|
||||
}
|
||||
|
||||
/// Process input and collect full response (non-streaming)
|
||||
public func processInputComplete(
|
||||
input: String,
|
||||
audioData: Data? = nil,
|
||||
images: [UIImage]? = nil
|
||||
) async throws -> String {
|
||||
var fullResponse = ""
|
||||
|
||||
for await event in processInput(input: input, audioData: audioData, images: images) {
|
||||
switch event {
|
||||
case .token(let text):
|
||||
fullResponse.append(text)
|
||||
case .complete(let response):
|
||||
return response
|
||||
case .error(let message):
|
||||
throw AgentError.processingFailed(underlying: NSError(
|
||||
domain: "AgentError",
|
||||
code: -1,
|
||||
userInfo: [NSLocalizedDescriptionKey: message]
|
||||
))
|
||||
default:
|
||||
break
|
||||
}
|
||||
}
|
||||
|
||||
return fullResponse
|
||||
}
|
||||
|
||||
// MARK: - Convenience Methods
|
||||
|
||||
/// Check if the agent is ready to process input
|
||||
public func isReady() async -> Bool {
|
||||
await llmEngine.isLoaded()
|
||||
}
|
||||
|
||||
/// Get current conversation history
|
||||
public func getHistory() async -> [Message] {
|
||||
await context.getMessages()
|
||||
}
|
||||
|
||||
/// Export conversation to JSON
|
||||
public func exportConversation() async throws -> String {
|
||||
try await context.exportToJSON()
|
||||
}
|
||||
|
||||
/// Import conversation from JSON
|
||||
public func importConversation(json: String) async throws {
|
||||
try await context.importFromJSON(json)
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - Convenience Initializers
|
||||
|
||||
extension Agent {
|
||||
/// Create an Agent with default web search tool
|
||||
public static func createWithWebSearch(
|
||||
llmEngine: any LlmEngine,
|
||||
searchBaseUrl: String,
|
||||
customSystemPrompt: String? = nil
|
||||
) async -> Agent {
|
||||
let context = ConversationContext()
|
||||
let registry = ToolRegistry()
|
||||
|
||||
let webSearchTool = WebSearchTool(baseUrl: searchBaseUrl)
|
||||
await registry.register(webSearchTool)
|
||||
|
||||
return Agent(
|
||||
llmEngine: llmEngine,
|
||||
context: context,
|
||||
toolRegistry: registry,
|
||||
customSystemPrompt: customSystemPrompt
|
||||
)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,332 @@
|
||||
import Foundation
|
||||
|
||||
// MARK: - Message Types
|
||||
|
||||
public enum Message: Codable, Equatable, Sendable {
|
||||
case user(content: String)
|
||||
case assistant(content: String, toolCalls: [ToolCall]?)
|
||||
case toolResult(toolCallId: String, toolName: String, result: String)
|
||||
case system(content: String)
|
||||
|
||||
private enum CodingKeys: String, CodingKey {
|
||||
case type, content, toolCalls, toolCallId, toolName, result
|
||||
}
|
||||
|
||||
private enum MessageType: String, Codable {
|
||||
case user, assistant, toolResult, system
|
||||
}
|
||||
|
||||
public func encode(to encoder: Encoder) throws {
|
||||
var container = encoder.container(keyedBy: CodingKeys.self)
|
||||
|
||||
switch self {
|
||||
case .user(let content):
|
||||
try container.encode(MessageType.user, forKey: .type)
|
||||
try container.encode(content, forKey: .content)
|
||||
|
||||
case .assistant(let content, let toolCalls):
|
||||
try container.encode(MessageType.assistant, forKey: .type)
|
||||
try container.encode(content, forKey: .content)
|
||||
try container.encode(toolCalls, forKey: .toolCalls)
|
||||
|
||||
case .toolResult(let toolCallId, let toolName, let result):
|
||||
try container.encode(MessageType.toolResult, forKey: .type)
|
||||
try container.encode(toolCallId, forKey: .toolCallId)
|
||||
try container.encode(toolName, forKey: .toolName)
|
||||
try container.encode(result, forKey: .result)
|
||||
|
||||
case .system(let content):
|
||||
try container.encode(MessageType.system, forKey: .type)
|
||||
try container.encode(content, forKey: .content)
|
||||
}
|
||||
}
|
||||
|
||||
public init(from decoder: Decoder) throws {
|
||||
let container = try decoder.container(keyedBy: CodingKeys.self)
|
||||
let type = try container.decode(MessageType.self, forKey: .type)
|
||||
|
||||
switch type {
|
||||
case .user:
|
||||
let content = try container.decode(String.self, forKey: .content)
|
||||
self = .user(content: content)
|
||||
|
||||
case .assistant:
|
||||
let content = try container.decode(String.self, forKey: .content)
|
||||
let toolCalls = try container.decodeIfPresent([ToolCall].self, forKey: .toolCalls)
|
||||
self = .assistant(content: content, toolCalls: toolCalls)
|
||||
|
||||
case .toolResult:
|
||||
let toolCallId = try container.decode(String.self, forKey: .toolCallId)
|
||||
let toolName = try container.decode(String.self, forKey: .toolName)
|
||||
let result = try container.decode(String.self, forKey: .result)
|
||||
self = .toolResult(toolCallId: toolCallId, toolName: toolName, result: result)
|
||||
|
||||
case .system:
|
||||
let content = try container.decode(String.self, forKey: .content)
|
||||
self = .system(content: content)
|
||||
}
|
||||
}
|
||||
|
||||
/// Convert message to text for token estimation
|
||||
public func toText() -> String {
|
||||
switch self {
|
||||
case .user(let content):
|
||||
return content
|
||||
case .assistant(let content, let toolCalls):
|
||||
let toolText = toolCalls?.map { "\($0.name) \($0.arguments)" }.joined(separator: " ") ?? ""
|
||||
return content + toolText
|
||||
case .toolResult(let toolName, _, let result):
|
||||
return "\(toolName) \(result)"
|
||||
case .system(let content):
|
||||
return content
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - ToolCall
|
||||
|
||||
public struct ToolCall: Codable, Equatable, Sendable, Identifiable {
|
||||
public let id: String
|
||||
public let name: String
|
||||
public let arguments: [String: String]
|
||||
|
||||
public init(id: String, name: String, arguments: [String: String]) {
|
||||
self.id = id
|
||||
self.name = name
|
||||
self.arguments = arguments
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - Conversation Context Errors
|
||||
|
||||
public enum ConversationContextError: LocalizedError {
|
||||
case messageTooLarge
|
||||
case serializationFailed(underlying: Error)
|
||||
case persistenceFailed(underlying: Error)
|
||||
|
||||
public var errorDescription: String? {
|
||||
switch self {
|
||||
case .messageTooLarge:
|
||||
return "Message exceeds available token budget"
|
||||
case .serializationFailed(let error):
|
||||
return "Failed to serialize messages: \(error.localizedDescription)"
|
||||
case .persistenceFailed(let error):
|
||||
return "Failed to persist conversation: \(error.localizedDescription)"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - Conversation Context
|
||||
|
||||
/// Manages conversation history with token budgeting and persistence
|
||||
public actor ConversationContext {
|
||||
|
||||
// MARK: - Constants
|
||||
|
||||
public static let charsPerToken = 4
|
||||
private static let defaultMaxTokens = 32768
|
||||
private static let defaultReservedForResponse = 4096
|
||||
|
||||
// MARK: - Properties
|
||||
|
||||
private let systemPrompt: String
|
||||
private let maxTokens: Int
|
||||
private let reservedForResponse: Int
|
||||
private var effectiveBudget: Int { maxTokens - reservedForResponse }
|
||||
|
||||
private var messages: [Message] = []
|
||||
|
||||
// MARK: - Initialization
|
||||
|
||||
public init(
|
||||
systemPrompt: String = "You are a helpful AI assistant.",
|
||||
maxTokens: Int = ConversationContext.defaultMaxTokens,
|
||||
reservedForResponse: Int = ConversationContext.defaultReservedForResponse
|
||||
) {
|
||||
self.systemPrompt = systemPrompt
|
||||
self.maxTokens = maxTokens
|
||||
self.reservedForResponse = reservedForResponse
|
||||
}
|
||||
|
||||
// MARK: - Message Management
|
||||
|
||||
/// Adds a message to the conversation context
|
||||
/// - Parameter message: The message to add
|
||||
/// - Returns: true if message was added successfully, false if it exceeds budget
|
||||
@discardableResult
|
||||
public func addMessage(_ message: Message) -> Bool {
|
||||
let messageTokens = estimateTokens(message.toText())
|
||||
|
||||
// If even with empty context this message exceeds budget, reject it
|
||||
if messageTokens > effectiveBudget {
|
||||
return false
|
||||
}
|
||||
|
||||
messages.append(message)
|
||||
|
||||
// Prune if necessary to stay within budget
|
||||
pruneIfNeeded()
|
||||
|
||||
return true
|
||||
}
|
||||
|
||||
/// Returns all messages in the conversation, including system prompt as first message
|
||||
public func getMessages() -> [Message] {
|
||||
[.system(content: systemPrompt)] + messages
|
||||
}
|
||||
|
||||
/// Returns only the user/assistant/tool messages (excluding system)
|
||||
public func getConversationMessages() -> [Message] {
|
||||
messages
|
||||
}
|
||||
|
||||
/// Clears all conversation messages (except system prompt)
|
||||
public func clear() {
|
||||
messages.removeAll()
|
||||
}
|
||||
|
||||
// MARK: - Prompt Building
|
||||
|
||||
/// Builds a formatted prompt string with XML-style tags for LLM consumption
|
||||
public func buildPrompt() -> String {
|
||||
var result = ""
|
||||
|
||||
// System message first
|
||||
result += "<system>\(escapeXml(systemPrompt))</system>\n"
|
||||
|
||||
// User messages
|
||||
for message in messages {
|
||||
switch message {
|
||||
case .user(let content):
|
||||
result += "<user>\(escapeXml(content))</user>\n"
|
||||
|
||||
case .assistant(let content, let toolCalls):
|
||||
result += "<assistant>\(escapeXml(content))"
|
||||
if let toolCalls = toolCalls {
|
||||
for toolCall in toolCalls {
|
||||
result += "\n<tool_call id=\"\(escapeXml(toolCall.id))\""
|
||||
result += " name=\"\(escapeXml(toolCall.name))\">"
|
||||
let argsStr = toolCall.arguments.map { (key, value) in
|
||||
"\"\(escapeXml(key))\": \"\(escapeXml(value))\""
|
||||
}.joined(separator: ", ")
|
||||
result += "{\(argsStr)}"
|
||||
result += "</tool_call>"
|
||||
}
|
||||
}
|
||||
result += "</assistant>\n"
|
||||
|
||||
case .toolResult(let toolCallId, let toolName, let resultContent):
|
||||
result += "<tool_result"
|
||||
result += " id=\"\(escapeXml(toolCallId))\""
|
||||
result += " tool=\"\(escapeXml(toolName))\">"
|
||||
result += "\(escapeXml(resultContent))"
|
||||
result += "</tool_result>\n"
|
||||
|
||||
case .system:
|
||||
// System messages from the list shouldn't appear here
|
||||
break
|
||||
}
|
||||
}
|
||||
|
||||
return result.trimmingCharacters(in: .whitespacesAndNewlines)
|
||||
}
|
||||
|
||||
// MARK: - Token Management
|
||||
|
||||
/// Estimates token count for given text
|
||||
/// Uses a simple heuristic: ~4 characters per token for English text
|
||||
public func estimateTokens(_ text: String) -> Int {
|
||||
text.count / ConversationContext.charsPerToken
|
||||
}
|
||||
|
||||
/// Returns the current total token count including system prompt
|
||||
public func getTokenCount() -> Int {
|
||||
let systemTokens = estimateTokens(systemPrompt)
|
||||
let messagesTokens = messages.map { estimateTokens($0.toText()) }.reduce(0, +)
|
||||
return systemTokens + messagesTokens
|
||||
}
|
||||
|
||||
/// Returns the available token budget for new messages
|
||||
public func getAvailableTokens() -> Int {
|
||||
effectiveBudget - getTokenCount() + estimateTokens(systemPrompt)
|
||||
}
|
||||
|
||||
// MARK: - Persistence
|
||||
|
||||
/// Saves conversation to JSON file
|
||||
public func save(to url: URL) throws {
|
||||
do {
|
||||
let encoder = JSONEncoder()
|
||||
encoder.outputFormatting = [.prettyPrinted, .sortedKeys]
|
||||
let data = try encoder.encode(messages)
|
||||
try data.write(to: url)
|
||||
} catch {
|
||||
throw ConversationContextError.persistenceFailed(underlying: error)
|
||||
}
|
||||
}
|
||||
|
||||
/// Loads conversation from JSON file
|
||||
public func load(from url: URL) throws {
|
||||
do {
|
||||
let data = try Data(contentsOf: url)
|
||||
let decoder = JSONDecoder()
|
||||
let loadedMessages = try decoder.decode([Message].self, from: data)
|
||||
messages = loadedMessages
|
||||
} catch {
|
||||
throw ConversationContextError.persistenceFailed(underlying: error)
|
||||
}
|
||||
}
|
||||
|
||||
/// Exports conversation to JSON string
|
||||
public func exportToJSON() throws -> String {
|
||||
do {
|
||||
let encoder = JSONEncoder()
|
||||
encoder.outputFormatting = [.prettyPrinted, .sortedKeys]
|
||||
let data = try encoder.encode(messages)
|
||||
return String(data: data, encoding: .utf8) ?? "[]"
|
||||
} catch {
|
||||
throw ConversationContextError.serializationFailed(underlying: error)
|
||||
}
|
||||
}
|
||||
|
||||
/// Imports conversation from JSON string
|
||||
public func importFromJSON(_ json: String) throws {
|
||||
do {
|
||||
guard let data = json.data(using: .utf8) else {
|
||||
throw ConversationContextError.serializationFailed(underlying: NSError(domain: "Invalid JSON string", code: -1))
|
||||
}
|
||||
let decoder = JSONDecoder()
|
||||
let loadedMessages = try decoder.decode([Message].self, from: data)
|
||||
messages = loadedMessages
|
||||
} catch {
|
||||
throw ConversationContextError.serializationFailed(underlying: error)
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - Private Helpers
|
||||
|
||||
private func pruneIfNeeded() {
|
||||
while getTokenCount() > effectiveBudget && !messages.isEmpty {
|
||||
// Find and remove the oldest non-system message
|
||||
if let indexToRemove = messages.firstIndex(where: { message in
|
||||
if case .system = message {
|
||||
return false
|
||||
}
|
||||
return true
|
||||
}) {
|
||||
messages.remove(at: indexToRemove)
|
||||
} else {
|
||||
// No more non-system messages to remove
|
||||
break
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private func escapeXml(_ text: String) -> String {
|
||||
text
|
||||
.replacingOccurrences(of: "&", with: "&")
|
||||
.replacingOccurrences(of: "<", with: "<")
|
||||
.replacingOccurrences(of: ">", with: ">")
|
||||
.replacingOccurrences(of: "\"", with: """)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,335 @@
|
||||
import Foundation
|
||||
import LiteRT
|
||||
import UIKit
|
||||
|
||||
// MARK: - Errors
|
||||
|
||||
public enum LlmEngineError: LocalizedError {
|
||||
case modelNotFound(path: String)
|
||||
case modelNotLoaded
|
||||
case conversationClosed
|
||||
case generationFailed(underlying: Error)
|
||||
case invalidMultimodalInput
|
||||
case engineInitializationFailed(underlying: Error)
|
||||
|
||||
public var errorDescription: String? {
|
||||
switch self {
|
||||
case .modelNotFound(let path):
|
||||
return "Model file not found at: \(path)"
|
||||
case .modelNotLoaded:
|
||||
return "No model is currently loaded"
|
||||
case .conversationClosed:
|
||||
return "Conversation has been closed"
|
||||
case .generationFailed(let error):
|
||||
return "Generation failed: \(error.localizedDescription)"
|
||||
case .invalidMultimodalInput:
|
||||
return "Invalid multimodal input provided"
|
||||
case .engineInitializationFailed(let error):
|
||||
return "Failed to initialize engine: \(error.localizedDescription)"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - LiteRT Conversation Wrapper
|
||||
|
||||
/// Wrapper for LiteRT Conversation to manage lifecycle
|
||||
public final class Conversation: @unchecked Sendable {
|
||||
internal let liteRtConversation: LRTConversation
|
||||
private let lock = NSLock()
|
||||
private var isClosed = false
|
||||
|
||||
public var isAlive: Bool {
|
||||
lock.lock()
|
||||
defer { lock.unlock() }
|
||||
return !isClosed
|
||||
}
|
||||
|
||||
internal init(liteRtConversation: LRTConversation) {
|
||||
self.liteRtConversation = liteRtConversation
|
||||
}
|
||||
|
||||
public func close() {
|
||||
lock.lock()
|
||||
defer { lock.unlock() }
|
||||
guard !isClosed else { return }
|
||||
isClosed = true
|
||||
liteRtConversation.close()
|
||||
}
|
||||
|
||||
deinit {
|
||||
close()
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - LlmEngine Protocol
|
||||
|
||||
/// LLM Engine interface for text generation with optional multimodal inputs
|
||||
public protocol LlmEngine: Actor {
|
||||
/// Load a model from the given path
|
||||
func loadModel(path: String) async throws
|
||||
|
||||
/// Creates a new conversation with the given system prompt
|
||||
/// This should be called once per chat session to enable KV cache reuse
|
||||
func createConversation(systemPrompt: String) throws -> Conversation
|
||||
|
||||
/// Generate a response within an existing conversation
|
||||
/// This reuses the KV cache from previous turns
|
||||
func generate(
|
||||
conversation: Conversation,
|
||||
prompt: String,
|
||||
audioData: Data?,
|
||||
images: [UIImage]?
|
||||
) async throws -> String
|
||||
|
||||
/// Generate a streaming response within an existing conversation
|
||||
func generateStream(
|
||||
conversation: Conversation,
|
||||
prompt: String,
|
||||
audioData: Data?,
|
||||
images: [UIImage]?
|
||||
) -> AsyncThrowingStream<String, Error>
|
||||
|
||||
/// Check if a model is currently loaded
|
||||
func isLoaded() -> Bool
|
||||
|
||||
/// Unload the current model and free resources
|
||||
func unload()
|
||||
}
|
||||
|
||||
// MARK: - LiteRT-LM Engine Implementation
|
||||
|
||||
/// LiteRT-LM based LLM Engine implementation for Gemma models
|
||||
/// Uses .litert model format - download from HuggingFace LiteRT Community
|
||||
@globalActor
|
||||
public actor LiteRtLlmEngine: LlmEngine {
|
||||
public static let shared = LiteRtLlmEngine()
|
||||
|
||||
private var engine: LRTEngine?
|
||||
private var currentModelPath: String?
|
||||
|
||||
private let maxTokens = 16384
|
||||
private let cacheDirName = "litertlm_cache"
|
||||
|
||||
private init() {}
|
||||
|
||||
// MARK: - Model Loading
|
||||
|
||||
public func loadModel(path: String) async throws {
|
||||
unload()
|
||||
|
||||
let modelFile = URL(fileURLWithPath: path)
|
||||
guard FileManager.default.fileExists(atPath: path) else {
|
||||
throw LlmEngineError.modelNotFound(path: path)
|
||||
}
|
||||
|
||||
// Ensure cache directory exists
|
||||
let cacheDir = FileManager.default.temporaryDirectory.appendingPathComponent(cacheDirName)
|
||||
try? FileManager.default.createDirectory(at: cacheDir, withIntermediateDirectories: true)
|
||||
|
||||
do {
|
||||
let engineConfig = LRTEngineConfig(
|
||||
modelPath: path,
|
||||
backend: .cpu,
|
||||
visionBackend: .cpu,
|
||||
audioBackend: .cpu,
|
||||
maxNumTokens: maxTokens,
|
||||
cacheDir: cacheDir.path
|
||||
)
|
||||
|
||||
let newEngine = LRTEngine(config: engineConfig)
|
||||
try newEngine.initialize()
|
||||
|
||||
self.engine = newEngine
|
||||
self.currentModelPath = path
|
||||
|
||||
} catch {
|
||||
throw LlmEngineError.engineInitializationFailed(underlying: error)
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - Conversation Management
|
||||
|
||||
public func createConversation(systemPrompt: String) throws -> Conversation {
|
||||
guard let engine = engine else {
|
||||
throw LlmEngineError.modelNotLoaded
|
||||
}
|
||||
|
||||
let systemContent = LRTContent.text(systemPrompt)
|
||||
let conversationConfig = LRTConversationConfig(systemInstruction: systemContent)
|
||||
|
||||
let liteRtConversation = engine.createConversation(config: conversationConfig)
|
||||
return Conversation(liteRtConversation: liteRtConversation)
|
||||
}
|
||||
|
||||
// MARK: - Generation
|
||||
|
||||
public func generate(
|
||||
conversation: Conversation,
|
||||
prompt: String,
|
||||
audioData: Data? = nil,
|
||||
images: [UIImage]? = nil
|
||||
) async throws -> String {
|
||||
guard conversation.isAlive else {
|
||||
throw LlmEngineError.conversationClosed
|
||||
}
|
||||
|
||||
let contents = try buildContents(prompt: prompt, audioData: audioData, images: images)
|
||||
let response = try conversation.liteRtConversation.sendMessage(contents)
|
||||
return response.stringValue ?? ""
|
||||
}
|
||||
|
||||
public func generateStream(
|
||||
conversation: Conversation,
|
||||
prompt: String,
|
||||
audioData: Data? = nil,
|
||||
images: [UIImage]? = nil
|
||||
) -> AsyncThrowingStream<String, Error> {
|
||||
AsyncThrowingStream { continuation in
|
||||
Task {
|
||||
do {
|
||||
guard conversation.isAlive else {
|
||||
throw LlmEngineError.conversationClosed
|
||||
}
|
||||
|
||||
// For multimodal inputs, use Contents API (non-streaming for now)
|
||||
if audioData != nil || !(images?.isEmpty ?? true) {
|
||||
let contents = try buildContents(prompt: prompt, audioData: audioData, images: images)
|
||||
let response = try conversation.liteRtConversation.sendMessage(contents)
|
||||
if let text = response.stringValue {
|
||||
continuation.yield(text)
|
||||
}
|
||||
continuation.finish()
|
||||
return
|
||||
}
|
||||
|
||||
// Text-only streaming - reuses KV cache
|
||||
let stream = conversation.liteRtConversation.sendMessageAsync(prompt)
|
||||
|
||||
for try await message in stream {
|
||||
if let text = message.stringValue {
|
||||
continuation.yield(text)
|
||||
}
|
||||
}
|
||||
|
||||
continuation.finish()
|
||||
|
||||
} catch {
|
||||
continuation.finish(throwing: LlmEngineError.generationFailed(underlying: error))
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - Utility Methods
|
||||
|
||||
public func isLoaded() -> Bool {
|
||||
engine != nil
|
||||
}
|
||||
|
||||
public func unload() {
|
||||
engine?.close()
|
||||
engine = nil
|
||||
currentModelPath = nil
|
||||
}
|
||||
|
||||
// MARK: - Private Helpers
|
||||
|
||||
private func buildContents(
|
||||
prompt: String,
|
||||
audioData: Data?,
|
||||
images: [UIImage]?
|
||||
) throws -> LRTContents {
|
||||
var contents: [LRTContent] = []
|
||||
|
||||
// Add images first if provided (max 1 for efficiency)
|
||||
if let images = images {
|
||||
for image in images.prefix(1) {
|
||||
if let resizedImage = resizeImage(image, maxSize: CGSize(width: 512, height: 512)),
|
||||
let jpegData = resizedImage.jpegData(compressionQuality: 0.85) {
|
||||
contents.append(LRTContent.imageData(jpegData))
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Add audio if provided
|
||||
if let audioData = audioData, audioData.count >= 6400 {
|
||||
// Assume audio is already in WAV format or convert if needed
|
||||
let wavData = isWavData(audioData) ? audioData : try convertPcmToWav(audioData)
|
||||
contents.append(LRTContent.audioData(wavData))
|
||||
}
|
||||
|
||||
// Add text prompt
|
||||
contents.append(LRTContent.text(prompt))
|
||||
|
||||
return LRTContents(contents: contents)
|
||||
}
|
||||
|
||||
private func resizeImage(_ image: UIImage, maxSize: CGSize) -> UIImage? {
|
||||
let size = image.size
|
||||
|
||||
guard size.width > maxSize.width || size.height > maxSize.height else {
|
||||
return image
|
||||
}
|
||||
|
||||
let widthRatio = maxSize.width / size.width
|
||||
let heightRatio = maxSize.height / size.height
|
||||
let ratio = min(widthRatio, heightRatio)
|
||||
|
||||
let newSize = CGSize(width: size.width * ratio, height: size.height * ratio)
|
||||
|
||||
UIGraphicsBeginImageContextWithOptions(newSize, false, 1.0)
|
||||
defer { UIGraphicsEndImageContext() }
|
||||
|
||||
image.draw(in: CGRect(origin: .zero, size: newSize))
|
||||
return UIGraphicsGetImageFromCurrentImageContext()
|
||||
}
|
||||
|
||||
private func isWavData(_ data: Data) -> Bool {
|
||||
// Check for WAV header: "RIFF" magic number
|
||||
guard data.count >= 12 else { return false }
|
||||
let header = data.prefix(4)
|
||||
return header.elementsEqual([0x52, 0x49, 0x46, 0x46]) // "RIFF"
|
||||
}
|
||||
|
||||
private func convertPcmToWav(_ pcmData: Data, sampleRate: Int32 = 16000, channels: UInt16 = 1) throws -> Data {
|
||||
var wavData = Data()
|
||||
|
||||
// RIFF header
|
||||
wavData.append("RIFF".data(using: .ascii)!)
|
||||
|
||||
// File size (will be filled later)
|
||||
let fileSize = UInt32(pcmData.count + 36)
|
||||
wavData.append(withUnsafeBytes(of: fileSize.littleEndian) { Data($0) })
|
||||
|
||||
// WAVE header
|
||||
wavData.append("WAVE".data(using: .ascii)!)
|
||||
|
||||
// fmt chunk
|
||||
wavData.append("fmt ".data(using: .ascii)!)
|
||||
let fmtChunkSize: UInt32 = 16
|
||||
wavData.append(withUnsafeBytes(of: fmtChunkSize.littleEndian) { Data($0) })
|
||||
|
||||
let audioFormat: UInt16 = 1 // PCM
|
||||
wavData.append(withUnsafeBytes(of: audioFormat.littleEndian) { Data($0) })
|
||||
|
||||
wavData.append(withUnsafeBytes(of: channels.littleEndian) { Data($0) })
|
||||
wavData.append(withUnsafeBytes(of: sampleRate.littleEndian) { Data($0) })
|
||||
|
||||
let byteRate = UInt32(sampleRate) * UInt32(channels) * 2 // 16-bit
|
||||
wavData.append(withUnsafeBytes(of: byteRate.littleEndian) { Data($0) })
|
||||
|
||||
let blockAlign = channels * 2
|
||||
wavData.append(withUnsafeBytes(of: blockAlign.littleEndian) { Data($0) })
|
||||
|
||||
let bitsPerSample: UInt16 = 16
|
||||
wavData.append(withUnsafeBytes(of: bitsPerSample.littleEndian) { Data($0) })
|
||||
|
||||
// data chunk
|
||||
wavData.append("data".data(using: .ascii)!)
|
||||
let dataChunkSize = UInt32(pcmData.count)
|
||||
wavData.append(withUnsafeBytes(of: dataChunkSize.littleEndian) { Data($0) })
|
||||
wavData.append(pcmData)
|
||||
|
||||
return wavData
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,421 @@
|
||||
import Foundation
|
||||
|
||||
// MARK: - Tool Protocol
|
||||
|
||||
/// Protocol for tools that can be called by the Agent
|
||||
public protocol Tool: Sendable {
|
||||
var name: String { get }
|
||||
var displayName: String { get }
|
||||
var description: String { get }
|
||||
|
||||
func execute(arguments: [String: String]) async throws -> String
|
||||
}
|
||||
|
||||
// MARK: - Web Search Tool
|
||||
|
||||
/// Web search tool using SearxNG API
|
||||
public actor WebSearchTool: Tool {
|
||||
public let name = "web_search"
|
||||
public let displayName = "Web Search"
|
||||
public let description = "Search the web for information. Parameters: query (string, required)"
|
||||
|
||||
private var baseUrl: String
|
||||
private let urlSession: URLSession
|
||||
|
||||
/// Response models for SearxNG API
|
||||
private struct SearxngResponse: Codable {
|
||||
let query: String
|
||||
let results: [SearchResult]
|
||||
}
|
||||
|
||||
private struct SearchResult: Codable {
|
||||
let title: String
|
||||
let url: String
|
||||
let content: String
|
||||
let engine: String?
|
||||
}
|
||||
|
||||
public init(baseUrl: String, urlSession: URLSession = .shared) {
|
||||
self.baseUrl = baseUrl.trimmingCharacters(in: CharacterSet(charactersIn: "/"))
|
||||
self.urlSession = urlSession
|
||||
}
|
||||
|
||||
/// Updates the base URL for the search endpoint
|
||||
public func updateBaseUrl(_ newUrl: String) {
|
||||
self.baseUrl = newUrl.trimmingCharacters(in: CharacterSet(charactersIn: "/"))
|
||||
}
|
||||
|
||||
public func execute(arguments: [String: String]) async throws -> String {
|
||||
guard let query = arguments["query"] else {
|
||||
return "Error: 'query' parameter is required"
|
||||
}
|
||||
|
||||
return try await performSearch(query: query)
|
||||
}
|
||||
|
||||
private func performSearch(query: String) async throws -> String {
|
||||
guard var urlComponents = URLComponents(string: "\(baseUrl)/search") else {
|
||||
throw ToolError.invalidURL
|
||||
}
|
||||
|
||||
urlComponents.queryItems = [
|
||||
URLQueryItem(name: "q", value: query),
|
||||
URLQueryItem(name: "format", value: "json"),
|
||||
URLQueryItem(name: "safesearch", value: "0")
|
||||
]
|
||||
|
||||
guard let url = urlComponents.url else {
|
||||
throw ToolError.invalidURL
|
||||
}
|
||||
|
||||
let (data, response) = try await urlSession.data(from: url)
|
||||
|
||||
guard let httpResponse = response as? HTTPURLResponse else {
|
||||
throw ToolError.invalidResponse
|
||||
}
|
||||
|
||||
guard httpResponse.statusCode == 200 else {
|
||||
throw ToolError.httpError(statusCode: httpResponse.statusCode)
|
||||
}
|
||||
|
||||
let decoder = JSONDecoder()
|
||||
decoder.keyDecodingStrategy = .convertFromSnakeCase
|
||||
|
||||
let searchResponse = try decoder.decode(SearxngResponse.self, from: data)
|
||||
|
||||
if searchResponse.results.isEmpty {
|
||||
return "No results found for '\(query)'"
|
||||
}
|
||||
|
||||
return searchResponse.results.prefix(5).map { result in
|
||||
"""
|
||||
Title: \(result.title)
|
||||
URL: \(result.url)
|
||||
Content: \(result.content)
|
||||
"""
|
||||
}.joined(separator: "\n\n")
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - Tool Errors
|
||||
|
||||
public enum ToolError: LocalizedError {
|
||||
case invalidURL
|
||||
case invalidResponse
|
||||
case httpError(statusCode: Int)
|
||||
case executionFailed(underlying: Error)
|
||||
|
||||
public var errorDescription: String? {
|
||||
switch self {
|
||||
case .invalidURL:
|
||||
return "Invalid URL for tool execution"
|
||||
case .invalidResponse:
|
||||
return "Invalid response from tool"
|
||||
case .httpError(let statusCode):
|
||||
return "HTTP error: \(statusCode)"
|
||||
case .executionFailed(let error):
|
||||
return "Tool execution failed: \(error.localizedDescription)"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - Tool Result
|
||||
|
||||
/// Result of a tool execution
|
||||
public struct ToolResult: Sendable, Identifiable {
|
||||
public let id: String
|
||||
public let name: String
|
||||
public let result: String
|
||||
|
||||
public init(id: String, name: String, result: String) {
|
||||
self.id = id
|
||||
self.name = name
|
||||
self.result = result
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - Tool Parser
|
||||
|
||||
/// Parses tool calls from LLM responses
|
||||
public struct ToolParser: Sendable {
|
||||
|
||||
/// Supported tool call patterns
|
||||
private let patterns: [(start: String, end: String)] = [
|
||||
("<|tool_call>call:", "<tool_call|>"),
|
||||
("<|tool_call>", "<tool_call|>"),
|
||||
("<tool_call>", "</tool_call>")
|
||||
]
|
||||
|
||||
/// Parses tool calls from model response
|
||||
/// Handles multiple Gemma tool call formats
|
||||
public func parseToolCalls(from response: String) -> [ToolCall] {
|
||||
var toolCalls: [ToolCall] = []
|
||||
|
||||
for (startTag, endTag) in patterns {
|
||||
var currentIndex = response.startIndex
|
||||
|
||||
while true {
|
||||
guard let startRange = response.range(of: startTag, range: currentIndex..<response.endIndex) else {
|
||||
break
|
||||
}
|
||||
|
||||
let contentStart = startRange.upperBound
|
||||
|
||||
guard let endRange = response.range(of: endTag, range: contentStart..<response.endIndex) else {
|
||||
break
|
||||
}
|
||||
|
||||
let content = String(response[contentStart..<endRange.lowerBound]).trimmingCharacters(in: .whitespacesAndNewlines)
|
||||
|
||||
if let toolCall = parseToolCallContent(content) {
|
||||
toolCalls.append(toolCall)
|
||||
}
|
||||
|
||||
currentIndex = endRange.upperBound
|
||||
}
|
||||
}
|
||||
|
||||
return toolCalls
|
||||
}
|
||||
|
||||
/// Extracts content before any tool calls
|
||||
public func extractContentBeforeTools(from response: String) -> String {
|
||||
var firstIndex: String.Index? = nil
|
||||
|
||||
for (startTag, _) in patterns {
|
||||
if let range = response.range(of: startTag) {
|
||||
if firstIndex == nil || range.lowerBound < firstIndex! {
|
||||
firstIndex = range.lowerBound
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
if let index = firstIndex {
|
||||
return String(response[..<index]).trimmingCharacters(in: .whitespacesAndNewlines)
|
||||
}
|
||||
|
||||
return response.trimmingCharacters(in: .whitespacesAndNewlines)
|
||||
}
|
||||
|
||||
/// Parse individual tool call content
|
||||
/// Handles:
|
||||
/// - tool_name{"name": "...", "arguments": {...}}
|
||||
/// - tool_name{query: "value"}
|
||||
/// - tool_name{query:<|"|>value<|"|>}
|
||||
/// - {"name": "...", ...} (legacy)
|
||||
private func parseToolCallContent(_ content: String) -> ToolCall? {
|
||||
// Clean up special quote tokens first
|
||||
let cleaned = content
|
||||
.replacingOccurrences(of: "<|\"|>", with: "\"")
|
||||
.replacingOccurrences(of: "\">|>", with: "\"")
|
||||
|
||||
// Has braces - parse as {args} or tool_name{args}
|
||||
if let braceRange = cleaned.range(of: "{") {
|
||||
let toolNamePart = String(cleaned[..<braceRange.lowerBound]).trimmingCharacters(in: .whitespacesAndNewlines)
|
||||
let inner = String(cleaned[braceRange.lowerBound...])
|
||||
|
||||
// Try JSON first, then direct args
|
||||
return parseAsJson(toolNamePrefix: toolNamePart, jsonStr: inner)
|
||||
?? parseAsDirectArgs(toolName: toolNamePart, argsStr: inner)
|
||||
}
|
||||
|
||||
// Try as pure JSON
|
||||
if cleaned.trimmingCharacters(in: .whitespacesAndNewlines).hasPrefix("{") {
|
||||
return parseAsJson(toolNamePrefix: "", jsonStr: cleaned)
|
||||
}
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
private func parseAsJson(toolNamePrefix: String, jsonStr: String) -> ToolCall? {
|
||||
guard let data = jsonStr.data(using: .utf8) else {
|
||||
return nil
|
||||
}
|
||||
|
||||
do {
|
||||
guard let json = try JSONSerialization.jsonObject(with: data) as? [String: Any] else {
|
||||
return nil
|
||||
}
|
||||
|
||||
// Get tool name from "name" field or prefix
|
||||
let toolName = (json["name"] as? String)?.trimmingCharacters(in: .whitespacesAndNewlines)
|
||||
?? (toolNamePrefix.isEmpty ? nil : toolNamePrefix)
|
||||
|
||||
guard let name = toolName, !name.isEmpty else {
|
||||
return nil
|
||||
}
|
||||
|
||||
// Get arguments from "arguments" field or top-level
|
||||
let args: [String: String]
|
||||
if let arguments = json["arguments"] as? [String: Any] {
|
||||
args = arguments.compactMapValues { value in
|
||||
if let stringValue = value as? String {
|
||||
return stringValue
|
||||
}
|
||||
return String(describing: value)
|
||||
}
|
||||
} else {
|
||||
args = json
|
||||
.filter { $0.key != "name" }
|
||||
.compactMapValues { value in
|
||||
if let stringValue = value as? String {
|
||||
return stringValue
|
||||
}
|
||||
return String(describing: value)
|
||||
}
|
||||
}
|
||||
|
||||
return ToolCall(id: generateToolCallId(), name: name, arguments: args)
|
||||
|
||||
} catch {
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
private func parseAsDirectArgs(toolName: String, argsStr: String) -> ToolCall? {
|
||||
// Extract content between outer braces
|
||||
let inner = argsStr.trimmingCharacters(in: .whitespacesAndNewlines)
|
||||
.trimmingCharacters(in: CharacterSet(charactersIn: "{}"))
|
||||
|
||||
var args: [String: String] = [:]
|
||||
var depth = 0
|
||||
var current = ""
|
||||
var parts: [String] = []
|
||||
|
||||
// Split by comma, respecting nested structures
|
||||
for char in inner {
|
||||
switch char {
|
||||
case "{", "[":
|
||||
depth += 1
|
||||
current.append(char)
|
||||
case "}", "]":
|
||||
depth -= 1
|
||||
current.append(char)
|
||||
case ",":
|
||||
if depth == 0 {
|
||||
parts.append(current.trimmingCharacters(in: .whitespacesAndNewlines))
|
||||
current = ""
|
||||
} else {
|
||||
current.append(char)
|
||||
}
|
||||
default:
|
||||
current.append(char)
|
||||
}
|
||||
}
|
||||
|
||||
if !current.isEmpty {
|
||||
parts.append(current.trimmingCharacters(in: .whitespacesAndNewlines))
|
||||
}
|
||||
|
||||
// Parse each key:value pair
|
||||
for part in parts {
|
||||
if let colonRange = part.range(of: ":") {
|
||||
let key = String(part[..<colonRange.lowerBound])
|
||||
.trimmingCharacters(in: .whitespacesAndNewlines)
|
||||
.trimmingCharacters(in: CharacterSet(charactersIn: "\"'"))
|
||||
|
||||
var value = String(part[colonRange.upperBound...])
|
||||
.trimmingCharacters(in: .whitespacesAndNewlines)
|
||||
.trimmingCharacters(in: CharacterSet(charactersIn: "\"'{}"))
|
||||
|
||||
if !key.isEmpty {
|
||||
args[key] = value
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
guard !args.isEmpty && !toolName.isEmpty else {
|
||||
return nil
|
||||
}
|
||||
|
||||
return ToolCall(id: generateToolCallId(), name: toolName, arguments: args)
|
||||
}
|
||||
|
||||
private func generateToolCallId() -> String {
|
||||
let uuid = UUID().uuidString
|
||||
let prefix = String(uuid.prefix(8))
|
||||
return "call_\(prefix)"
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - Tool Registry
|
||||
|
||||
/// Registry for managing available tools
|
||||
public actor ToolRegistry {
|
||||
private var tools: [String: any Tool] = [:]
|
||||
|
||||
public init() {}
|
||||
|
||||
/// Register a tool
|
||||
public func register(_ tool: any Tool) {
|
||||
tools[tool.name] = tool
|
||||
}
|
||||
|
||||
/// Unregister a tool by name
|
||||
public func unregister(name: String) {
|
||||
tools.removeValue(forKey: name)
|
||||
}
|
||||
|
||||
/// Get a tool by name
|
||||
public func getTool(named name: String) -> (any Tool)? {
|
||||
tools[name]
|
||||
}
|
||||
|
||||
/// Get all registered tools
|
||||
public func getAllTools() -> [any Tool] {
|
||||
Array(tools.values)
|
||||
}
|
||||
|
||||
/// Execute a tool with given arguments
|
||||
public func executeTool(named name: String, arguments: [String: String]) async -> ToolResult {
|
||||
let id = generateToolCallId()
|
||||
|
||||
guard let tool = tools[name] else {
|
||||
return ToolResult(id: id, name: name, result: "Tool '\(name)' not found")
|
||||
}
|
||||
|
||||
do {
|
||||
let result = try await tool.execute(arguments: arguments)
|
||||
return ToolResult(id: id, name: name, result: result)
|
||||
} catch {
|
||||
return ToolResult(id: id, name: name, result: "Error: \(error.localizedDescription)")
|
||||
}
|
||||
}
|
||||
|
||||
/// Execute multiple tool calls
|
||||
public func executeToolCalls(_ toolCalls: [ToolCall]) async -> [ToolResult] {
|
||||
var results: [ToolResult] = []
|
||||
for toolCall in toolCalls {
|
||||
let result = await executeTool(named: toolCall.name, arguments: toolCall.arguments)
|
||||
results.append(result)
|
||||
}
|
||||
return results
|
||||
}
|
||||
|
||||
/// Build system prompt snippet describing available tools
|
||||
public func buildToolsDescription() -> String {
|
||||
guard !tools.isEmpty else { return "" }
|
||||
|
||||
var description = "\nAvailable tools:\n"
|
||||
for (_, tool) in tools.sorted(by: { $0.key < $1.key }) {
|
||||
description += "- \(tool.name): \(tool.description)\n"
|
||||
}
|
||||
|
||||
description += """
|
||||
\nWhen you need to use a tool, you MUST output EXACTLY in this format:
|
||||
<|tool_call>call:web_search{"name": "web_search", "arguments": {"query": "your search query here"}}<tool_call|>
|
||||
|
||||
IMPORTANT:
|
||||
- Replace 'web_search' with the actual tool name you want to use
|
||||
- Do NOT use 'tool_name' as a placeholder - use the real tool name
|
||||
"""
|
||||
|
||||
return description
|
||||
}
|
||||
|
||||
private func generateToolCallId() -> String {
|
||||
let uuid = UUID().uuidString
|
||||
let prefix = String(uuid.prefix(8))
|
||||
return "call_\(prefix)"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,67 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
|
||||
<plist version="1.0">
|
||||
<dict>
|
||||
<key>CFBundleDevelopmentRegion</key>
|
||||
<string>$(DEVELOPMENT_LANGUAGE)</string>
|
||||
<key>CFBundleDisplayName</key>
|
||||
<string>Sleepy Agent</string>
|
||||
<key>CFBundleExecutable</key>
|
||||
<string>$(EXECUTABLE_NAME)</string>
|
||||
<key>CFBundleIdentifier</key>
|
||||
<string>$(PRODUCT_BUNDLE_IDENTIFIER)</string>
|
||||
<key>CFBundleInfoDictionaryVersion</key>
|
||||
<string>6.0</string>
|
||||
<key>CFBundleName</key>
|
||||
<string>$(PRODUCT_NAME)</string>
|
||||
<key>CFBundlePackageType</key>
|
||||
<string>$(PRODUCT_BUNDLE_PACKAGE_TYPE)</string>
|
||||
<key>CFBundleShortVersionString</key>
|
||||
<string>1.0</string>
|
||||
<key>CFBundleVersion</key>
|
||||
<string>1</string>
|
||||
<key>LSRequiresIPhoneOS</key>
|
||||
<true/>
|
||||
<key>UIApplicationSceneManifest</key>
|
||||
<dict>
|
||||
<key>UIApplicationSupportsMultipleScenes</key>
|
||||
<false/>
|
||||
</dict>
|
||||
<key>UIApplicationSupportsIndirectInputEvents</key>
|
||||
<true/>
|
||||
<key>UILaunchScreen</key>
|
||||
<dict/>
|
||||
<key>UIRequiredDeviceCapabilities</key>
|
||||
<array>
|
||||
<string>arm64</string>
|
||||
</array>
|
||||
<key>UISupportedInterfaceOrientations</key>
|
||||
<array>
|
||||
<string>UIInterfaceOrientationPortrait</string>
|
||||
<string>UIInterfaceOrientationLandscapeLeft</string>
|
||||
<string>UIInterfaceOrientationLandscapeRight</string>
|
||||
</array>
|
||||
<key>UISupportedInterfaceOrientations~ipad</key>
|
||||
<array>
|
||||
<string>UIInterfaceOrientationPortrait</string>
|
||||
<string>UIInterfaceOrientationPortraitUpsideDown</string>
|
||||
<string>UIInterfaceOrientationLandscapeLeft</string>
|
||||
<string>UIInterfaceOrientationLandscapeRight</string>
|
||||
</array>
|
||||
|
||||
<!-- Privacy Permissions -->
|
||||
<key>NSCameraUsageDescription</key>
|
||||
<string>Sleepy Agent needs camera access to take photos for analysis</string>
|
||||
<key>NSMicrophoneUsageDescription</key>
|
||||
<string>Sleepy Agent needs microphone access for voice input</string>
|
||||
<key>NSPhotoLibraryUsageDescription</key>
|
||||
<string>Sleepy Agent needs photo library access to send images</string>
|
||||
|
||||
<!-- Network -->
|
||||
<key>NSAppTransportSecurity</key>
|
||||
<dict>
|
||||
<key>NSAllowsArbitraryLoads</key>
|
||||
<true/>
|
||||
</dict>
|
||||
</dict>
|
||||
</plist>
|
||||
@@ -0,0 +1,378 @@
|
||||
import Foundation
|
||||
import AVFoundation
|
||||
|
||||
// MARK: - Errors
|
||||
|
||||
enum AudioRecorderError: LocalizedError {
|
||||
case alreadyRecording
|
||||
case notRecording
|
||||
case permissionDenied
|
||||
case setupFailed(String)
|
||||
case audioSessionFailed(String)
|
||||
case engineFailed(String)
|
||||
case invalidFormat
|
||||
|
||||
var errorDescription: String? {
|
||||
switch self {
|
||||
case .alreadyRecording:
|
||||
return "Already recording"
|
||||
case .notRecording:
|
||||
return "Not currently recording"
|
||||
case .permissionDenied:
|
||||
return "Microphone permission denied"
|
||||
case .setupFailed(let reason):
|
||||
return "Setup failed: \(reason)"
|
||||
case .audioSessionFailed(let reason):
|
||||
return "Audio session failed: \(reason)"
|
||||
case .engineFailed(let reason):
|
||||
return "Audio engine failed: \(reason)"
|
||||
case .invalidFormat:
|
||||
return "Invalid audio format"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - Delegate Protocol
|
||||
|
||||
protocol AudioRecorderDelegate: AnyObject {
|
||||
func audioRecorder(_ recorder: AudioRecorder, didUpdateAudioLevel level: Float)
|
||||
func audioRecorderDidDetectSpeech(_ recorder: AudioRecorder)
|
||||
func audioRecorderDidDetectSilence(_ recorder: AudioRecorder)
|
||||
}
|
||||
|
||||
// MARK: - Audio Recorder
|
||||
|
||||
/// Records audio with Voice Activity Detection for automatic silence-based stopping.
|
||||
/// Outputs PCM 16-bit, 16kHz, mono WAV format suitable for Gemma 4 speech recognition.
|
||||
actor AudioRecorder {
|
||||
|
||||
// MARK: - Constants
|
||||
|
||||
static let sampleRate: Double = 16000
|
||||
static let channelCount: AVAudioChannelCount = 1
|
||||
static let bitsPerSample = 16
|
||||
static let bytesPerSample = 2 // 16-bit = 2 bytes
|
||||
|
||||
// VAD Configuration
|
||||
private let speechThresholdDb: Float = -40.0
|
||||
private let silenceThresholdDb: Float = -50.0
|
||||
private let silenceTimeoutMs: TimeInterval = 2.0
|
||||
private let minSpeechDurationMs: TimeInterval = 0.5
|
||||
|
||||
// MARK: - Properties
|
||||
|
||||
weak var delegate: AudioRecorderDelegate?
|
||||
|
||||
private var audioEngine: AVAudioEngine?
|
||||
private var isRecording = false
|
||||
private var audioData = Data()
|
||||
|
||||
// VAD State
|
||||
private var hasDetectedSpeech = false
|
||||
private var speechStartTime: Date?
|
||||
private var lastSpeechTime: Date?
|
||||
private var silenceTimer: Timer?
|
||||
|
||||
// MARK: - Initialization
|
||||
|
||||
init() {}
|
||||
|
||||
// MARK: - Public Methods
|
||||
|
||||
/// Checks if microphone permission is granted.
|
||||
func checkPermission() async -> AVAuthorizationStatus {
|
||||
await withCheckedContinuation { continuation in
|
||||
AVAudioApplication.requestRecordPermission { granted in
|
||||
continuation.resume(returning: granted ? .authorized : .denied)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Requests microphone permission if needed.
|
||||
func requestPermission() async -> Bool {
|
||||
let status = await checkPermission()
|
||||
return status == .authorized
|
||||
}
|
||||
|
||||
/// Starts recording audio with VAD.
|
||||
/// - Throws: AudioRecorderError if setup or permission fails
|
||||
func startRecording() async throws {
|
||||
guard !isRecording else {
|
||||
throw AudioRecorderError.alreadyRecording
|
||||
}
|
||||
|
||||
// Check permission
|
||||
let status = AVCaptureDevice.authorizationStatus(for: .audio)
|
||||
guard status == .authorized else {
|
||||
throw AudioRecorderError.permissionDenied
|
||||
}
|
||||
|
||||
do {
|
||||
try await setupAudioSession()
|
||||
try await setupAudioEngine()
|
||||
|
||||
isRecording = true
|
||||
audioData = Data()
|
||||
|
||||
// Reset VAD state
|
||||
hasDetectedSpeech = false
|
||||
speechStartTime = nil
|
||||
lastSpeechTime = nil
|
||||
|
||||
try audioEngine?.start()
|
||||
|
||||
// Start silence monitoring timer
|
||||
startSilenceMonitoring()
|
||||
|
||||
} catch {
|
||||
cleanup()
|
||||
throw AudioRecorderError.setupFailed(error.localizedDescription)
|
||||
}
|
||||
}
|
||||
|
||||
/// Stops recording and returns the recorded audio data as WAV.
|
||||
/// - Returns: WAV formatted audio data
|
||||
/// - Throws: AudioRecorderError if not recording or processing fails
|
||||
func stopRecording() async throws -> Data {
|
||||
guard isRecording else {
|
||||
throw AudioRecorderError.notRecording
|
||||
}
|
||||
|
||||
return try await performStopRecording()
|
||||
}
|
||||
|
||||
/// Returns whether currently recording.
|
||||
var isCurrentlyRecording: Bool {
|
||||
get { isRecording }
|
||||
}
|
||||
|
||||
// MARK: - Private Methods
|
||||
|
||||
private func setupAudioSession() async throws {
|
||||
let session = AVAudioSession.sharedInstance()
|
||||
try session.setCategory(.playAndRecord, mode: .default, options: [.defaultToSpeaker])
|
||||
try session.setActive(true)
|
||||
}
|
||||
|
||||
private func setupAudioEngine() throws {
|
||||
audioEngine = AVAudioEngine()
|
||||
|
||||
guard let engine = audioEngine else {
|
||||
throw AudioRecorderError.engineFailed("Failed to create engine")
|
||||
}
|
||||
|
||||
let inputNode = engine.inputNode
|
||||
let inputFormat = inputNode.outputFormat(forBus: 0)
|
||||
|
||||
// Configure format: 16kHz, mono, 16-bit PCM
|
||||
guard let format = AVAudioFormat(
|
||||
commonFormat: .pcmFormatInt16,
|
||||
sampleRate: Self.sampleRate,
|
||||
channels: Self.channelCount,
|
||||
interleaved: true
|
||||
) else {
|
||||
throw AudioRecorderError.invalidFormat
|
||||
}
|
||||
|
||||
// Install tap on input node
|
||||
inputNode.installTap(onBus: 0, bufferSize: 1024, format: inputFormat) { [weak self] buffer, _ in
|
||||
Task {
|
||||
await self?.processAudioBuffer(buffer, targetFormat: format)
|
||||
}
|
||||
}
|
||||
|
||||
try engine.start()
|
||||
engine.pause() // Pause until we're ready
|
||||
}
|
||||
|
||||
private func processAudioBuffer(_ buffer: AVAudioPCMBuffer, targetFormat: AVAudioFormat) {
|
||||
guard let converter = AVAudioConverter(from: buffer.format, to: targetFormat) else {
|
||||
return
|
||||
}
|
||||
|
||||
let convertedBuffer = AVAudioPCMBuffer(
|
||||
pcmFormat: targetFormat,
|
||||
frameCapacity: AVAudioFrameCount(Double(buffer.frameCapacity) * targetFormat.sampleRate / buffer.format.sampleRate)
|
||||
)!
|
||||
|
||||
var error: NSError?
|
||||
let inputBlock: AVAudioConverterInputBlock = { _, outStatus in
|
||||
outStatus.pointee = .haveData
|
||||
return buffer
|
||||
}
|
||||
|
||||
converter.convert(to: convertedBuffer, error: &error, withInputFrom: inputBlock)
|
||||
|
||||
if let error = error {
|
||||
print("Conversion error: \(error)")
|
||||
return
|
||||
}
|
||||
|
||||
// Extract PCM data
|
||||
guard let channelData = convertedBuffer.int16ChannelData?[0] else { return }
|
||||
let data = Data(bytes: channelData, count: Int(convertedBuffer.frameLength) * Self.bytesPerSample)
|
||||
|
||||
audioData.append(data)
|
||||
|
||||
// Calculate audio level for VAD
|
||||
let level = calculateAudioLevel(from: data)
|
||||
|
||||
Task { @MainActor in
|
||||
delegate?.audioRecorder(self, didUpdateAudioLevel: level)
|
||||
}
|
||||
|
||||
// Process VAD
|
||||
processVAD(audioLevel: level)
|
||||
}
|
||||
|
||||
private func calculateAudioLevel(from data: Data) -> Float {
|
||||
guard data.count >= 2 else { return -100.0 }
|
||||
|
||||
var sum: Double = 0
|
||||
var count = 0
|
||||
|
||||
var offset = 0
|
||||
while offset < data.count - 1 {
|
||||
let sample = data.withUnsafeBytes { ptr -> Int16 in
|
||||
ptr.load(fromByteOffset: offset, as: Int16.self)
|
||||
}
|
||||
let normalized = Double(sample) / Double(Int16.max)
|
||||
sum += normalized * normalized
|
||||
count += 1
|
||||
offset += 2
|
||||
}
|
||||
|
||||
guard count > 0 else { return -100.0 }
|
||||
|
||||
let rms = sqrt(sum / Double(count))
|
||||
let db = 20 * log10(max(rms, 1e-10))
|
||||
return Float(db)
|
||||
}
|
||||
|
||||
private func processVAD(audioLevel: Float) {
|
||||
let now = Date()
|
||||
|
||||
if audioLevel > speechThresholdDb {
|
||||
// Speech detected
|
||||
if !hasDetectedSpeech {
|
||||
hasDetectedSpeech = true
|
||||
speechStartTime = now
|
||||
lastSpeechTime = now
|
||||
|
||||
Task { @MainActor in
|
||||
delegate?.audioRecorderDidDetectSpeech(self)
|
||||
}
|
||||
} else {
|
||||
lastSpeechTime = now
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private func startSilenceMonitoring() {
|
||||
silenceTimer = Timer.scheduledTimer(withTimeInterval: 0.1, repeats: true) { [weak self] _ in
|
||||
Task {
|
||||
await self?.checkSilenceTimeout()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private func checkSilenceTimeout() {
|
||||
guard hasDetectedSpeech,
|
||||
let speechStart = speechStartTime,
|
||||
let lastSpeech = lastSpeechTime else { return }
|
||||
|
||||
let now = Date()
|
||||
let speechDuration = now.timeIntervalSince(speechStart)
|
||||
let silenceDuration = now.timeIntervalSince(lastSpeech)
|
||||
|
||||
if speechDuration > minSpeechDurationMs && silenceDuration > silenceTimeoutMs {
|
||||
Task { @MainActor in
|
||||
delegate?.audioRecorderDidDetectSilence(self)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private func performStopRecording() async throws -> Data {
|
||||
silenceTimer?.invalidate()
|
||||
silenceTimer = nil
|
||||
|
||||
audioEngine?.stop()
|
||||
audioEngine?.inputNode.removeTap(onBus: 0)
|
||||
|
||||
isRecording = false
|
||||
|
||||
// Validate audio data
|
||||
guard !audioData.isEmpty else {
|
||||
throw AudioRecorderError.setupFailed("No audio data recorded")
|
||||
}
|
||||
|
||||
// Minimum duration check (200ms)
|
||||
let minBytes = Int(Self.sampleRate * Self.bytesPerSample * 0.2)
|
||||
guard audioData.count >= minBytes else {
|
||||
throw AudioRecorderError.setupFailed("Audio too short")
|
||||
}
|
||||
|
||||
// Create WAV file
|
||||
let wavData = createWavFile(from: audioData)
|
||||
|
||||
cleanup()
|
||||
|
||||
return wavData
|
||||
}
|
||||
|
||||
private func createWavFile(from pcmData: Data) -> Data {
|
||||
let sampleRate = UInt32(Self.sampleRate)
|
||||
let channels = UInt16(Self.channelCount)
|
||||
let bitsPerSample = UInt16(Self.bitsPerSample)
|
||||
let byteRate = sampleRate * UInt32(channels) * UInt32(bitsPerSample / 8)
|
||||
let blockAlign = channels * UInt16(bitsPerSample / 8)
|
||||
|
||||
var wavData = Data()
|
||||
|
||||
// RIFF header
|
||||
wavData.append("RIFF".data(using: .ascii)!)
|
||||
wavData.append(UInt32(36 + pcmData.count).littleEndianBytes)
|
||||
wavData.append("WAVE".data(using: .ascii)!)
|
||||
|
||||
// fmt chunk
|
||||
wavData.append("fmt ".data(using: .ascii)!)
|
||||
wavData.append(UInt32(16).littleEndianBytes) // Subchunk1Size
|
||||
wavData.append(UInt16(1).littleEndianBytes) // AudioFormat (PCM)
|
||||
wavData.append(channels.littleEndianBytes)
|
||||
wavData.append(sampleRate.littleEndianBytes)
|
||||
wavData.append(byteRate.littleEndianBytes)
|
||||
wavData.append(blockAlign.littleEndianBytes)
|
||||
wavData.append(bitsPerSample.littleEndianBytes)
|
||||
|
||||
// data chunk
|
||||
wavData.append("data".data(using: .ascii)!)
|
||||
wavData.append(UInt32(pcmData.count).littleEndianBytes)
|
||||
wavData.append(pcmData)
|
||||
|
||||
return wavData
|
||||
}
|
||||
|
||||
private func cleanup() {
|
||||
silenceTimer?.invalidate()
|
||||
silenceTimer = nil
|
||||
|
||||
audioEngine?.stop()
|
||||
audioEngine?.inputNode.removeTap(onBus: 0)
|
||||
audioEngine = nil
|
||||
|
||||
isRecording = false
|
||||
audioData = Data()
|
||||
hasDetectedSpeech = false
|
||||
speechStartTime = nil
|
||||
lastSpeechTime = nil
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - Extensions
|
||||
|
||||
extension FixedWidthInteger {
|
||||
var littleEndianBytes: Data {
|
||||
var value = self.littleEndian
|
||||
return Data(bytes: &value, count: MemoryLayout<Self>.size)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,385 @@
|
||||
import Foundation
|
||||
|
||||
// MARK: - Errors
|
||||
|
||||
enum ConversationStorageError: LocalizedError {
|
||||
case saveFailed(Error)
|
||||
case loadFailed(Error)
|
||||
case deleteFailed(Error)
|
||||
case invalidData
|
||||
case conversationNotFound
|
||||
|
||||
var errorDescription: String? {
|
||||
switch self {
|
||||
case .saveFailed(let error):
|
||||
return "Failed to save conversation: \(error.localizedDescription)"
|
||||
case .loadFailed(let error):
|
||||
return "Failed to load conversation: \(error.localizedDescription)"
|
||||
case .deleteFailed(let error):
|
||||
return "Failed to delete conversation: \(error.localizedDescription)"
|
||||
case .invalidData:
|
||||
return "Invalid conversation data"
|
||||
case .conversationNotFound:
|
||||
return "Conversation not found"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - Data Models
|
||||
|
||||
/// Represents a single message in a conversation.
|
||||
struct ConversationMessage: Codable, Equatable, Identifiable {
|
||||
let id: String
|
||||
let text: String
|
||||
let isUser: Bool
|
||||
let isToolCall: Bool
|
||||
let timestamp: Date
|
||||
|
||||
init(
|
||||
id: String = UUID().uuidString,
|
||||
text: String,
|
||||
isUser: Bool,
|
||||
isToolCall: Bool = false,
|
||||
timestamp: Date = Date()
|
||||
) {
|
||||
self.id = id
|
||||
self.text = text
|
||||
self.isUser = isUser
|
||||
self.isToolCall = isToolCall
|
||||
self.timestamp = timestamp
|
||||
}
|
||||
}
|
||||
|
||||
/// Metadata about a saved conversation.
|
||||
struct ConversationInfo: Codable, Equatable, Identifiable {
|
||||
let id: String
|
||||
let title: String
|
||||
let timestamp: Date
|
||||
let messageCount: Int
|
||||
}
|
||||
|
||||
/// Full conversation data for storage.
|
||||
struct SavedConversation: Codable {
|
||||
let id: String
|
||||
let title: String
|
||||
let timestamp: Date
|
||||
let messages: [ConversationMessage]
|
||||
let version: Int
|
||||
|
||||
init(
|
||||
id: String,
|
||||
title: String,
|
||||
timestamp: Date = Date(),
|
||||
messages: [ConversationMessage],
|
||||
version: Int = 1
|
||||
) {
|
||||
self.id = id
|
||||
self.title = title
|
||||
self.timestamp = timestamp
|
||||
self.messages = messages
|
||||
self.version = version
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - Storage
|
||||
|
||||
/// Manages persistence of conversations using JSON files.
|
||||
actor ConversationStorage {
|
||||
|
||||
// MARK: - Properties
|
||||
|
||||
private let fileManager: FileManager
|
||||
private let encoder: JSONEncoder
|
||||
private let decoder: JSONDecoder
|
||||
|
||||
/// Directory where conversations are stored
|
||||
let storageDirectory: URL
|
||||
|
||||
/// Maximum number of conversations to keep
|
||||
var maxConversations: Int = 50
|
||||
|
||||
// MARK: - Initialization
|
||||
|
||||
init(
|
||||
fileManager: FileManager = .default,
|
||||
storageDirectory: URL? = nil
|
||||
) {
|
||||
self.fileManager = fileManager
|
||||
|
||||
// Setup encoder/decoder
|
||||
self.encoder = JSONEncoder()
|
||||
self.encoder.outputFormatting = [.prettyPrinted, .sortedKeys]
|
||||
self.encoder.dateEncodingStrategy = .iso8601
|
||||
|
||||
self.decoder = JSONDecoder()
|
||||
self.decoder.dateDecodingStrategy = .iso8601
|
||||
|
||||
// Setup storage directory
|
||||
if let directory = storageDirectory {
|
||||
self.storageDirectory = directory
|
||||
} else {
|
||||
let documentsDirectory = fileManager.urls(for: .documentDirectory, in: .userDomainMask).first!
|
||||
self.storageDirectory = documentsDirectory.appendingPathComponent("conversations", isDirectory: true)
|
||||
}
|
||||
|
||||
// Create directory if needed
|
||||
try? fileManager.createDirectory(at: self.storageDirectory, withIntermediateDirectories: true)
|
||||
}
|
||||
|
||||
// MARK: - Public Methods
|
||||
|
||||
/// Creates a new unique conversation ID.
|
||||
func createNewConversationId() -> String {
|
||||
UUID().uuidString
|
||||
}
|
||||
|
||||
/// Saves a conversation to storage.
|
||||
/// - Parameters:
|
||||
/// - id: Conversation identifier
|
||||
/// - messages: List of messages to save
|
||||
/// - Throws: ConversationStorageError if save fails
|
||||
func saveConversation(id: String, messages: [ConversationMessage]) throws {
|
||||
let title = generateTitle(from: messages)
|
||||
let conversation = SavedConversation(
|
||||
id: id,
|
||||
title: title,
|
||||
timestamp: Date(),
|
||||
messages: messages
|
||||
)
|
||||
|
||||
let fileURL = storageDirectory.appendingPathComponent("\(id).json")
|
||||
|
||||
do {
|
||||
let data = try encoder.encode(conversation)
|
||||
try data.write(to: fileURL, options: [.atomic, .completeFileProtection])
|
||||
|
||||
// Clean up old conversations if needed
|
||||
try cleanupOldConversations()
|
||||
} catch {
|
||||
throw ConversationStorageError.saveFailed(error)
|
||||
}
|
||||
}
|
||||
|
||||
/// Loads a conversation from storage.
|
||||
/// - Parameter id: Conversation identifier
|
||||
/// - Returns: Array of messages, or nil if not found
|
||||
/// - Throws: ConversationStorageError if load fails
|
||||
func loadConversation(id: String) throws -> [ConversationMessage]? {
|
||||
let fileURL = storageDirectory.appendingPathComponent("\(id).json")
|
||||
|
||||
guard fileManager.fileExists(atPath: fileURL.path) else {
|
||||
return nil
|
||||
}
|
||||
|
||||
do {
|
||||
let data = try Data(contentsOf: fileURL)
|
||||
let conversation = try decoder.decode(SavedConversation.self, from: data)
|
||||
return conversation.messages
|
||||
} catch {
|
||||
throw ConversationStorageError.loadFailed(error)
|
||||
}
|
||||
}
|
||||
|
||||
/// Deletes a conversation.
|
||||
/// - Parameter id: Conversation identifier
|
||||
/// - Throws: ConversationStorageError if delete fails
|
||||
func deleteConversation(id: String) throws {
|
||||
let fileURL = storageDirectory.appendingPathComponent("\(id).json")
|
||||
|
||||
guard fileManager.fileExists(atPath: fileURL.path) else {
|
||||
throw ConversationStorageError.conversationNotFound
|
||||
}
|
||||
|
||||
do {
|
||||
try fileManager.removeItem(at: fileURL)
|
||||
} catch {
|
||||
throw ConversationStorageError.deleteFailed(error)
|
||||
}
|
||||
}
|
||||
|
||||
/// Gets metadata for all saved conversations, sorted by most recent.
|
||||
/// - Returns: Array of conversation info
|
||||
func getAllConversations() -> [ConversationInfo] {
|
||||
do {
|
||||
let files = try fileManager.contentsOfDirectory(at: storageDirectory, includingPropertiesForKeys: nil)
|
||||
let jsonFiles = files.filter { $0.pathExtension == "json" }
|
||||
|
||||
return jsonFiles.compactMap { url -> ConversationInfo? in
|
||||
guard let conversation = try? loadConversationMetadata(from: url) else { return nil }
|
||||
return ConversationInfo(
|
||||
id: conversation.id,
|
||||
title: conversation.title,
|
||||
timestamp: conversation.timestamp,
|
||||
messageCount: conversation.messages.count
|
||||
)
|
||||
}.sorted { $0.timestamp > $1.timestamp }
|
||||
} catch {
|
||||
return []
|
||||
}
|
||||
}
|
||||
|
||||
/// Checks if a conversation exists.
|
||||
func conversationExists(id: String) -> Bool {
|
||||
let fileURL = storageDirectory.appendingPathComponent("\(id).json")
|
||||
return fileManager.fileExists(atPath: fileURL.path)
|
||||
}
|
||||
|
||||
/// Gets the total storage size used by conversations.
|
||||
func storageSize() -> Int64 {
|
||||
do {
|
||||
let files = try fileManager.contentsOfDirectory(at: storageDirectory, includingPropertiesForKeys: [.fileSizeKey])
|
||||
var totalSize: Int64 = 0
|
||||
for file in files where file.pathExtension == "json" {
|
||||
let attributes = try fileManager.attributesOfItem(atPath: file.path)
|
||||
totalSize += attributes[.size] as? Int64 ?? 0
|
||||
}
|
||||
return totalSize
|
||||
} catch {
|
||||
return 0
|
||||
}
|
||||
}
|
||||
|
||||
/// Exports a conversation to a shareable format.
|
||||
func exportConversation(id: String) throws -> Data {
|
||||
let fileURL = storageDirectory.appendingPathComponent("\(id).json")
|
||||
return try Data(contentsOf: fileURL)
|
||||
}
|
||||
|
||||
/// Imports a conversation from exported data.
|
||||
func importConversation(data: Data) throws -> String {
|
||||
let conversation = try decoder.decode(SavedConversation.self, from: data)
|
||||
let newId = createNewConversationId()
|
||||
|
||||
let importedConversation = SavedConversation(
|
||||
id: newId,
|
||||
title: conversation.title,
|
||||
timestamp: Date(),
|
||||
messages: conversation.messages
|
||||
)
|
||||
|
||||
let fileURL = storageDirectory.appendingPathComponent("\(newId).json")
|
||||
let encodedData = try encoder.encode(importedConversation)
|
||||
try encodedData.write(to: fileURL, options: [.atomic, .completeFileProtection])
|
||||
|
||||
return newId
|
||||
}
|
||||
|
||||
/// Clears all conversations.
|
||||
func clearAllConversations() throws {
|
||||
let files = try fileManager.contentsOfDirectory(at: storageDirectory, includingPropertiesForKeys: nil)
|
||||
for file in files where file.pathExtension == "json" {
|
||||
try fileManager.removeItem(at: file)
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - Private Methods
|
||||
|
||||
private func loadConversationMetadata(from url: URL) throws -> SavedConversation {
|
||||
let data = try Data(contentsOf: url)
|
||||
return try decoder.decode(SavedConversation.self, from: data)
|
||||
}
|
||||
|
||||
private func generateTitle(from messages: [ConversationMessage]) -> String {
|
||||
let maxLength = 50
|
||||
|
||||
// Use first user message as title
|
||||
if let firstUserMessage = messages.first(where: { $0.isUser }) {
|
||||
let text = firstUserMessage.text.trimmingCharacters(in: .whitespacesAndNewlines)
|
||||
if text.count > maxLength {
|
||||
let index = text.index(text.startIndex, offsetBy: maxLength)
|
||||
return String(text[..<index]) + "..."
|
||||
}
|
||||
return text
|
||||
}
|
||||
|
||||
// Fallback to first message or default
|
||||
if let firstMessage = messages.first {
|
||||
return firstMessage.text.prefix(maxLength).trimmingCharacters(in: .whitespacesAndNewlines)
|
||||
}
|
||||
|
||||
return "New Chat"
|
||||
}
|
||||
|
||||
private func cleanupOldConversations() throws {
|
||||
let conversations = getAllConversations()
|
||||
|
||||
if conversations.count > maxConversations {
|
||||
let toDelete = conversations.dropFirst(maxConversations)
|
||||
for info in toDelete {
|
||||
try? deleteConversation(id: info.id)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - Conversation Observer
|
||||
|
||||
/// Observes conversation storage changes.
|
||||
actor ConversationObserver {
|
||||
private var continuations: [UUID: AsyncStream<[ConversationInfo]>.Continuation] = [:]
|
||||
private let storage: ConversationStorage
|
||||
private var lastKnownConversations: [ConversationInfo] = []
|
||||
private var timer: Timer?
|
||||
|
||||
init(storage: ConversationStorage) {
|
||||
self.storage = storage
|
||||
}
|
||||
|
||||
/// Creates a stream that emits when conversations change.
|
||||
func observe() -> AsyncStream<[ConversationInfo]> {
|
||||
AsyncStream { continuation in
|
||||
let id = UUID()
|
||||
continuations[id] = continuation
|
||||
|
||||
// Initial emit
|
||||
Task {
|
||||
let conversations = await storage.getAllConversations()
|
||||
continuation.yield(conversations)
|
||||
}
|
||||
|
||||
continuation.onTermination = { [weak self] _ in
|
||||
Task {
|
||||
await self?.removeContinuation(id: id)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private func removeContinuation(id: UUID) {
|
||||
continuations.removeValue(forKey: id)
|
||||
}
|
||||
|
||||
/// Manually trigger a check for changes.
|
||||
func checkForChanges() async {
|
||||
let current = await storage.getAllConversations()
|
||||
if current != lastKnownConversations {
|
||||
lastKnownConversations = current
|
||||
for continuation in continuations.values {
|
||||
continuation.yield(current)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - Preview Helpers
|
||||
|
||||
extension ConversationStorage {
|
||||
/// Creates a storage instance with sample data for previews.
|
||||
static func preview() -> ConversationStorage {
|
||||
let storage = ConversationStorage()
|
||||
|
||||
// Add sample conversations in background
|
||||
Task {
|
||||
let sampleMessages = [
|
||||
ConversationMessage(text: "Hello, how can you help me?", isUser: true),
|
||||
ConversationMessage(text: "I'm an AI assistant. I can help with various tasks like answering questions, searching the web, or having conversations.", isUser: false)
|
||||
]
|
||||
try? await storage.saveConversation(
|
||||
id: "preview-1",
|
||||
messages: sampleMessages
|
||||
)
|
||||
}
|
||||
|
||||
return storage
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,435 @@
|
||||
import Foundation
|
||||
|
||||
// MARK: - Errors
|
||||
|
||||
enum ModelDownloadError: LocalizedError {
|
||||
case invalidURL
|
||||
case networkError(Error)
|
||||
case serverError(Int)
|
||||
case insufficientStorage
|
||||
case fileError(Error)
|
||||
case invalidResponse
|
||||
case checksumMismatch
|
||||
case cancelled
|
||||
|
||||
var errorDescription: String? {
|
||||
switch self {
|
||||
case .invalidURL:
|
||||
return "Invalid download URL"
|
||||
case .networkError(let error):
|
||||
return "Network error: \(error.localizedDescription)"
|
||||
case .serverError(let code):
|
||||
return "Server error: HTTP \(code)"
|
||||
case .insufficientStorage:
|
||||
return "Insufficient storage space"
|
||||
case .fileError(let error):
|
||||
return "File error: \(error.localizedDescription)"
|
||||
case .invalidResponse:
|
||||
return "Invalid server response"
|
||||
case .checksumMismatch:
|
||||
return "Downloaded file checksum mismatch"
|
||||
case .cancelled:
|
||||
return "Download cancelled"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - Progress & State
|
||||
|
||||
enum DownloadState: Equatable {
|
||||
case idle
|
||||
case checking
|
||||
case downloading(progress: Double, bytesDownloaded: Int64, totalBytes: Int64)
|
||||
case paused(bytesDownloaded: Int64, totalBytes: Int64)
|
||||
case completed(url: URL)
|
||||
case error(ModelDownloadError)
|
||||
|
||||
static func == (lhs: DownloadState, rhs: DownloadState) -> Bool {
|
||||
switch (lhs, rhs) {
|
||||
case (.idle, .idle), (.checking, .checking):
|
||||
return true
|
||||
case let (.downloading(p1, b1, t1), .downloading(p2, b2, t2)):
|
||||
return abs(p1 - p2) < 0.001 && b1 == b2 && t1 == t2
|
||||
case let (.paused(b1, t1), .paused(b2, t2)):
|
||||
return b1 == b2 && t1 == t2
|
||||
case let (.completed(u1), .completed(u2)):
|
||||
return u1 == u2
|
||||
case let (.error(e1), .error(e2)):
|
||||
return e1.localizedDescription == e2.localizedDescription
|
||||
default:
|
||||
return false
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - Model Info
|
||||
|
||||
struct ModelInfo {
|
||||
let url: URL
|
||||
let filename: String
|
||||
let expectedSize: Int64?
|
||||
let variant: ModelVariant
|
||||
|
||||
enum ModelVariant: String, CaseIterable {
|
||||
case e2b = "gemma-4-E2B-it"
|
||||
case e4b = "gemma-4-E4B-it"
|
||||
|
||||
var filename: String {
|
||||
"\(rawValue).litertlm"
|
||||
}
|
||||
|
||||
var huggingFaceURL: URL {
|
||||
URL(string: "https://huggingface.co/litert-community/\(rawValue)-litert-lm/resolve/main/\(filename)")!
|
||||
}
|
||||
|
||||
var expectedSize: Int64 {
|
||||
switch self {
|
||||
case .e2b:
|
||||
return 2_717_263_232 // ~2.53 GB
|
||||
case .e4b:
|
||||
return 4_831_838_208 // ~4.5 GB
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
static func e2b() -> ModelInfo {
|
||||
ModelInfo(
|
||||
url: ModelVariant.e2b.huggingFaceURL,
|
||||
filename: ModelVariant.e2b.filename,
|
||||
expectedSize: ModelVariant.e2b.expectedSize,
|
||||
variant: .e2b
|
||||
)
|
||||
}
|
||||
|
||||
static func e4b() -> ModelInfo {
|
||||
ModelInfo(
|
||||
url: ModelVariant.e4b.huggingFaceURL,
|
||||
filename: ModelVariant.e4b.filename,
|
||||
expectedSize: ModelVariant.e4b.expectedSize,
|
||||
variant: .e4b
|
||||
)
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - Download Task
|
||||
|
||||
actor ModelDownloadService {
|
||||
|
||||
// MARK: - Properties
|
||||
|
||||
private let fileManager: FileManager
|
||||
private let urlSession: URLSession
|
||||
private var currentTask: Task<Void, Never>?
|
||||
private var downloadContinuation: AsyncStream<DownloadState>.Continuation?
|
||||
|
||||
private var currentState: DownloadState = .idle
|
||||
private var tempFileURL: URL?
|
||||
private var destinationURL: URL?
|
||||
|
||||
/// Current download state
|
||||
var state: DownloadState { currentState }
|
||||
|
||||
/// Path to the models directory
|
||||
let modelsDirectory: URL
|
||||
|
||||
// MARK: - Constants
|
||||
|
||||
private let chunkSize = 8192 // 8KB chunks
|
||||
private let progressUpdateInterval: TimeInterval = 0.5
|
||||
|
||||
// MARK: - Initialization
|
||||
|
||||
init(
|
||||
fileManager: FileManager = .default,
|
||||
urlSession: URLSession = .shared
|
||||
) {
|
||||
self.fileManager = fileManager
|
||||
self.urlSession = urlSession
|
||||
|
||||
// Setup models directory in documents
|
||||
let documentsDirectory = fileManager.urls(for: .documentDirectory, in: .userDomainMask).first!
|
||||
self.modelsDirectory = documentsDirectory.appendingPathComponent("models", isDirectory: true)
|
||||
|
||||
// Create models directory if needed
|
||||
try? fileManager.createDirectory(at: modelsDirectory, withIntermediateDirectories: true)
|
||||
}
|
||||
|
||||
// MARK: - Public Methods
|
||||
|
||||
/// Checks if a model variant is downloaded.
|
||||
func isDownloaded(variant: ModelInfo.ModelVariant) -> Bool {
|
||||
let fileURL = modelsDirectory.appendingPathComponent(variant.filename)
|
||||
guard fileManager.fileExists(atPath: fileURL.path) else { return false }
|
||||
|
||||
let minSize: Int64 = 100_000_000 // At least 100MB
|
||||
do {
|
||||
let attributes = try fileManager.attributesOfItem(atPath: fileURL.path)
|
||||
let fileSize = attributes[.size] as? Int64 ?? 0
|
||||
return fileSize > minSize
|
||||
} catch {
|
||||
return false
|
||||
}
|
||||
}
|
||||
|
||||
/// Gets the URL for a model file.
|
||||
func modelFileURL(variant: ModelInfo.ModelVariant) -> URL {
|
||||
modelsDirectory.appendingPathComponent(variant.filename)
|
||||
}
|
||||
|
||||
/// Gets download progress for a variant.
|
||||
func downloadProgress(variant: ModelInfo.ModelVariant) -> Double {
|
||||
let fileURL = modelsDirectory.appendingPathComponent(variant.filename)
|
||||
guard fileManager.fileExists(atPath: fileURL.path) else { return 0 }
|
||||
|
||||
do {
|
||||
let attributes = try fileManager.attributesOfItem(atPath: fileURL.path)
|
||||
let fileSize = attributes[.size] as? Int64 ?? 0
|
||||
return min(Double(fileSize) / Double(variant.expectedSize), 1.0)
|
||||
} catch {
|
||||
return 0
|
||||
}
|
||||
}
|
||||
|
||||
/// Downloads a model with progress reporting.
|
||||
/// - Parameter modelInfo: The model to download
|
||||
/// - Returns: AsyncStream of download state updates
|
||||
func download(modelInfo: ModelInfo) -> AsyncStream<DownloadState> {
|
||||
// Cancel any existing download
|
||||
cancelDownload()
|
||||
|
||||
return AsyncStream { continuation in
|
||||
self.downloadContinuation = continuation
|
||||
|
||||
currentTask = Task {
|
||||
await performDownload(modelInfo: modelInfo, continuation: continuation)
|
||||
}
|
||||
|
||||
continuation.onTermination = { [weak self] _ in
|
||||
Task {
|
||||
await self?.cancelDownload()
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Cancels the current download.
|
||||
func cancelDownload() {
|
||||
currentTask?.cancel()
|
||||
currentTask = nil
|
||||
|
||||
// Clean up temp file
|
||||
if let tempURL = tempFileURL {
|
||||
try? fileManager.removeItem(at: tempURL)
|
||||
tempFileURL = nil
|
||||
}
|
||||
|
||||
updateState(.idle)
|
||||
}
|
||||
|
||||
/// Deletes a downloaded model.
|
||||
func deleteModel(variant: ModelInfo.ModelVariant) throws {
|
||||
let fileURL = modelsDirectory.appendingPathComponent(variant.filename)
|
||||
if fileManager.fileExists(atPath: fileURL.path) {
|
||||
try fileManager.removeItem(at: fileURL)
|
||||
}
|
||||
|
||||
// Also delete temp file if exists
|
||||
let tempURL = modelsDirectory.appendingPathComponent("\(variant.filename).tmp")
|
||||
if fileManager.fileExists(atPath: tempURL.path) {
|
||||
try fileManager.removeItem(at: tempURL)
|
||||
}
|
||||
}
|
||||
|
||||
/// Gets available storage space in bytes.
|
||||
func availableStorage() -> Int64? {
|
||||
do {
|
||||
let attributes = try fileManager.attributesOfFileSystem(forPath: modelsDirectory.path)
|
||||
return attributes[.systemFreeSize] as? Int64
|
||||
} catch {
|
||||
return nil
|
||||
}
|
||||
}
|
||||
|
||||
/// Formats bytes to human-readable string.
|
||||
static func formatBytes(_ bytes: Int64) -> String {
|
||||
let formatter = ByteCountFormatter()
|
||||
formatter.countStyle = .file
|
||||
return formatter.string(fromByteCount: bytes)
|
||||
}
|
||||
|
||||
// MARK: - Private Methods
|
||||
|
||||
private func performDownload(
|
||||
modelInfo: ModelInfo,
|
||||
continuation: AsyncStream<DownloadState>.Continuation
|
||||
) async {
|
||||
updateState(.checking)
|
||||
|
||||
destinationURL = modelsDirectory.appendingPathComponent(modelInfo.filename)
|
||||
let tempURL = modelsDirectory.appendingPathComponent("\(modelInfo.filename).tmp")
|
||||
tempFileURL = tempURL
|
||||
|
||||
// Check available storage
|
||||
if let available = availableStorage(), let expected = modelInfo.expectedSize {
|
||||
if available < expected {
|
||||
updateState(.error(.insufficientStorage))
|
||||
continuation.finish()
|
||||
return
|
||||
}
|
||||
}
|
||||
|
||||
// Check for partial download to resume
|
||||
let resumeFrom: Int64
|
||||
if fileManager.fileExists(atPath: tempURL.path) {
|
||||
do {
|
||||
let attributes = try fileManager.attributesOfItem(atPath: tempURL.path)
|
||||
resumeFrom = attributes[.size] as? Int64 ?? 0
|
||||
} catch {
|
||||
resumeFrom = 0
|
||||
}
|
||||
} else {
|
||||
resumeFrom = 0
|
||||
}
|
||||
|
||||
do {
|
||||
try await downloadFile(
|
||||
from: modelInfo.url,
|
||||
to: tempURL,
|
||||
resumeFrom: resumeFrom,
|
||||
expectedSize: modelInfo.expectedSize,
|
||||
continuation: continuation
|
||||
)
|
||||
|
||||
// Move temp file to final location
|
||||
if fileManager.fileExists(atPath: destinationURL!.path) {
|
||||
try fileManager.removeItem(at: destinationURL!)
|
||||
}
|
||||
try fileManager.moveItem(at: tempURL, to: destinationURL!)
|
||||
|
||||
tempFileURL = nil
|
||||
updateState(.completed(url: destinationURL!))
|
||||
|
||||
} catch is CancellationError {
|
||||
updateState(.idle)
|
||||
} catch let error as ModelDownloadError {
|
||||
updateState(.error(error))
|
||||
} catch {
|
||||
updateState(.error(.networkError(error)))
|
||||
}
|
||||
|
||||
continuation.finish()
|
||||
}
|
||||
|
||||
private func downloadFile(
|
||||
from url: URL,
|
||||
to destination: URL,
|
||||
resumeFrom: Int64,
|
||||
expectedSize: Int64?,
|
||||
continuation: AsyncStream<DownloadState>.Continuation
|
||||
) async throws {
|
||||
var request = URLRequest(url: url)
|
||||
request.timeoutInterval = 30
|
||||
|
||||
// Set up resume with Range header
|
||||
if resumeFrom > 0 {
|
||||
request.setValue("bytes=\(resumeFrom)-", forHTTPHeaderField: "Range")
|
||||
}
|
||||
|
||||
request.setValue("SleepyAgent-iOS/1.0", forHTTPHeaderField: "User-Agent")
|
||||
|
||||
let (asyncBytes, response) = try await urlSession.bytes(for: request)
|
||||
|
||||
guard let httpResponse = response as? HTTPURLResponse else {
|
||||
throw ModelDownloadError.invalidResponse
|
||||
}
|
||||
|
||||
let validCodes = resumeFrom > 0 ? [200, 206] : [200]
|
||||
guard validCodes.contains(httpResponse.statusCode) else {
|
||||
throw ModelDownloadError.serverError(httpResponse.statusCode)
|
||||
}
|
||||
|
||||
// Determine total size
|
||||
let totalBytes: Int64
|
||||
if let contentLength = httpResponse.value(forHTTPHeaderField: "Content-Length"),
|
||||
let length = Int64(contentLength) {
|
||||
if httpResponse.statusCode == 206 && resumeFrom > 0 {
|
||||
totalBytes = resumeFrom + length
|
||||
} else {
|
||||
totalBytes = length
|
||||
}
|
||||
} else if let expected = expectedSize {
|
||||
totalBytes = expected
|
||||
} else {
|
||||
totalBytes = 0
|
||||
}
|
||||
|
||||
// Open file for writing/appending
|
||||
let fileHandle: FileHandle
|
||||
if resumeFrom > 0 && fileManager.fileExists(atPath: destination.path) {
|
||||
fileHandle = try FileHandle(forWritingTo: destination)
|
||||
try fileHandle.seekToEnd()
|
||||
} else {
|
||||
if fileManager.fileExists(atPath: destination.path) {
|
||||
try fileManager.removeItem(at: destination)
|
||||
}
|
||||
fileManager.createFile(atPath: destination.path, contents: nil)
|
||||
fileHandle = try FileHandle(forWritingTo: destination)
|
||||
}
|
||||
defer {
|
||||
try? fileHandle.close()
|
||||
}
|
||||
|
||||
var bytesDownloaded = resumeFrom
|
||||
var lastProgressUpdate = Date()
|
||||
|
||||
for try await byte in asyncBytes {
|
||||
try Task.checkCancellation()
|
||||
|
||||
fileHandle.write(Data([byte]))
|
||||
bytesDownloaded += 1
|
||||
|
||||
// Update progress periodically
|
||||
let now = Date()
|
||||
if now.timeIntervalSince(lastProgressUpdate) > progressUpdateInterval {
|
||||
let progress = totalBytes > 0 ? Double(bytesDownloaded) / Double(totalBytes) : 0
|
||||
updateState(.downloading(
|
||||
progress: progress,
|
||||
bytesDownloaded: bytesDownloaded,
|
||||
totalBytes: totalBytes
|
||||
))
|
||||
lastProgressUpdate = now
|
||||
}
|
||||
}
|
||||
|
||||
// Final progress update
|
||||
let finalProgress = totalBytes > 0 ? Double(bytesDownloaded) / Double(totalBytes) : 1.0
|
||||
updateState(.downloading(
|
||||
progress: finalProgress,
|
||||
bytesDownloaded: bytesDownloaded,
|
||||
totalBytes: totalBytes
|
||||
))
|
||||
|
||||
// Verify download completed
|
||||
if let expected = expectedSize, bytesDownloaded < expected - 1000 {
|
||||
throw ModelDownloadError.networkError(NSError(domain: "Download incomplete", code: -1))
|
||||
}
|
||||
}
|
||||
|
||||
private func updateState(_ state: DownloadState) {
|
||||
currentState = state
|
||||
downloadContinuation?.yield(state)
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - Background Download Support
|
||||
|
||||
extension ModelDownloadService {
|
||||
/// Creates a background URLSession configuration for downloads.
|
||||
static func backgroundConfiguration(identifier: String) -> URLSessionConfiguration {
|
||||
let config = URLSessionConfiguration.background(withIdentifier: identifier)
|
||||
config.isDiscretionary = false
|
||||
config.sessionSendsLaunchEvents = true
|
||||
config.allowsCellularAccess = true
|
||||
return config
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,222 @@
|
||||
import Foundation
|
||||
import AVFoundation
|
||||
|
||||
// MARK: - Errors
|
||||
|
||||
enum TtsServiceError: LocalizedError {
|
||||
case notAvailable
|
||||
case synthesisFailed(String)
|
||||
|
||||
var errorDescription: String? {
|
||||
switch self {
|
||||
case .notAvailable:
|
||||
return "Text-to-speech is not available on this device"
|
||||
case .synthesisFailed(let reason):
|
||||
return "Speech synthesis failed: \(reason)"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - State
|
||||
|
||||
enum TtsState: Equatable {
|
||||
case initializing
|
||||
case ready
|
||||
case speaking
|
||||
case error(String)
|
||||
}
|
||||
|
||||
// MARK: - TTS Service
|
||||
|
||||
/// Text-to-Speech service using AVSpeechSynthesizer.
|
||||
actor TtsService: NSObject {
|
||||
|
||||
// MARK: - Properties
|
||||
|
||||
private var synthesizer: AVSpeechSynthesizer?
|
||||
private var currentState: TtsState = .initializing
|
||||
private var completionContinuation: CheckedContinuation<Void, Never>?
|
||||
|
||||
/// Current TTS state
|
||||
var state: TtsState { currentState }
|
||||
|
||||
/// Whether the synthesizer is currently speaking
|
||||
var isSpeaking: Bool {
|
||||
synthesizer?.isSpeaking ?? false
|
||||
}
|
||||
|
||||
/// Whether TTS is available on this device
|
||||
var isAvailable: Bool {
|
||||
AVSpeechSynthesisVoice.speechVoices().count > 0
|
||||
}
|
||||
|
||||
// MARK: - Initialization
|
||||
|
||||
override init() {
|
||||
super.init()
|
||||
}
|
||||
|
||||
deinit {
|
||||
synthesizer?.stopSpeaking(at: .immediate)
|
||||
}
|
||||
|
||||
// MARK: - Setup
|
||||
|
||||
/// Initializes the TTS engine.
|
||||
/// - Returns: Async stream of state changes
|
||||
func initialize() async -> AsyncStream<TtsState> {
|
||||
AsyncStream { continuation in
|
||||
Task {
|
||||
await setupSynthesizer()
|
||||
continuation.yield(currentState)
|
||||
continuation.finish()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private func setupSynthesizer() {
|
||||
guard isAvailable else {
|
||||
currentState = .error("No voices available")
|
||||
return
|
||||
}
|
||||
|
||||
synthesizer = AVSpeechSynthesizer()
|
||||
synthesizer?.delegate = self
|
||||
currentState = .ready
|
||||
}
|
||||
|
||||
// MARK: - Speech Methods
|
||||
|
||||
/// Speaks the given text.
|
||||
/// - Parameters:
|
||||
/// - text: The text to speak
|
||||
/// - language: Optional language code (e.g., "en-US"). Defaults to system language.
|
||||
/// - Throws: TtsServiceError if synthesis fails
|
||||
func speak(text: String, language: String? = nil) async throws {
|
||||
guard let synthesizer = synthesizer else {
|
||||
throw TtsServiceError.notAvailable
|
||||
}
|
||||
|
||||
// Stop any current speech
|
||||
synthesizer.stopSpeaking(at: .immediate)
|
||||
|
||||
let utterance = AVSpeechUtterance(string: text)
|
||||
|
||||
// Configure voice
|
||||
if let language = language {
|
||||
utterance.voice = AVSpeechSynthesisVoice(language: language)
|
||||
} else {
|
||||
// Try system language, fallback to US English
|
||||
let systemLanguage = Locale.current.language.languageCode?.identifier ?? "en"
|
||||
let region = Locale.current.region?.identifier ?? "US"
|
||||
let locale = "\(systemLanguage)-\(region)"
|
||||
|
||||
if let voice = AVSpeechSynthesisVoice(language: locale) {
|
||||
utterance.voice = voice
|
||||
} else if let voice = AVSpeechSynthesisVoice(language: "en-US") {
|
||||
utterance.voice = voice
|
||||
}
|
||||
}
|
||||
|
||||
// Configure speech parameters
|
||||
utterance.rate = AVSpeechUtteranceDefaultSpeechRate
|
||||
utterance.pitchMultiplier = 1.0
|
||||
utterance.volume = 1.0
|
||||
|
||||
// Speak and wait for completion
|
||||
await withCheckedContinuation { continuation in
|
||||
self.completionContinuation = continuation
|
||||
synthesizer.speak(utterance)
|
||||
}
|
||||
}
|
||||
|
||||
/// Speaks the given text with completion callback.
|
||||
/// - Parameters:
|
||||
/// - text: The text to speak
|
||||
/// - language: Optional language code
|
||||
/// - onComplete: Called when speech completes or fails
|
||||
func speak(text: String, language: String? = nil, onComplete: (() -> Void)? = nil) {
|
||||
Task {
|
||||
do {
|
||||
try await speak(text: text, language: language)
|
||||
} catch {
|
||||
print("TTS error: \(error)")
|
||||
}
|
||||
onComplete?()
|
||||
}
|
||||
}
|
||||
|
||||
/// Stops the current speech immediately.
|
||||
func stop() {
|
||||
synthesizer?.stopSpeaking(at: .immediate)
|
||||
completionContinuation?.resume()
|
||||
completionContinuation = nil
|
||||
currentState = .ready
|
||||
}
|
||||
|
||||
/// Stops the current speech at the end of the word.
|
||||
func stopAtEndOfWord() {
|
||||
synthesizer?.stopSpeaking(at: .word)
|
||||
}
|
||||
|
||||
/// Shuts down the TTS engine and releases resources.
|
||||
func shutdown() {
|
||||
synthesizer?.stopSpeaking(at: .immediate)
|
||||
synthesizer?.delegate = nil
|
||||
synthesizer = nil
|
||||
completionContinuation = nil
|
||||
currentState = .initializing
|
||||
}
|
||||
|
||||
/// Sets the speech rate (0.0 to 1.0, default is 0.5).
|
||||
func setRate(_ rate: Float) {
|
||||
// Applied per utterance
|
||||
}
|
||||
|
||||
/// Sets the speech volume (0.0 to 1.0).
|
||||
func setVolume(_ volume: Float) {
|
||||
// Applied per utterance
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - AVSpeechSynthesizerDelegate
|
||||
|
||||
extension TtsService: AVSpeechSynthesizerDelegate {
|
||||
nonisolated func speechSynthesizer(_ synthesizer: AVSpeechSynthesizer, didStart utterance: AVSpeechUtterance) {
|
||||
Task {
|
||||
await updateState(.speaking)
|
||||
}
|
||||
}
|
||||
|
||||
nonisolated func speechSynthesizer(_ synthesizer: AVSpeechSynthesizer, didFinish utterance: AVSpeechUtterance) {
|
||||
Task {
|
||||
await completeSpeech()
|
||||
}
|
||||
}
|
||||
|
||||
nonisolated func speechSynthesizer(_ synthesizer: AVSpeechSynthesizer, didCancel utterance: AVSpeechUtterance) {
|
||||
Task {
|
||||
await completeSpeech()
|
||||
}
|
||||
}
|
||||
|
||||
nonisolated func speechSynthesizer(_ synthesizer: AVSpeechSynthesizer, didPause utterance: AVSpeechUtterance) {
|
||||
// Handle pause if needed
|
||||
}
|
||||
|
||||
nonisolated func speechSynthesizer(_ synthesizer: AVSpeechSynthesizer, didContinue utterance: AVSpeechUtterance) {
|
||||
Task {
|
||||
await updateState(.speaking)
|
||||
}
|
||||
}
|
||||
|
||||
private func updateState(_ state: TtsState) {
|
||||
currentState = state
|
||||
}
|
||||
|
||||
private func completeSpeech() {
|
||||
completionContinuation?.resume()
|
||||
completionContinuation = nil
|
||||
currentState = .ready
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,234 @@
|
||||
import Foundation
|
||||
|
||||
// MARK: - Errors
|
||||
|
||||
enum WebSearchError: LocalizedError {
|
||||
case invalidURL
|
||||
case networkError(Error)
|
||||
case invalidResponse
|
||||
case decodingError(Error)
|
||||
case serverError(Int)
|
||||
case noResults
|
||||
|
||||
var errorDescription: String? {
|
||||
switch self {
|
||||
case .invalidURL:
|
||||
return "Invalid search URL"
|
||||
case .networkError(let error):
|
||||
return "Network error: \(error.localizedDescription)"
|
||||
case .invalidResponse:
|
||||
return "Invalid response from server"
|
||||
case .decodingError(let error):
|
||||
return "Failed to parse response: \(error.localizedDescription)"
|
||||
case .serverError(let code):
|
||||
return "Server error: HTTP \(code)"
|
||||
case .noResults:
|
||||
return "No search results found"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - Data Models
|
||||
|
||||
struct SearxngResponse: Codable {
|
||||
let query: String
|
||||
let results: [SearchResult]
|
||||
|
||||
enum CodingKeys: String, CodingKey {
|
||||
case query
|
||||
case results
|
||||
}
|
||||
|
||||
init(from decoder: Decoder) throws {
|
||||
let container = try decoder.container(keyedBy: CodingKeys.self)
|
||||
self.query = try container.decodeIfPresent(String.self, forKey: .query) ?? ""
|
||||
self.results = try container.decodeIfPresent([SearchResult].self, forKey: .results) ?? []
|
||||
}
|
||||
}
|
||||
|
||||
struct SearchResult: Codable {
|
||||
let title: String
|
||||
let url: String
|
||||
let content: String
|
||||
let engine: String?
|
||||
|
||||
enum CodingKeys: String, CodingKey {
|
||||
case title
|
||||
case url
|
||||
case content
|
||||
case engine
|
||||
}
|
||||
|
||||
init(from decoder: Decoder) throws {
|
||||
let container = try decoder.container(keyedBy: CodingKeys.self)
|
||||
self.title = try container.decodeIfPresent(String.self, forKey: .title) ?? ""
|
||||
self.url = try container.decodeIfPresent(String.self, forKey: .url) ?? ""
|
||||
self.content = try container.decodeIfPresent(String.self, forKey: .content) ?? ""
|
||||
self.engine = try container.decodeIfPresent(String.self, forKey: .engine)
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - Configuration
|
||||
|
||||
struct WebSearchConfiguration {
|
||||
var baseURL: String
|
||||
var safeSearch: Int
|
||||
var timeout: TimeInterval
|
||||
var maxResults: Int
|
||||
|
||||
static let `default` = WebSearchConfiguration(
|
||||
baseURL: "https://search.sleepy.io",
|
||||
safeSearch: 0,
|
||||
timeout: 30.0,
|
||||
maxResults: 5
|
||||
)
|
||||
}
|
||||
|
||||
// MARK: - Web Search Service
|
||||
|
||||
/// SearXNG client for web search functionality.
|
||||
actor WebSearchService {
|
||||
|
||||
// MARK: - Properties
|
||||
|
||||
private var configuration: WebSearchConfiguration
|
||||
private let urlSession: URLSession
|
||||
private let decoder: JSONDecoder
|
||||
|
||||
/// Current base URL for the SearXNG instance
|
||||
var baseURL: String {
|
||||
get { configuration.baseURL }
|
||||
set { configuration.baseURL = newValue.trimmingCharacters(in: .whitespacesAndNewlines).trimmingCharacters(in: CharacterSet(charactersIn: "/")) }
|
||||
}
|
||||
|
||||
// MARK: - Initialization
|
||||
|
||||
init(
|
||||
configuration: WebSearchConfiguration = .default,
|
||||
urlSession: URLSession = .shared
|
||||
) {
|
||||
self.configuration = configuration
|
||||
self.urlSession = urlSession
|
||||
self.decoder = JSONDecoder()
|
||||
self.decoder.keyDecodingStrategy = .convertFromSnakeCase
|
||||
}
|
||||
|
||||
/// Updates the base URL for the search endpoint.
|
||||
func updateBaseURL(_ url: String) {
|
||||
baseURL = url
|
||||
}
|
||||
|
||||
// MARK: - Search Methods
|
||||
|
||||
/// Performs a web search query.
|
||||
/// - Parameters:
|
||||
/// - query: The search query string
|
||||
/// - maxResults: Maximum number of results to return (defaults to config)
|
||||
/// - Returns: Formatted search results as text for LLM consumption
|
||||
/// - Throws: WebSearchError if the search fails
|
||||
func search(query: String, maxResults: Int? = nil) async throws -> String {
|
||||
let results = try await performSearch(query: query)
|
||||
return formatResults(results, query: query, maxResults: maxResults ?? configuration.maxResults)
|
||||
}
|
||||
|
||||
/// Performs a web search and returns raw results.
|
||||
/// - Parameter query: The search query string
|
||||
/// - Returns: Array of search results
|
||||
/// - Throws: WebSearchError if the search fails
|
||||
func searchRaw(query: String) async throws -> [SearchResult] {
|
||||
try await performSearch(query: query)
|
||||
}
|
||||
|
||||
// MARK: - Private Methods
|
||||
|
||||
private func performSearch(query: String) async throws -> [SearchResult] {
|
||||
guard var urlComponents = URLComponents(string: "\(configuration.baseURL)/search") else {
|
||||
throw WebSearchError.invalidURL
|
||||
}
|
||||
|
||||
urlComponents.queryItems = [
|
||||
URLQueryItem(name: "q", value: query),
|
||||
URLQueryItem(name: "format", value: "json"),
|
||||
URLQueryItem(name: "safesearch", value: String(configuration.safeSearch))
|
||||
]
|
||||
|
||||
guard let url = urlComponents.url else {
|
||||
throw WebSearchError.invalidURL
|
||||
}
|
||||
|
||||
var request = URLRequest(url: url)
|
||||
request.timeoutInterval = configuration.timeout
|
||||
request.setValue("application/json", forHTTPHeaderField: "Accept")
|
||||
request.setValue("SleepyAgent-iOS/1.0", forHTTPHeaderField: "User-Agent")
|
||||
|
||||
let data: Data
|
||||
let response: URLResponse
|
||||
|
||||
do {
|
||||
(data, response) = try await urlSession.data(for: request)
|
||||
} catch {
|
||||
throw WebSearchError.networkError(error)
|
||||
}
|
||||
|
||||
guard let httpResponse = response as? HTTPURLResponse else {
|
||||
throw WebSearchError.invalidResponse
|
||||
}
|
||||
|
||||
guard (200...299).contains(httpResponse.statusCode) else {
|
||||
throw WebSearchError.serverError(httpResponse.statusCode)
|
||||
}
|
||||
|
||||
let searchResponse: SearxngResponse
|
||||
do {
|
||||
searchResponse = try decoder.decode(SearxngResponse.self, from: data)
|
||||
} catch {
|
||||
throw WebSearchError.decodingError(error)
|
||||
}
|
||||
|
||||
return searchResponse.results
|
||||
}
|
||||
|
||||
private func formatResults(_ results: [SearchResult], query: String, maxResults: Int) -> String {
|
||||
guard !results.isEmpty else {
|
||||
return "No results found for '\(query)'"
|
||||
}
|
||||
|
||||
return results.prefix(maxResults).map { result in
|
||||
var text = "Title: \(result.title)\n"
|
||||
text += "URL: \(result.url)\n"
|
||||
text += "Content: \(result.content)"
|
||||
return text
|
||||
}.joined(separator: "\n\n")
|
||||
}
|
||||
|
||||
// MARK: - Tool Protocol Support
|
||||
|
||||
/// Executes search as a tool call with arguments.
|
||||
/// - Parameter arguments: Dictionary containing "query" key
|
||||
/// - Returns: Formatted search results
|
||||
func execute(arguments: [String: String]) async -> String {
|
||||
guard let query = arguments["query"] else {
|
||||
return "Error: 'query' parameter is required"
|
||||
}
|
||||
|
||||
do {
|
||||
return try await search(query: query)
|
||||
} catch {
|
||||
return "Error performing web search: \(error.localizedDescription)"
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - Convenience Extensions
|
||||
|
||||
extension WebSearchService {
|
||||
/// Quick search with default settings.
|
||||
static func quickSearch(query: String, baseURL: String? = nil) async throws -> String {
|
||||
var config = WebSearchConfiguration.default
|
||||
if let baseURL = baseURL {
|
||||
config.baseURL = baseURL
|
||||
}
|
||||
let service = WebSearchService(configuration: config)
|
||||
return try await service.search(query: query)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,12 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
|
||||
<plist version="1.0">
|
||||
<dict>
|
||||
<key>com.apple.security.app-sandbox</key>
|
||||
<true/>
|
||||
<key>com.apple.security.network.client</key>
|
||||
<true/>
|
||||
<key>com.apple.security.network.server</key>
|
||||
<true/>
|
||||
</dict>
|
||||
</plist>
|
||||
@@ -0,0 +1,166 @@
|
||||
import Foundation
|
||||
import SwiftUI
|
||||
import Combine
|
||||
|
||||
@MainActor
|
||||
class MainViewModel: ObservableObject {
|
||||
@Published var messages: [Message] = []
|
||||
@Published var inputText: String = ""
|
||||
@Published var isGenerating: Bool = false
|
||||
@Published var isRecording: Bool = false
|
||||
@Published var currentResponse: String = ""
|
||||
@Published var errorMessage: String?
|
||||
@Published var showError: Bool = false
|
||||
@Published var showImagePicker: Bool = false
|
||||
@Published var selectedImage: UIImage?
|
||||
@Published var conversations: [ConversationInfo] = []
|
||||
@Published var currentConversationId: UUID = UUID()
|
||||
|
||||
private let agent: Agent
|
||||
private let audioRecorder: AudioRecorder
|
||||
private let ttsService: TtsService
|
||||
private let conversationStorage: ConversationStorage
|
||||
private let llmEngine: LlmEngine
|
||||
|
||||
private var firstInputWasVoice: Bool?
|
||||
|
||||
init(agent: Agent, audioRecorder: AudioRecorder, ttsService: TtsService, conversationStorage: ConversationStorage, llmEngine: LlmEngine) {
|
||||
self.agent = agent
|
||||
self.audioRecorder = audioRecorder
|
||||
self.ttsService = ttsService
|
||||
self.conversationStorage = conversationStorage
|
||||
self.llmEngine = llmEngine
|
||||
loadConversations()
|
||||
loadCurrentConversation()
|
||||
}
|
||||
|
||||
func sendMessage() async {
|
||||
guard !inputText.isEmpty else { return }
|
||||
let text = inputText
|
||||
inputText = ""
|
||||
await processTextMessage(text)
|
||||
}
|
||||
|
||||
func sendImage(_ image: UIImage, text: String = "") async {
|
||||
selectedImage = image
|
||||
let displayText = text.isEmpty ? "[Image]" : text
|
||||
messages.append(Message.user("🖼️ \(displayText)"))
|
||||
saveConversation()
|
||||
firstInputWasVoice = false
|
||||
await generateResponse(withImage: image, text: text)
|
||||
}
|
||||
|
||||
func startRecording() {
|
||||
guard !isRecording else { return }
|
||||
isRecording = true
|
||||
if firstInputWasVoice == nil {
|
||||
firstInputWasVoice = true
|
||||
}
|
||||
audioRecorder.startRecording()
|
||||
}
|
||||
|
||||
func stopRecording() async {
|
||||
guard isRecording else { return }
|
||||
isRecording = false
|
||||
do {
|
||||
let audioData = try await audioRecorder.stopRecording()
|
||||
messages.append(Message.user("🎤 [Voice message]"))
|
||||
saveConversation()
|
||||
await generateResponse(audioData: audioData)
|
||||
} catch {
|
||||
showError(error.localizedDescription)
|
||||
}
|
||||
}
|
||||
|
||||
func toggleRecording() async {
|
||||
if isRecording {
|
||||
await stopRecording()
|
||||
} else {
|
||||
startRecording()
|
||||
}
|
||||
}
|
||||
|
||||
func newConversation() {
|
||||
saveConversation()
|
||||
currentConversationId = UUID()
|
||||
messages = []
|
||||
agent.resetConversation()
|
||||
}
|
||||
|
||||
func loadConversation(id: UUID) {
|
||||
saveConversation()
|
||||
currentConversationId = id
|
||||
messages = conversationStorage.loadConversation(id: id) ?? []
|
||||
}
|
||||
|
||||
func deleteConversation(id: UUID) {
|
||||
conversationStorage.deleteConversation(id: id)
|
||||
loadConversations()
|
||||
if currentConversationId == id {
|
||||
newConversation()
|
||||
}
|
||||
}
|
||||
|
||||
private func processTextMessage(_ text: String) async {
|
||||
if firstInputWasVoice == nil {
|
||||
firstInputWasVoice = false
|
||||
}
|
||||
messages.append(Message.user(text))
|
||||
saveConversation()
|
||||
await generateResponse()
|
||||
}
|
||||
|
||||
private func generateResponse(withImage: UIImage? = nil, text: String? = nil, audioData: Data? = nil) async {
|
||||
guard !isGenerating else { return }
|
||||
|
||||
if !llmEngine.isLoaded {
|
||||
currentResponse = "Loading model..."
|
||||
}
|
||||
|
||||
isGenerating = true
|
||||
currentResponse = ""
|
||||
defer { isGenerating = false }
|
||||
|
||||
do {
|
||||
let stream = agent.processStream(input: text ?? "", image: withImage, audioData: audioData)
|
||||
var fullResponse = ""
|
||||
for try await token in stream {
|
||||
currentResponse += token
|
||||
fullResponse += token
|
||||
}
|
||||
messages.append(Message.assistant(fullResponse))
|
||||
saveConversation()
|
||||
currentResponse = ""
|
||||
if firstInputWasVoice == true {
|
||||
ttsService.speak(fullResponse)
|
||||
}
|
||||
} catch {
|
||||
showError(error.localizedDescription)
|
||||
}
|
||||
}
|
||||
|
||||
private func saveConversation() {
|
||||
conversationStorage.saveConversation(id: currentConversationId, messages: messages)
|
||||
loadConversations()
|
||||
}
|
||||
|
||||
private func loadCurrentConversation() {
|
||||
messages = conversationStorage.loadConversation(id: currentConversationId) ?? []
|
||||
}
|
||||
|
||||
private func loadConversations() {
|
||||
conversations = conversationStorage.listConversations()
|
||||
}
|
||||
|
||||
private func showError(_ message: String) {
|
||||
errorMessage = message
|
||||
showError = true
|
||||
}
|
||||
}
|
||||
|
||||
struct ConversationInfo: Identifiable {
|
||||
let id: UUID
|
||||
let title: String
|
||||
let messageCount: Int
|
||||
let lastUpdated: Date
|
||||
}
|
||||
@@ -0,0 +1,152 @@
|
||||
import Foundation
|
||||
import SwiftUI
|
||||
|
||||
@MainActor
|
||||
class SettingsViewModel: ObservableObject {
|
||||
@Published var selectedModel: ModelVariant = .e2b
|
||||
@Published var modelPath: String = ""
|
||||
@Published var isModelLoaded: Bool = false
|
||||
@Published var isModelLoading: Bool = false
|
||||
@Published var searchServerUrl: String = ""
|
||||
@Published var delegateServerUrl: String = ""
|
||||
@Published var ttsEnabled: Bool = true
|
||||
@Published var ttsAutoMode: Bool = true
|
||||
@Published var downloadProgress: Double = 0
|
||||
@Published var isDownloading: Bool = false
|
||||
@Published var downloadStatus: String = ""
|
||||
@Published var deviceInfo: DeviceInfo = DeviceInfo()
|
||||
@Published var errorMessage: String?
|
||||
@Published var showError: Bool = false
|
||||
|
||||
private let modelDownloadService: ModelDownloadService
|
||||
private let llmEngine: LlmEngine
|
||||
private let userDefaults = UserDefaults.standard
|
||||
|
||||
init(modelDownloadService: ModelDownloadService, llmEngine: LlmEngine) {
|
||||
self.modelDownloadService = modelDownloadService
|
||||
self.llmEngine = llmEngine
|
||||
loadSettings()
|
||||
updateDeviceInfo()
|
||||
}
|
||||
|
||||
func loadModel() async {
|
||||
guard !modelPath.isEmpty else {
|
||||
showError("No model selected")
|
||||
return
|
||||
}
|
||||
isModelLoading = true
|
||||
defer { isModelLoading = false }
|
||||
do {
|
||||
try await llmEngine.loadModel(path: modelPath)
|
||||
isModelLoaded = true
|
||||
saveSettings()
|
||||
} catch {
|
||||
showError("Failed to load model: \(error.localizedDescription)")
|
||||
isModelLoaded = false
|
||||
}
|
||||
}
|
||||
|
||||
func downloadModel(variant: ModelVariant) async {
|
||||
guard !isDownloading else { return }
|
||||
isDownloading = true
|
||||
downloadProgress = 0
|
||||
|
||||
let (url, filename): (String, String)
|
||||
switch variant {
|
||||
case .e2b:
|
||||
url = "https://huggingface.co/litert-community/gemma-4-E2B-it-litert-lm/resolve/main/gemma-4-E2B-it.litertlm"
|
||||
filename = "gemma-4-E2B-it.litertlm"
|
||||
case .e4b:
|
||||
url = "https://huggingface.co/litert-community/gemma-4-E4B-it-litert-lm/resolve/main/gemma-4-E4B-it.litertlm"
|
||||
filename = "gemma-4-E4B-it.litertlm"
|
||||
case .custom:
|
||||
showError("Cannot download custom model")
|
||||
isDownloading = false
|
||||
return
|
||||
}
|
||||
|
||||
do {
|
||||
let stream = modelDownloadService.downloadModel(from: url, filename: filename)
|
||||
for try await progress in stream {
|
||||
downloadProgress = progress
|
||||
downloadStatus = String(format: "%.1f%%", progress * 100)
|
||||
}
|
||||
selectedModel = variant
|
||||
modelPath = modelDownloadService.localPath(for: filename)
|
||||
await loadModel()
|
||||
} catch {
|
||||
showError("Download failed: \(error.localizedDescription)")
|
||||
}
|
||||
isDownloading = false
|
||||
}
|
||||
|
||||
func deleteModel(variant: ModelVariant) {
|
||||
let filename: String
|
||||
switch variant {
|
||||
case .e2b: filename = "gemma-4-E2B-it.litertlm"
|
||||
case .e4b: filename = "gemma-4-E4B-it.litertlm"
|
||||
case .custom: return
|
||||
}
|
||||
modelDownloadService.deleteModel(filename: filename)
|
||||
if selectedModel == variant {
|
||||
selectedModel = .e2b
|
||||
modelPath = ""
|
||||
isModelLoaded = false
|
||||
}
|
||||
}
|
||||
|
||||
func isModelDownloaded(variant: ModelVariant) -> Bool {
|
||||
let filename: String
|
||||
switch variant {
|
||||
case .e2b: filename = "gemma-4-E2B-it.litertlm"
|
||||
case .e4b: filename = "gemma-4-E4B-it.litertlm"
|
||||
case .custom: return !modelPath.isEmpty
|
||||
}
|
||||
return modelDownloadService.isDownloaded(filename: filename)
|
||||
}
|
||||
|
||||
func saveSettings() {
|
||||
userDefaults.set(selectedModel.rawValue, forKey: "selectedModel")
|
||||
userDefaults.set(modelPath, forKey: "modelPath")
|
||||
userDefaults.set(searchServerUrl, forKey: "searchServerUrl")
|
||||
userDefaults.set(delegateServerUrl, forKey: "delegateServerUrl")
|
||||
userDefaults.set(ttsEnabled, forKey: "ttsEnabled")
|
||||
userDefaults.set(ttsAutoMode, forKey: "ttsAutoMode")
|
||||
}
|
||||
|
||||
func updateDeviceInfo() {
|
||||
deviceInfo = DeviceInfo.current()
|
||||
}
|
||||
|
||||
private func loadSettings() {
|
||||
if let modelRaw = userDefaults.string(forKey: "selectedModel"),
|
||||
let model = ModelVariant(rawValue: modelRaw) {
|
||||
selectedModel = model
|
||||
}
|
||||
modelPath = userDefaults.string(forKey: "modelPath") ?? ""
|
||||
searchServerUrl = userDefaults.string(forKey: "searchServerUrl") ?? ""
|
||||
delegateServerUrl = userDefaults.string(forKey: "delegateServerUrl") ?? ""
|
||||
ttsEnabled = userDefaults.object(forKey: "ttsEnabled") as? Bool ?? true
|
||||
ttsAutoMode = userDefaults.object(forKey: "ttsAutoMode") as? Bool ?? true
|
||||
isModelLoaded = llmEngine.isLoaded
|
||||
}
|
||||
|
||||
private func showError(_ message: String) {
|
||||
errorMessage = message
|
||||
showError = true
|
||||
}
|
||||
}
|
||||
|
||||
struct DeviceInfo {
|
||||
let totalRAM: String
|
||||
let freeRAM: String
|
||||
let deviceModel: String
|
||||
let systemVersion: String
|
||||
|
||||
static func current() -> DeviceInfo {
|
||||
let device = UIDevice.current
|
||||
let physicalMemory = ProcessInfo.processInfo.physicalMemory
|
||||
let totalRAM = String(format: "%.0f GB", Double(physicalMemory) / 1024 / 1024 / 1024)
|
||||
return DeviceInfo(totalRAM: totalRAM, freeRAM: "Unknown", deviceModel: device.model, systemVersion: "\(device.systemName) \(device.systemVersion)")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,273 @@
|
||||
import SwiftUI
|
||||
|
||||
struct InputBar: View {
|
||||
@Binding var text: String
|
||||
let onSend: (String) -> Void
|
||||
let onVoiceTap: () -> Void
|
||||
let onImageTap: () -> Void
|
||||
let isRecording: Bool
|
||||
let isProcessing: Bool
|
||||
let isExecutingTool: Bool
|
||||
|
||||
@FocusState private var isFocused: Bool
|
||||
@State private var showVoiceOverlay = false
|
||||
|
||||
var body: some View {
|
||||
HStack(spacing: 12) {
|
||||
// Text input field
|
||||
textField
|
||||
|
||||
// Action buttons
|
||||
actionButtons
|
||||
}
|
||||
.padding(.horizontal, 4)
|
||||
.padding(.vertical, 8)
|
||||
.background(Color(.systemBackground))
|
||||
.overlay(
|
||||
// Voice recording overlay
|
||||
voiceOverlay
|
||||
)
|
||||
}
|
||||
|
||||
private var textField: some View {
|
||||
HStack(spacing: 8) {
|
||||
TextField(placeholderText, text: $text, axis: .vertical)
|
||||
.focused($isFocused)
|
||||
.lineLimit(1...4)
|
||||
.disabled(isProcessing || isExecutingTool)
|
||||
.submitLabel(.send)
|
||||
.onSubmit {
|
||||
sendMessage()
|
||||
}
|
||||
|
||||
if !text.isEmpty {
|
||||
Button(action: { text = "" }) {
|
||||
Image(systemName: "xmark.circle.fill")
|
||||
.foregroundColor(.secondary)
|
||||
}
|
||||
}
|
||||
}
|
||||
.padding(.horizontal, 12)
|
||||
.padding(.vertical, 10)
|
||||
.background(Color(.systemGray6))
|
||||
.cornerRadius(20)
|
||||
}
|
||||
|
||||
private var actionButtons: some View {
|
||||
HStack(spacing: 8) {
|
||||
// Image picker button
|
||||
Button(action: onImageTap) {
|
||||
Image(systemName: "photo")
|
||||
.font(.system(size: 22))
|
||||
.frame(width: 40, height: 40)
|
||||
}
|
||||
.disabled(isProcessing || isExecutingTool)
|
||||
.opacity(isProcessing || isExecutingTool ? 0.5 : 1)
|
||||
|
||||
// Voice/Record button
|
||||
Button(action: handleVoiceTap) {
|
||||
ZStack {
|
||||
Circle()
|
||||
.fill(buttonBackgroundColor)
|
||||
.frame(width: 44, height: 44)
|
||||
|
||||
if isProcessing || isExecutingTool {
|
||||
ProgressView()
|
||||
.scaleEffect(0.8)
|
||||
.tint(.white)
|
||||
} else if isRecording {
|
||||
Image(systemName: "stop.fill")
|
||||
.font(.system(size: 20))
|
||||
.foregroundColor(.white)
|
||||
} else {
|
||||
Image(systemName: "mic.fill")
|
||||
.font(.system(size: 20))
|
||||
.foregroundColor(.white)
|
||||
}
|
||||
}
|
||||
}
|
||||
.disabled(isProcessing || isExecutingTool)
|
||||
|
||||
// Send button
|
||||
Button(action: sendMessage) {
|
||||
Image(systemName: "arrow.up.circle.fill")
|
||||
.font(.system(size: 32))
|
||||
.foregroundColor(canSend ? .accentColor : .secondary.opacity(0.5))
|
||||
}
|
||||
.disabled(!canSend)
|
||||
}
|
||||
}
|
||||
|
||||
@ViewBuilder
|
||||
private var voiceOverlay: some View {
|
||||
if isRecording {
|
||||
VoiceRecordingOverlay()
|
||||
.transition(.opacity)
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - Computed Properties
|
||||
|
||||
private var placeholderText: String {
|
||||
if isExecutingTool {
|
||||
return "🔧 Executing tool..."
|
||||
} else if isProcessing {
|
||||
return "Thinking..."
|
||||
} else if isRecording {
|
||||
return "Recording..."
|
||||
} else {
|
||||
return "Type a message..."
|
||||
}
|
||||
}
|
||||
|
||||
private var buttonBackgroundColor: Color {
|
||||
if isRecording {
|
||||
return .red
|
||||
} else if isProcessing || isExecutingTool {
|
||||
return .orange
|
||||
} else {
|
||||
return .accentColor
|
||||
}
|
||||
}
|
||||
|
||||
private var canSend: Bool {
|
||||
!text.trimmingCharacters(in: .whitespacesAndNewlines).isEmpty &&
|
||||
!isProcessing &&
|
||||
!isExecutingTool
|
||||
}
|
||||
|
||||
// MARK: - Actions
|
||||
|
||||
private func sendMessage() {
|
||||
let trimmed = text.trimmingCharacters(in: .whitespacesAndNewlines)
|
||||
guard !trimmed.isEmpty else { return }
|
||||
|
||||
onSend(trimmed)
|
||||
text = ""
|
||||
isFocused = false
|
||||
}
|
||||
|
||||
private func handleVoiceTap() {
|
||||
withAnimation(.spring()) {
|
||||
onVoiceTap()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - Voice Recording Overlay
|
||||
|
||||
struct VoiceRecordingOverlay: View {
|
||||
@State private var pulseScale: CGFloat = 1.0
|
||||
|
||||
var body: some View {
|
||||
ZStack {
|
||||
Color.black.opacity(0.5)
|
||||
.ignoresSafeArea()
|
||||
|
||||
VStack(spacing: 24) {
|
||||
Spacer()
|
||||
|
||||
// Recording indicator
|
||||
ZStack {
|
||||
Circle()
|
||||
.fill(Color.red.opacity(0.3))
|
||||
.frame(width: 120, height: 120)
|
||||
.scaleEffect(pulseScale)
|
||||
|
||||
Circle()
|
||||
.fill(Color.red)
|
||||
.frame(width: 80, height: 80)
|
||||
|
||||
Image(systemName: "mic.fill")
|
||||
.font(.system(size: 40))
|
||||
.foregroundColor(.white)
|
||||
}
|
||||
|
||||
Text("Recording...")
|
||||
.font(.title3)
|
||||
.foregroundColor(.white)
|
||||
|
||||
Text("Tap stop button to finish")
|
||||
.font(.body)
|
||||
.foregroundColor(.white.opacity(0.7))
|
||||
|
||||
Spacer()
|
||||
}
|
||||
}
|
||||
.onAppear {
|
||||
withAnimation(.easeInOut(duration: 1).repeatForever(autoreverses: true)) {
|
||||
pulseScale = 1.3
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - Audio Waveform View (Visual feedback while recording)
|
||||
|
||||
struct AudioWaveformView: View {
|
||||
@State private var bars: [CGFloat] = Array(repeating: 0.5, count: 20)
|
||||
|
||||
var body: some View {
|
||||
HStack(spacing: 4) {
|
||||
ForEach(0..<bars.count, id: \.self) { index in
|
||||
RoundedRectangle(cornerRadius: 2)
|
||||
.fill(Color.red)
|
||||
.frame(width: 4, height: bars[index] * 40)
|
||||
.animation(.easeInOut(duration: 0.1).delay(Double(index) * 0.02), value: bars[index])
|
||||
}
|
||||
}
|
||||
.frame(height: 40)
|
||||
.onAppear {
|
||||
startAnimating()
|
||||
}
|
||||
}
|
||||
|
||||
private func startAnimating() {
|
||||
Timer.scheduledTimer(withTimeInterval: 0.1, repeats: true) { _ in
|
||||
for i in bars.indices {
|
||||
bars[i] = CGFloat.random(in: 0.3...1.0)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - Preview
|
||||
|
||||
struct InputBar_Previews: PreviewProvider {
|
||||
static var previews: some View {
|
||||
VStack {
|
||||
InputBar(
|
||||
text: .constant("Hello"),
|
||||
onSend: { _ in },
|
||||
onVoiceTap: {},
|
||||
onImageTap: {},
|
||||
isRecording: false,
|
||||
isProcessing: false,
|
||||
isExecutingTool: false
|
||||
)
|
||||
|
||||
InputBar(
|
||||
text: .constant(""),
|
||||
onSend: { _ in },
|
||||
onVoiceTap: {},
|
||||
onImageTap: {},
|
||||
isRecording: true,
|
||||
isProcessing: false,
|
||||
isExecutingTool: false
|
||||
)
|
||||
|
||||
InputBar(
|
||||
text: .constant(""),
|
||||
onSend: { _ in },
|
||||
onVoiceTap: {},
|
||||
onImageTap: {},
|
||||
isRecording: false,
|
||||
isProcessing: true,
|
||||
isExecutingTool: false
|
||||
)
|
||||
|
||||
Spacer()
|
||||
}
|
||||
.padding()
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,333 @@
|
||||
import SwiftUI
|
||||
|
||||
struct MainView: View {
|
||||
@StateObject private var viewModel = MainViewModel()
|
||||
@State private var showingSettings = false
|
||||
@State private var showingSidebar = false
|
||||
@State private var inputText = ""
|
||||
@State private var scrollProxy: ScrollViewProxy?
|
||||
|
||||
var body: some View {
|
||||
NavigationView {
|
||||
ZStack {
|
||||
// Main content
|
||||
VStack(spacing: 0) {
|
||||
// Messages list
|
||||
messagesList
|
||||
|
||||
// Loading indicator
|
||||
if viewModel.isLoading {
|
||||
ProgressView()
|
||||
.padding(.vertical, 8)
|
||||
}
|
||||
|
||||
// Input bar
|
||||
InputBar(
|
||||
text: $inputText,
|
||||
onSend: { text in
|
||||
viewModel.sendMessage(text: text)
|
||||
inputText = ""
|
||||
},
|
||||
onVoiceTap: {
|
||||
viewModel.toggleRecording()
|
||||
},
|
||||
onImageTap: {
|
||||
viewModel.showImagePicker = true
|
||||
},
|
||||
isRecording: viewModel.isRecording,
|
||||
isProcessing: viewModel.isProcessing,
|
||||
isExecutingTool: viewModel.isExecutingTool
|
||||
)
|
||||
.padding(.horizontal)
|
||||
.padding(.bottom, 8)
|
||||
}
|
||||
|
||||
// Sidebar overlay
|
||||
if showingSidebar {
|
||||
sidebarOverlay
|
||||
}
|
||||
}
|
||||
.navigationTitle("Sleepy Agent")
|
||||
.navigationBarTitleDisplayMode(.large)
|
||||
.toolbar {
|
||||
ToolbarItem(placement: .navigationBarLeading) {
|
||||
Button(action: { showingSidebar.toggle() }) {
|
||||
Image(systemName: "line.3.horizontal")
|
||||
}
|
||||
}
|
||||
ToolbarItem(placement: .navigationBarTrailing) {
|
||||
Button(action: { showingSettings = true }) {
|
||||
Image(systemName: "gear")
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
.sheet(isPresented: $showingSettings) {
|
||||
SettingsView()
|
||||
}
|
||||
.sheet(isPresented: $viewModel.showImagePicker) {
|
||||
ImagePicker(selectedImage: $viewModel.selectedImage)
|
||||
}
|
||||
.alert("Error", isPresented: $viewModel.showError) {
|
||||
Button("OK") { viewModel.dismissError() }
|
||||
} message: {
|
||||
Text(viewModel.errorMessage)
|
||||
}
|
||||
.onChange(of: viewModel.messages.count) { _ in
|
||||
scrollToBottom()
|
||||
}
|
||||
.onChange(of: viewModel.streamingText) { _ in
|
||||
scrollToBottom()
|
||||
}
|
||||
}
|
||||
|
||||
private var messagesList: some View {
|
||||
ScrollViewReader { proxy in
|
||||
ScrollView {
|
||||
LazyVStack(spacing: 12) {
|
||||
if viewModel.messages.isEmpty {
|
||||
welcomeView
|
||||
} else {
|
||||
ForEach(viewModel.messages) { message in
|
||||
MessageBubble(message: message)
|
||||
.id(message.id)
|
||||
}
|
||||
|
||||
// Streaming response
|
||||
if !viewModel.streamingText.isEmpty && viewModel.isResponding {
|
||||
MessageBubble(
|
||||
message: ChatMessage(
|
||||
id: "streaming",
|
||||
text: viewModel.streamingText + (viewModel.isSpeaking ? "▋" : ""),
|
||||
isUser: false,
|
||||
isStreaming: true
|
||||
)
|
||||
)
|
||||
.id("streaming")
|
||||
}
|
||||
}
|
||||
}
|
||||
.padding()
|
||||
}
|
||||
.onAppear {
|
||||
scrollProxy = proxy
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private var welcomeView: some View {
|
||||
VStack(spacing: 16) {
|
||||
Spacer()
|
||||
|
||||
Text("👋 Welcome to Sleepy Agent")
|
||||
.font(.title2)
|
||||
.fontWeight(.semibold)
|
||||
.foregroundColor(.accentColor)
|
||||
|
||||
Text("Tap the microphone to start speaking\nor type a message below")
|
||||
.font(.body)
|
||||
.foregroundColor(.secondary)
|
||||
.multilineTextAlignment(.center)
|
||||
|
||||
if viewModel.uiState == .error {
|
||||
Text(viewModel.errorMessage)
|
||||
.font(.callout)
|
||||
.foregroundColor(.red)
|
||||
.multilineTextAlignment(.center)
|
||||
.padding(.horizontal, 32)
|
||||
}
|
||||
|
||||
Spacer()
|
||||
}
|
||||
.frame(maxWidth: .infinity, minHeight: 300)
|
||||
}
|
||||
|
||||
private var sidebarOverlay: some View {
|
||||
GeometryReader { geometry in
|
||||
HStack(spacing: 0) {
|
||||
// Sidebar content
|
||||
ConversationSidebar(
|
||||
conversations: viewModel.conversations,
|
||||
currentId: viewModel.currentConversationId,
|
||||
onSelect: { id in
|
||||
viewModel.loadConversation(id: id)
|
||||
showingSidebar = false
|
||||
},
|
||||
onNewChat: {
|
||||
viewModel.startNewConversation()
|
||||
showingSidebar = false
|
||||
},
|
||||
onDelete: { id in
|
||||
viewModel.deleteConversation(id: id)
|
||||
}
|
||||
)
|
||||
.frame(width: min(300, geometry.size.width * 0.75))
|
||||
.background(Color(.systemBackground))
|
||||
|
||||
// Tap to dismiss
|
||||
Color.black.opacity(0.3)
|
||||
.onTapGesture {
|
||||
showingSidebar = false
|
||||
}
|
||||
}
|
||||
}
|
||||
.transition(.move(edge: .leading))
|
||||
.zIndex(1)
|
||||
}
|
||||
|
||||
private func scrollToBottom() {
|
||||
guard let proxy = scrollProxy else { return }
|
||||
|
||||
withAnimation {
|
||||
if viewModel.isResponding && !viewModel.streamingText.isEmpty {
|
||||
proxy.scrollTo("streaming", anchor: .bottom)
|
||||
} else if let lastMessage = viewModel.messages.last {
|
||||
proxy.scrollTo(lastMessage.id, anchor: .bottom)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - Conversation Sidebar
|
||||
|
||||
struct ConversationSidebar: View {
|
||||
let conversations: [ConversationInfo]
|
||||
let currentId: String
|
||||
let onSelect: (String) -> Void
|
||||
let onNewChat: () -> Void
|
||||
let onDelete: (String) -> Void
|
||||
|
||||
var body: some View {
|
||||
VStack(alignment: .leading, spacing: 0) {
|
||||
// Header
|
||||
HStack {
|
||||
Text("Chat History")
|
||||
.font(.headline)
|
||||
|
||||
Spacer()
|
||||
|
||||
Button(action: onNewChat) {
|
||||
Image(systemName: "plus")
|
||||
}
|
||||
}
|
||||
.padding()
|
||||
|
||||
Divider()
|
||||
|
||||
// New Chat button
|
||||
Button(action: onNewChat) {
|
||||
Label("New Chat", systemImage: "plus.circle")
|
||||
.frame(maxWidth: .infinity, alignment: .leading)
|
||||
}
|
||||
.buttonStyle(.bordered)
|
||||
.padding()
|
||||
|
||||
Divider()
|
||||
|
||||
// Conversations list
|
||||
if conversations.isEmpty {
|
||||
Spacer()
|
||||
Text("No previous chats")
|
||||
.foregroundColor(.secondary)
|
||||
.frame(maxWidth: .infinity, alignment: .center)
|
||||
Spacer()
|
||||
} else {
|
||||
List {
|
||||
ForEach(conversations) { conversation in
|
||||
ConversationRow(
|
||||
conversation: conversation,
|
||||
isSelected: conversation.id == currentId
|
||||
)
|
||||
.contentShape(Rectangle())
|
||||
.onTapGesture {
|
||||
onSelect(conversation.id)
|
||||
}
|
||||
.swipeActions(edge: .trailing) {
|
||||
Button(role: .destructive) {
|
||||
onDelete(conversation.id)
|
||||
} label: {
|
||||
Label("Delete", systemImage: "trash")
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
.listStyle(.plain)
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
struct ConversationRow: View {
|
||||
let conversation: ConversationInfo
|
||||
let isSelected: Bool
|
||||
|
||||
private var dateFormatter: DateFormatter {
|
||||
let formatter = DateFormatter()
|
||||
formatter.dateStyle = .short
|
||||
formatter.timeStyle = .short
|
||||
return formatter
|
||||
}
|
||||
|
||||
var body: some View {
|
||||
VStack(alignment: .leading, spacing: 4) {
|
||||
Text(conversation.title)
|
||||
.font(.body)
|
||||
.lineLimit(1)
|
||||
.foregroundColor(isSelected ? .accentColor : .primary)
|
||||
|
||||
Text("\(dateFormatter.string(from: conversation.date)) • \(conversation.messageCount) messages")
|
||||
.font(.caption)
|
||||
.foregroundColor(.secondary)
|
||||
}
|
||||
.padding(.vertical, 4)
|
||||
.background(isSelected ? Color.accentColor.opacity(0.1) : Color.clear)
|
||||
.cornerRadius(8)
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - Image Picker
|
||||
|
||||
struct ImagePicker: UIViewControllerRepresentable {
|
||||
@Binding var selectedImage: UIImage?
|
||||
@Environment(\.presentationMode) var presentationMode
|
||||
|
||||
func makeUIViewController(context: Context) -> UIImagePickerController {
|
||||
let picker = UIImagePickerController()
|
||||
picker.delegate = context.coordinator
|
||||
picker.sourceType = .photoLibrary
|
||||
return picker
|
||||
}
|
||||
|
||||
func updateUIViewController(_ uiViewController: UIImagePickerController, context: Context) {}
|
||||
|
||||
func makeCoordinator() -> Coordinator {
|
||||
Coordinator(self)
|
||||
}
|
||||
|
||||
class Coordinator: NSObject, UIImagePickerControllerDelegate, UINavigationControllerDelegate {
|
||||
let parent: ImagePicker
|
||||
|
||||
init(_ parent: ImagePicker) {
|
||||
self.parent = parent
|
||||
}
|
||||
|
||||
func imagePickerController(_ picker: UIImagePickerController, didFinishPickingMediaWithInfo info: [UIImagePickerController.InfoKey: Any]) {
|
||||
if let image = info[.originalImage] as? UIImage {
|
||||
parent.selectedImage = image
|
||||
}
|
||||
parent.presentationMode.wrappedValue.dismiss()
|
||||
}
|
||||
|
||||
func imagePickerControllerDidCancel(_ picker: UIImagePickerController) {
|
||||
parent.presentationMode.wrappedValue.dismiss()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - Preview
|
||||
|
||||
struct MainView_Previews: PreviewProvider {
|
||||
static var previews: some View {
|
||||
MainView()
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,364 @@
|
||||
import SwiftUI
|
||||
|
||||
struct MessageBubble: View {
|
||||
let message: ChatMessage
|
||||
@State private var isCopied = false
|
||||
|
||||
var body: some View {
|
||||
HStack {
|
||||
if message.isUser {
|
||||
Spacer(minLength: 60)
|
||||
}
|
||||
|
||||
VStack(alignment: message.isUser ? .trailing : .leading, spacing: 4) {
|
||||
// Tool call indicator
|
||||
if message.isToolCall {
|
||||
Label("Tool Call", systemImage: "wrench.fill")
|
||||
.font(.caption)
|
||||
.foregroundColor(.orange)
|
||||
.padding(.horizontal, 12)
|
||||
.padding(.top, 8)
|
||||
}
|
||||
|
||||
// Message content
|
||||
messageContent
|
||||
.padding(.horizontal, 12)
|
||||
.padding(.vertical, message.isToolCall ? 4 : 10)
|
||||
.background(backgroundColor)
|
||||
.foregroundColor(foregroundColor)
|
||||
.cornerRadius(16, corners: cornerRadii)
|
||||
.contextMenu {
|
||||
Button(action: copyText) {
|
||||
Label(isCopied ? "Copied!" : "Copy", systemImage: isCopied ? "checkmark" : "doc.on.doc")
|
||||
}
|
||||
|
||||
if !message.isUser {
|
||||
Button(action: { /* Regenerate action */ }) {
|
||||
Label("Regenerate", systemImage: "arrow.clockwise")
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// Timestamp
|
||||
if !message.isStreaming {
|
||||
Text(formattedTime)
|
||||
.font(.caption2)
|
||||
.foregroundColor(.secondary)
|
||||
.padding(.horizontal, 4)
|
||||
}
|
||||
}
|
||||
|
||||
if !message.isUser {
|
||||
Spacer(minLength: 60)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@ViewBuilder
|
||||
private var messageContent: some View {
|
||||
if message.isUser {
|
||||
// Plain text for user messages
|
||||
Text(message.text)
|
||||
.font(.body)
|
||||
.textSelection(.enabled)
|
||||
} else {
|
||||
// Markdown for AI messages
|
||||
MarkdownText(message.text)
|
||||
.font(.body)
|
||||
}
|
||||
}
|
||||
|
||||
private var backgroundColor: Color {
|
||||
if message.isToolCall {
|
||||
return Color.orange.opacity(0.15)
|
||||
}
|
||||
return message.isUser ? Color.accentColor.opacity(0.9) : Color(.systemGray5)
|
||||
}
|
||||
|
||||
private var foregroundColor: Color {
|
||||
if message.isToolCall {
|
||||
return .orange
|
||||
}
|
||||
return message.isUser ? .white : .primary
|
||||
}
|
||||
|
||||
private var cornerRadii: UIRectCorner {
|
||||
if message.isUser {
|
||||
return [.topLeft, .topRight, .bottomLeft]
|
||||
} else {
|
||||
return [.topLeft, .topRight, .bottomRight]
|
||||
}
|
||||
}
|
||||
|
||||
private var formattedTime: String {
|
||||
let formatter = DateFormatter()
|
||||
formatter.timeStyle = .short
|
||||
return formatter.string(from: message.timestamp)
|
||||
}
|
||||
|
||||
private func copyText() {
|
||||
UIPasteboard.general.string = message.text
|
||||
isCopied = true
|
||||
|
||||
DispatchQueue.main.asyncAfter(deadline: .now() + 2) {
|
||||
isCopied = false
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - Markdown Text View
|
||||
|
||||
struct MarkdownText: UIViewRepresentable {
|
||||
let markdown: String
|
||||
var font: UIFont = .preferredFont(forTextStyle: .body)
|
||||
|
||||
func makeUIView(context: Context) -> UITextView {
|
||||
let textView = UITextView()
|
||||
textView.isEditable = false
|
||||
textView.isSelectable = true
|
||||
textView.isScrollEnabled = false
|
||||
textView.backgroundColor = .clear
|
||||
textView.textContainerInset = .zero
|
||||
textView.textContainer.lineFragmentPadding = 0
|
||||
return textView
|
||||
}
|
||||
|
||||
func updateUIView(_ textView: UITextView, context: Context) {
|
||||
// Use NSAttributedString with markdown parsing for iOS 15+
|
||||
if let attributedString = parseMarkdown(markdown) {
|
||||
textView.attributedText = attributedString
|
||||
} else {
|
||||
textView.text = markdown
|
||||
}
|
||||
}
|
||||
|
||||
private func parseMarkdown(_ text: String) -> NSAttributedString? {
|
||||
// Basic markdown parsing using data detectors and attributes
|
||||
let attributedString = NSMutableAttributedString(string: text)
|
||||
|
||||
// Apply base font
|
||||
let range = NSRange(location: 0, length: attributedString.length)
|
||||
attributedString.addAttribute(.font, value: font, range: range)
|
||||
|
||||
// Parse inline code `code`
|
||||
parseInlineCode(in: attributedString)
|
||||
|
||||
// Parse bold **text**
|
||||
parseBold(in: attributedString)
|
||||
|
||||
// Parse italic *text*
|
||||
parseItalic(in: attributedString)
|
||||
|
||||
// Parse code blocks ```code```
|
||||
parseCodeBlocks(in: attributedString)
|
||||
|
||||
// Parse headers
|
||||
parseHeaders(in: attributedString)
|
||||
|
||||
// Parse links [text](url)
|
||||
parseLinks(in: attributedString)
|
||||
|
||||
return attributedString
|
||||
}
|
||||
|
||||
private func parseInlineCode(in string: NSMutableAttributedString) {
|
||||
let pattern = "`([^`]+)`"
|
||||
guard let regex = try? NSRegularExpression(pattern: pattern, options: []) else { return }
|
||||
|
||||
let matches = regex.matches(in: string.string, options: [], range: NSRange(location: 0, length: string.length))
|
||||
|
||||
for match in matches.reversed() {
|
||||
let codeRange = match.range(at: 1)
|
||||
let fullRange = match.range
|
||||
|
||||
string.addAttribute(.font, value: UIFont.monospacedSystemFont(ofSize: font.pointSize * 0.9, weight: .regular), range: codeRange)
|
||||
string.addAttribute(.backgroundColor, value: UIColor.systemGray5, range: codeRange)
|
||||
string.addAttribute(.foregroundColor, value: UIColor.systemRed, range: codeRange)
|
||||
|
||||
// Remove backticks
|
||||
string.replaceCharacters(in: NSRange(location: fullRange.location, length: 1), with: "")
|
||||
string.replaceCharacters(in: NSRange(location: fullRange.location + codeRange.length, length: 1), with: "")
|
||||
}
|
||||
}
|
||||
|
||||
private func parseBold(in string: NSMutableAttributedString) {
|
||||
let pattern = "\\*\\*([^*]+)\\*\\*"
|
||||
guard let regex = try? NSRegularExpression(pattern: pattern, options: []) else { return }
|
||||
|
||||
let matches = regex.matches(in: string.string, options: [], range: NSRange(location: 0, length: string.length))
|
||||
|
||||
for match in matches.reversed() {
|
||||
let boldRange = match.range(at: 1)
|
||||
let fullRange = match.range
|
||||
|
||||
string.addAttribute(.font, value: UIFont.systemFont(ofSize: font.pointSize, weight: .bold), range: boldRange)
|
||||
|
||||
// Remove **
|
||||
string.replaceCharacters(in: NSRange(location: fullRange.location, length: 2), with: "")
|
||||
string.replaceCharacters(in: NSRange(location: fullRange.location + boldRange.length, length: 2), with: "")
|
||||
}
|
||||
}
|
||||
|
||||
private func parseItalic(in string: NSMutableAttributedString) {
|
||||
let pattern = "(?<!\\*)\\*([^*]+)\\*(?!\\*)"
|
||||
guard let regex = try? NSRegularExpression(pattern: pattern, options: []) else { return }
|
||||
|
||||
let matches = regex.matches(in: string.string, options: [], range: NSRange(location: 0, length: string.length))
|
||||
|
||||
for match in matches.reversed() {
|
||||
let italicRange = match.range(at: 1)
|
||||
let fullRange = match.range
|
||||
|
||||
string.addAttribute(.font, value: UIFont.italicSystemFont(ofSize: font.pointSize), range: italicRange)
|
||||
|
||||
// Remove *
|
||||
string.replaceCharacters(in: NSRange(location: fullRange.location, length: 1), with: "")
|
||||
string.replaceCharacters(in: NSRange(location: fullRange.location + italicRange.length, length: 1), with: "")
|
||||
}
|
||||
}
|
||||
|
||||
private func parseCodeBlocks(in string: NSMutableAttributedString) {
|
||||
let pattern = "```(?:\\w+\\n)?([^`]+)```"
|
||||
guard let regex = try? NSRegularExpression(pattern: pattern, options: [.dotMatchesLineSeparators]) else { return }
|
||||
|
||||
let matches = regex.matches(in: string.string, options: [], range: NSRange(location: 0, length: string.length))
|
||||
|
||||
for match in matches.reversed() {
|
||||
let codeRange = match.range(at: 1)
|
||||
let fullRange = match.range
|
||||
|
||||
// Style the code block
|
||||
string.addAttribute(.font, value: UIFont.monospacedSystemFont(ofSize: font.pointSize * 0.85, weight: .regular), range: codeRange)
|
||||
string.addAttribute(.backgroundColor, value: UIColor.systemGray6, range: codeRange)
|
||||
|
||||
// Replace the entire block with just the code
|
||||
let code = (string.string as NSString).substring(with: codeRange)
|
||||
string.replaceCharacters(in: fullRange, with: "\n\(code)\n")
|
||||
}
|
||||
}
|
||||
|
||||
private func parseHeaders(in string: NSMutableAttributedString) {
|
||||
let pattern = "^(#{1,6})\\s*(.+)$"
|
||||
guard let regex = try? NSRegularExpression(pattern: pattern, options: [.anchorsMatchLines]) else { return }
|
||||
|
||||
let matches = regex.matches(in: string.string, options: [], range: NSRange(location: 0, length: string.length))
|
||||
|
||||
for match in matches.reversed() {
|
||||
let headerRange = match.range(at: 2)
|
||||
let hashesRange = match.range(at: 1)
|
||||
let level = (string.string as NSString).substring(with: hashesRange).count
|
||||
|
||||
let fontSize = font.pointSize + CGFloat(6 - level) * 2
|
||||
string.addAttribute(.font, value: UIFont.boldSystemFont(ofSize: fontSize), range: headerRange)
|
||||
|
||||
// Remove #
|
||||
string.replaceCharacters(in: NSRange(location: match.range.location, length: hashesRange.length + 1), with: "")
|
||||
}
|
||||
}
|
||||
|
||||
private func parseLinks(in string: NSMutableAttributedString) {
|
||||
let pattern = "\\[([^\\]]+)\\]\\(([^\\)]+)\\)"
|
||||
guard let regex = try? NSRegularExpression(pattern: pattern, options: []) else { return }
|
||||
|
||||
let matches = regex.matches(in: string.string, options: [], range: NSRange(location: 0, length: string.length))
|
||||
|
||||
for match in matches.reversed() {
|
||||
let textRange = match.range(at: 1)
|
||||
let urlRange = match.range(at: 2)
|
||||
let fullRange = match.range
|
||||
|
||||
let urlString = (string.string as NSString).substring(with: urlRange)
|
||||
if let url = URL(string: urlString) {
|
||||
string.addAttribute(.link, value: url, range: textRange)
|
||||
}
|
||||
string.addAttribute(.foregroundColor, value: UIColor.systemBlue, range: textRange)
|
||||
string.addAttribute(.underlineStyle, value: NSUnderlineStyle.single.rawValue, range: textRange)
|
||||
|
||||
// Replace with just the text
|
||||
let text = (string.string as NSString).substring(with: textRange)
|
||||
string.replaceCharacters(in: fullRange, with: text)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - Rounded Corner Modifier
|
||||
|
||||
extension View {
|
||||
func cornerRadius(_ radius: CGFloat, corners: UIRectCorner) -> some View {
|
||||
clipShape(RoundedCorner(radius: radius, corners: corners))
|
||||
}
|
||||
}
|
||||
|
||||
struct RoundedCorner: Shape {
|
||||
var radius: CGFloat = .infinity
|
||||
var corners: UIRectCorner = .allCorners
|
||||
|
||||
func path(in rect: CGRect) -> Path {
|
||||
let path = UIBezierPath(
|
||||
roundedRect: rect,
|
||||
byRoundingCorners: corners,
|
||||
cornerRadii: CGSize(width: radius, height: radius)
|
||||
)
|
||||
return Path(path.cgPath)
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - Chat Message Model
|
||||
|
||||
struct ChatMessage: Identifiable {
|
||||
let id: String
|
||||
var text: String
|
||||
let isUser: Bool
|
||||
let isToolCall: Bool
|
||||
let timestamp: Date
|
||||
let isStreaming: Bool
|
||||
|
||||
init(
|
||||
id: String = UUID().uuidString,
|
||||
text: String,
|
||||
isUser: Bool,
|
||||
isToolCall: Bool = false,
|
||||
timestamp: Date = Date(),
|
||||
isStreaming: Bool = false
|
||||
) {
|
||||
self.id = id
|
||||
self.text = text
|
||||
self.isUser = isUser
|
||||
self.isToolCall = isToolCall
|
||||
self.timestamp = timestamp
|
||||
self.isStreaming = isStreaming
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - Preview
|
||||
|
||||
struct MessageBubble_Previews: PreviewProvider {
|
||||
static var previews: some View {
|
||||
VStack(spacing: 16) {
|
||||
MessageBubble(message: ChatMessage(
|
||||
text: "Hello! How can I help you today?",
|
||||
isUser: false
|
||||
))
|
||||
|
||||
MessageBubble(message: ChatMessage(
|
||||
text: "I have a question about Swift programming.",
|
||||
isUser: true
|
||||
))
|
||||
|
||||
MessageBubble(message: ChatMessage(
|
||||
text: "Here's some `inline code` and **bold text**.",
|
||||
isUser: false
|
||||
))
|
||||
|
||||
MessageBubble(message: ChatMessage(
|
||||
text: "Searching the web...",
|
||||
isUser: false,
|
||||
isToolCall: true
|
||||
))
|
||||
|
||||
Spacer()
|
||||
}
|
||||
.padding()
|
||||
.previewLayout(.sizeThatFits)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,345 @@
|
||||
import SwiftUI
|
||||
|
||||
struct SettingsView: View {
|
||||
@StateObject private var viewModel = SettingsViewModel()
|
||||
@Environment(\.dismiss) private var dismiss
|
||||
|
||||
var body: some View {
|
||||
NavigationView {
|
||||
Form {
|
||||
// Model Section
|
||||
modelSection
|
||||
|
||||
// Server Section
|
||||
serverSection
|
||||
|
||||
// TTS Section
|
||||
ttsSection
|
||||
|
||||
// Device Info
|
||||
deviceSection
|
||||
|
||||
// About
|
||||
aboutSection
|
||||
}
|
||||
.navigationTitle("Settings")
|
||||
.navigationBarTitleDisplayMode(.large)
|
||||
.toolbar {
|
||||
ToolbarItem(placement: .navigationBarTrailing) {
|
||||
Button("Done") {
|
||||
dismiss()
|
||||
}
|
||||
}
|
||||
}
|
||||
.onAppear {
|
||||
viewModel.loadSettings()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - Model Section
|
||||
|
||||
private var modelSection: some View {
|
||||
Section("Models") {
|
||||
// Current model status
|
||||
HStack {
|
||||
Text("Status")
|
||||
Spacer()
|
||||
modelStatusView
|
||||
}
|
||||
|
||||
// Download progress
|
||||
if let variant = viewModel.downloadingVariant {
|
||||
VStack(alignment: .leading, spacing: 4) {
|
||||
Text("Downloading \(variant.uppercased())")
|
||||
.font(.caption)
|
||||
ProgressView(value: viewModel.downloadProgress, total: 100)
|
||||
Text("\(Int(viewModel.downloadProgress))%")
|
||||
.font(.caption2)
|
||||
.foregroundColor(.secondary)
|
||||
}
|
||||
}
|
||||
|
||||
// Gemma 4 E2B
|
||||
ModelVariantRow(
|
||||
name: "Gemma 4 E2B",
|
||||
description: "2B params, fastest, good for most tasks (~2.7GB)",
|
||||
isDownloaded: viewModel.isE2BDownloaded,
|
||||
isSelected: viewModel.selectedVariant == "e2b",
|
||||
isDownloading: viewModel.downloadingVariant == "e2b",
|
||||
onSelect: { viewModel.selectVariant("e2b") },
|
||||
onDownload: { viewModel.downloadModel("e2b") },
|
||||
onDelete: { viewModel.deleteModel("e2b") }
|
||||
)
|
||||
|
||||
// Gemma 4 E4B
|
||||
ModelVariantRow(
|
||||
name: "Gemma 4 E4B",
|
||||
description: "4B params, better quality, slower (~4.5GB)",
|
||||
isDownloaded: viewModel.isE4BDownloaded,
|
||||
isSelected: viewModel.selectedVariant == "e4b",
|
||||
isDownloading: viewModel.downloadingVariant == "e4b",
|
||||
onSelect: { viewModel.selectVariant("e4b") },
|
||||
onDownload: { viewModel.downloadModel("e4b") },
|
||||
onDelete: { viewModel.deleteModel("e4b") }
|
||||
)
|
||||
|
||||
// Custom model
|
||||
Button(action: { viewModel.showDocumentPicker = true }) {
|
||||
Label("Select from Files", systemImage: "folder")
|
||||
}
|
||||
.sheet(isPresented: $viewModel.showDocumentPicker) {
|
||||
DocumentPicker(selectedURL: $viewModel.customModelURL)
|
||||
}
|
||||
|
||||
// Load button
|
||||
if !viewModel.modelPath.isEmpty && !viewModel.modelLoaded && !viewModel.isLoadingModel {
|
||||
Button(action: { viewModel.loadModel() }) {
|
||||
Label("Load Selected Model", systemImage: "play.circle")
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@ViewBuilder
|
||||
private var modelStatusView: some View {
|
||||
if viewModel.isLoadingModel {
|
||||
HStack(spacing: 8) {
|
||||
ProgressView()
|
||||
.scaleEffect(0.8)
|
||||
Text("Loading...")
|
||||
}
|
||||
.foregroundColor(.secondary)
|
||||
} else if viewModel.modelLoaded {
|
||||
Label("Loaded", systemImage: "checkmark.circle.fill")
|
||||
.foregroundColor(.green)
|
||||
} else if let error = viewModel.modelLoadError {
|
||||
Label("Error", systemImage: "xmark.circle.fill")
|
||||
.foregroundColor(.red)
|
||||
.help(error)
|
||||
} else {
|
||||
Text("Not loaded")
|
||||
.foregroundColor(.secondary)
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - Server Section
|
||||
|
||||
private var serverSection: some View {
|
||||
Section("Servers") {
|
||||
HStack {
|
||||
TextField("Search Server (SearXNG)", text: $viewModel.searchServerURL)
|
||||
.textContentType(.URL)
|
||||
.keyboardType(.URL)
|
||||
.autocapitalization(.none)
|
||||
|
||||
if viewModel.isCheckingSearchHealth {
|
||||
ProgressView()
|
||||
.scaleEffect(0.8)
|
||||
} else if let healthy = viewModel.searchServerHealthy {
|
||||
Image(systemName: healthy ? "checkmark.circle.fill" : "xmark.circle.fill")
|
||||
.foregroundColor(healthy ? .green : .red)
|
||||
}
|
||||
}
|
||||
|
||||
HStack {
|
||||
TextField("Delegate Server (LLM)", text: $viewModel.delegateServerURL)
|
||||
.textContentType(.URL)
|
||||
.keyboardType(.URL)
|
||||
.autocapitalization(.none)
|
||||
|
||||
if viewModel.isCheckingDelegateHealth {
|
||||
ProgressView()
|
||||
.scaleEffect(0.8)
|
||||
} else if let healthy = viewModel.delegateServerHealthy {
|
||||
Image(systemName: healthy ? "checkmark.circle.fill" : "xmark.circle.fill")
|
||||
.foregroundColor(healthy ? .green : .red)
|
||||
}
|
||||
}
|
||||
|
||||
Text("Leave empty to disable server features. URLs are saved automatically.")
|
||||
.font(.caption)
|
||||
.foregroundColor(.secondary)
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - TTS Section
|
||||
|
||||
private var ttsSection: some View {
|
||||
Section("Text to Speech") {
|
||||
Toggle("Enable TTS", isOn: $viewModel.ttsEnabled)
|
||||
|
||||
Toggle("Auto-detect mode", isOn: $viewModel.ttsAutoMode)
|
||||
.disabled(!viewModel.ttsEnabled)
|
||||
|
||||
if viewModel.ttsEnabled {
|
||||
Text("Voice input → speaks response, Text input → silent")
|
||||
.font(.caption)
|
||||
.foregroundColor(.secondary)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - Device Section
|
||||
|
||||
private var deviceSection: some View {
|
||||
Section("Your Device") {
|
||||
HStack {
|
||||
Text("Total RAM")
|
||||
Spacer()
|
||||
Text(viewModel.totalRAM)
|
||||
.foregroundColor(.secondary)
|
||||
}
|
||||
|
||||
HStack {
|
||||
Text("Available RAM")
|
||||
Spacer()
|
||||
Text(viewModel.availableRAM)
|
||||
.foregroundColor(.secondary)
|
||||
}
|
||||
|
||||
HStack {
|
||||
Text("Device")
|
||||
Spacer()
|
||||
Text(viewModel.deviceModel)
|
||||
.foregroundColor(.secondary)
|
||||
}
|
||||
|
||||
HStack {
|
||||
Text("iOS Version")
|
||||
Spacer()
|
||||
Text(viewModel.iOSVersion)
|
||||
.foregroundColor(.secondary)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - About Section
|
||||
|
||||
private var aboutSection: some View {
|
||||
Section {
|
||||
HStack {
|
||||
Spacer()
|
||||
VStack(spacing: 4) {
|
||||
Text("Sleepy Agent")
|
||||
.font(.headline)
|
||||
Text("Local LLM inference with Gemma 4")
|
||||
.font(.caption)
|
||||
.foregroundColor(.secondary)
|
||||
}
|
||||
Spacer()
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - Model Variant Row
|
||||
|
||||
struct ModelVariantRow: View {
|
||||
let name: String
|
||||
let description: String
|
||||
let isDownloaded: Bool
|
||||
let isSelected: Bool
|
||||
let isDownloading: Bool
|
||||
let onSelect: () -> Void
|
||||
let onDownload: () -> Void
|
||||
let onDelete: () -> Void
|
||||
|
||||
var body: some View {
|
||||
VStack(alignment: .leading, spacing: 8) {
|
||||
HStack {
|
||||
VStack(alignment: .leading, spacing: 2) {
|
||||
Text(name)
|
||||
.font(.body)
|
||||
Text(description)
|
||||
.font(.caption)
|
||||
.foregroundColor(.secondary)
|
||||
}
|
||||
|
||||
Spacer()
|
||||
|
||||
if isDownloaded {
|
||||
Image(systemName: "checkmark.circle.fill")
|
||||
.foregroundColor(.green)
|
||||
}
|
||||
}
|
||||
|
||||
HStack(spacing: 8) {
|
||||
Button(action: onSelect) {
|
||||
Text(isSelected ? "Selected" : "Select")
|
||||
}
|
||||
.buttonStyle(.bordered)
|
||||
.disabled(!isDownloaded || isSelected)
|
||||
|
||||
Spacer()
|
||||
|
||||
if isDownloaded {
|
||||
Button(action: onDelete) {
|
||||
Label("Delete", systemImage: "trash")
|
||||
}
|
||||
.buttonStyle(.borderless)
|
||||
.tint(.red)
|
||||
} else {
|
||||
Button(action: onDownload) {
|
||||
if isDownloading {
|
||||
ProgressView()
|
||||
.scaleEffect(0.8)
|
||||
} else {
|
||||
Label("Download", systemImage: "arrow.down.circle")
|
||||
}
|
||||
}
|
||||
.buttonStyle(.borderedProminent)
|
||||
.disabled(isDownloading)
|
||||
}
|
||||
}
|
||||
}
|
||||
.padding(.vertical, 4)
|
||||
.background(isSelected ? Color.accentColor.opacity(0.1) : Color.clear)
|
||||
.cornerRadius(8)
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - Document Picker
|
||||
|
||||
struct DocumentPicker: UIViewControllerRepresentable {
|
||||
@Binding var selectedURL: URL?
|
||||
@Environment(\.presentationMode) var presentationMode
|
||||
|
||||
func makeUIViewController(context: Context) -> UIDocumentPickerViewController {
|
||||
let picker = UIDocumentPickerViewController(forOpeningContentTypes: [.data, .item])
|
||||
picker.delegate = context.coordinator
|
||||
picker.allowsMultipleSelection = false
|
||||
return picker
|
||||
}
|
||||
|
||||
func updateUIViewController(_ uiViewController: UIDocumentPickerViewController, context: Context) {}
|
||||
|
||||
func makeCoordinator() -> Coordinator {
|
||||
Coordinator(self)
|
||||
}
|
||||
|
||||
class Coordinator: NSObject, UIDocumentPickerDelegate {
|
||||
let parent: DocumentPicker
|
||||
|
||||
init(_ parent: DocumentPicker) {
|
||||
self.parent = parent
|
||||
}
|
||||
|
||||
func documentPicker(_ controller: UIDocumentPickerViewController, didPickDocumentsAt urls: [URL]) {
|
||||
parent.selectedURL = urls.first
|
||||
parent.presentationMode.wrappedValue.dismiss()
|
||||
}
|
||||
|
||||
func documentPickerWasCancelled(_ controller: UIDocumentPickerViewController) {
|
||||
parent.presentationMode.wrappedValue.dismiss()
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// MARK: - Preview
|
||||
|
||||
struct SettingsView_Previews: PreviewProvider {
|
||||
static var previews: some View {
|
||||
SettingsView()
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,110 @@
|
||||
#!/bin/bash
|
||||
|
||||
# Create Xcode Project Script for Sleepy Agent iOS
|
||||
# This script creates a basic Xcode project structure
|
||||
|
||||
set -e
|
||||
|
||||
APP_NAME="SleepyAgent"
|
||||
BUNDLE_ID="com.sleepy.agent"
|
||||
TEAM_ID="" # Leave empty for manual signing
|
||||
|
||||
# Colors for output
|
||||
RED='\033[0;31m'
|
||||
GREEN='\033[0;32m'
|
||||
YELLOW='\033[1;33m'
|
||||
NC='\033[0m' # No Color
|
||||
|
||||
echo -e "${GREEN}Sleepy Agent iOS - Project Generator${NC}"
|
||||
echo ""
|
||||
|
||||
# Check for Xcode
|
||||
if ! command -v xcodebuild &> /dev/null; then
|
||||
echo -e "${RED}ERROR: Xcode is not installed or not in PATH${NC}"
|
||||
echo "Please install Xcode from the App Store"
|
||||
exit 1
|
||||
fi
|
||||
|
||||
XCODE_VERSION=$(xcodebuild -version | head -1)
|
||||
echo "Found: $XCODE_VERSION"
|
||||
|
||||
# Create project directory structure
|
||||
echo ""
|
||||
echo "Creating project structure..."
|
||||
mkdir -p "$APP_NAME/Assets.xcassets/AppIcon.appiconset"
|
||||
mkdir -p "$APP_NAME/Preview Content"
|
||||
|
||||
# Create AppIcon contents
|
||||
cat > "$APP_NAME/Assets.xcassets/AppIcon.appiconset/Contents.json" << 'EOF'
|
||||
{
|
||||
"images" : [
|
||||
{
|
||||
"idiom" : "universal",
|
||||
"platform" : "ios",
|
||||
"size" : "1024x1024"
|
||||
}
|
||||
],
|
||||
"info" : {
|
||||
"author" : "xcode",
|
||||
"version" : 1
|
||||
}
|
||||
}
|
||||
EOF
|
||||
|
||||
# Create Assets catalog
|
||||
cat > "$APP_NAME/Assets.xcassets/Contents.json" << 'EOF'
|
||||
{
|
||||
"info" : {
|
||||
"author" : "xcode",
|
||||
"version" : 1
|
||||
}
|
||||
}
|
||||
EOF
|
||||
|
||||
# Create Preview Assets
|
||||
cat > "$APP_NAME/Preview Content/Preview Assets.xcassets/Contents.json" << 'EOF'
|
||||
{
|
||||
"info" : {
|
||||
"author" : "xcode",
|
||||
"version" : 1
|
||||
}
|
||||
}
|
||||
EOF
|
||||
|
||||
echo -e "${GREEN}Project structure created!${NC}"
|
||||
echo ""
|
||||
|
||||
# Check if we should use CocoaPods
|
||||
if command -v pod &> /dev/null; then
|
||||
echo "CocoaPods detected. Setting up Pods..."
|
||||
|
||||
if [ -f "Podfile" ]; then
|
||||
echo "Installing dependencies..."
|
||||
pod install --repo-update || pod install
|
||||
|
||||
echo ""
|
||||
echo -e "${GREEN}Setup complete!${NC}"
|
||||
echo ""
|
||||
echo "Open the project with:"
|
||||
echo " open ${APP_NAME}.xcworkspace"
|
||||
else
|
||||
echo -e "${YELLOW}WARNING: No Podfile found${NC}"
|
||||
fi
|
||||
else
|
||||
echo -e "${YELLOW}WARNING: CocoaPods not installed${NC}"
|
||||
echo "Install with: sudo gem install cocoapods"
|
||||
echo ""
|
||||
echo "Without CocoaPods, you'll need to manually add TensorFlow Lite."
|
||||
fi
|
||||
|
||||
echo ""
|
||||
echo "Next steps:"
|
||||
echo "1. Open ${APP_NAME}.xcworkspace (or ${APP_NAME}.xcodeproj) in Xcode"
|
||||
echo "2. Set your Team in Signing & Capabilities"
|
||||
echo "3. Update the Bundle Identifier if needed"
|
||||
echo "4. Build and run!"
|
||||
echo ""
|
||||
echo "To build from command line:"
|
||||
echo " make build # Debug build for simulator"
|
||||
echo " make archive # Create archive for distribution"
|
||||
echo " make ipa # Create .ipa file"
|
||||
@@ -0,0 +1,23 @@
|
||||
<?xml version="1.0" encoding="UTF-8"?>
|
||||
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
|
||||
<plist version="1.0">
|
||||
<dict>
|
||||
<key>method</key>
|
||||
<string>development</string>
|
||||
<key>teamID</key>
|
||||
<string>YOUR_TEAM_ID_HERE</string>
|
||||
<key>provisioningProfiles</key>
|
||||
<dict>
|
||||
<key>com.sleepy.agent</key>
|
||||
<string>SleepyAgent Development</string>
|
||||
</dict>
|
||||
<key>signingCertificate</key>
|
||||
<string>Apple Development</string>
|
||||
<key>signingStyle</key>
|
||||
<string>automatic</string>
|
||||
<key>stripSwiftSymbols</key>
|
||||
<true/>
|
||||
<key>thinning</key>
|
||||
<string><none></string>
|
||||
</dict>
|
||||
</plist>
|
||||
Reference in New Issue
Block a user