# Design Decision: Fix Tool Execution and Token Reporting

**Date:** 2024-02-24
**Scope:** src/api/routes.py tool_instructions and token counting

## Problem Statement

User report shows three critical failures:

1. **Instruction vs Execution:** Model says "You should run mkdir..." instead of TOOL: format
2. **Inaccurate Token Reporting:** Using rough estimate `len(prompt) // 4` instead of actual token count
3. **Interactive Commands:** npx create-react-app prompts for confirmation, causing 300s timeout

## Evidence

```
🖥️  BASH: mkdir react-hello-world && cd react-hello-world && npx create-react-app .
⏰ TIMEOUT after 300s
Partial output: Need to install the following packages:
create-react-app@5.1.0
Ok to proceed? (y)
```

**Additional Context:**
- Directory created but empty (no files)
- Model posts instructions for user to follow instead of executing

## Root Cause Analysis

### 1. Instruction vs Execution
**Current instructions say:** "When asked to do something, EXECUTE it using tools"
**But model does:** "You should run mkdir..."
**Why:** Instructions aren't strong enough - need explicit anti-patterns

### 2. Token Counting
**Current:** `prompt_tokens = len(prompt) // 4` (rough approximation)
**Problem:** Inaccurate for opencode context management
**Solution:** Use tiktoken for accurate counting

### 3. Interactive Commands
**Current:** npx commands prompt for confirmation
**Problem:** Tool executor waits indefinitely, times out at 300s
**Solution:** Either:
- Add --yes flag automatically
- Forbid npx entirely, use manual file creation

## Options Considered

### Option 1: Strengthen Instructions Only
- Add more explicit "DO NOT" language
- Add complete React example
- Keep rough token estimation

**Pros:** Simple, focused fix
**Cons:** Doesn't fix token accuracy or interactive command issue
**Verdict:** REJECTED - Incomplete fix

### Option 2: Comprehensive Fix
- Strengthen instructions with anti-patterns
- Use tiktoken for accurate token counting
- Add non-interactive flags to package manager commands
- Update examples to show manual file creation

**Pros:** Fixes all three issues
**Cons:** More complex changes
**Verdict:** ACCEPTED - Complete solution

### Option 3: Change Architecture
- Move to client-side tool execution
- Different token counting approach

**Pros:** Could solve multiple issues
**Cons:** Breaking change, out of scope
**Verdict:** REJECTED - Too broad

## Decision

Implement Option 2: Comprehensive fix addressing all three issues.

### Changes

#### 1. Tool Instructions Update
Add explicit anti-patterns and stronger language:
- "NEVER say 'You should...' - EXECUTE immediately"
- "DO NOT USE npx create-react-app - manually create files"
- Complete React example showing manual file creation

#### 2. Token Counting Fix
Replace rough estimate with tiktoken:
```python
# Before
prompt_tokens = len(prompt) // 4

# After  
import tiktoken
encoding = tiktoken.get_encoding('cl100k_base')
prompt_tokens = len(encoding.encode(prompt))
completion_tokens = len(encoding.encode(content))
```

#### 3. Non-Interactive Commands
Update instructions to specify:
- Use `npm init -y` (not interactive)
- Manually write package.json instead of npx
- All examples show manual file creation

## Impact

### Token Budget (Exact Count - cl100k_base)
- **New Instructions:** 586 tokens (2,067 characters)
- **Status:** Within 2000 token limit ✓
- **Context window:** 16K model leaves ~15.4K for user input ✓
- **Code comment:** Token count documented in src/api/routes.py ✓

### Breaking Changes
- **None** - Instructions clearer, format unchanged
- Token reporting more accurate (good thing)

### Code Changes
- `src/api/routes.py`:
  - Update tool_instructions (~+15 lines)
  - Add tiktoken import
  - Replace token estimation logic (~5 lines)

## Testing Strategy

1. **Token Accuracy Test:**
   ```python
   def test_token_accuracy():
       prompt = "Hello world"
       content = "Hi there"
       # Calculate with tiktoken
       # Verify API returns same values
   ```

2. **Instruction Content Test:**
   - Verify "DO NOT USE npx" present
   - Verify manual creation examples present
   - Verify "EXECUTE not DESCRIBE" present

3. **Integration Test:**
   - Request: "Create React app"
   - Expect: Manual file creation via write tool
   - Not expect: npx create-react-app

## Rollback Plan

If issues arise:
1. Revert to previous instructions
2. Keep tiktoken for token counting (beneficial)
3. Document why manual creation didn't work

## Success Metrics

- [ ] Model uses TOOL: format 100% of time (not descriptions)
- [ ] Token counts accurate within ±2%
- [ ] React projects created via write tool (not npx)
- [ ] No timeouts on package manager commands

## Implementation Notes

### Token Counting
Need to ensure tiktoken is in requirements.txt

### Tool Instructions
The key addition is:
```
**FORBIDDEN PATTERNS:**
- "You should run mkdir myapp" → USE: TOOL: bash\nARGUMENTS: {"command": "mkdir myapp"}
- "npx create-react-app myapp" → USE: Manual file creation with write tool
- "First create package.json, then..." → USE: Execute immediately, don't list steps

**REACT PROJECT - CORRECT APPROACH:**
1. TOOL: bash, ARGUMENTS: {"command": "mkdir myapp"}
2. TOOL: write, ARGUMENTS: {"filePath": "myapp/package.json", "content": "{\"name\": \"myapp\"...}"}
3. TOOL: write, ARGUMENTS: {"filePath": "myapp/src/index.js", "content": "..."}
4. Continue until all files created
```