78 lines
2.6 KiB
Markdown
78 lines
2.6 KiB
Markdown
# AGENTS.md
|
|
|
|
**Last Updated:** April 9, 2026
|
|
|
|
## Research Project: Coding Agent Harness Analysis
|
|
|
|
### Objective
|
|
Collect data and feedback on four coding agent harnesses to determine what works best for different model sizes, particularly smaller/local models.
|
|
|
|
### Harnesses Under Analysis
|
|
1. **opencode** - Go-based coding agent
|
|
2. **pi** (pi-mono) - Mario Zechner's minimal terminal coding agent
|
|
3. **hermes** - Nous Research's agent that grows with you
|
|
4. **forgecode** - AI pair programmer with sub-agents
|
|
|
|
### Data Collection Strategy
|
|
|
|
#### Performance Benchmarks
|
|
- Run terminal-bench and similar benchmarks across all harnesses
|
|
- Track relative performance metrics
|
|
- Document success rates, speed, and quality of outputs
|
|
|
|
#### Community Feedback Collection
|
|
Feedback organized by harness and model tier:
|
|
- `opencode/feedback/localllm/` - Community feedback for local models
|
|
- `opencode/feedback/frontier/` - Community feedback for frontier models
|
|
- `pi/feedback/localllm/` - Community feedback for local models
|
|
- `pi/feedback/frontier/` - Community feedback for frontier models
|
|
- `hermes/feedback/localllm/` - Community feedback for local models
|
|
- `hermes/feedback/frontier/` - Community feedback for frontier models
|
|
- `forgecode/feedback/localllm/` - Community feedback for local models
|
|
- `forgecode/feedback/frontier/` - Community feedback for frontier models
|
|
|
|
### Folder Structure
|
|
```
|
|
opencode/
|
|
repo/ - opencode-ai/opencode source
|
|
feedback/
|
|
localllm/ - Local model feedback
|
|
frontier/ - Frontier model feedback
|
|
|
|
pi/
|
|
repo/ - badlogic/pi-mono source
|
|
feedback/
|
|
localllm/ - Local model feedback
|
|
frontier/ - Frontier model feedback
|
|
|
|
hermes/
|
|
repo/ - NousResearch/hermes-agent source
|
|
feedback/
|
|
localllm/ - Local model feedback
|
|
frontier/ - Frontier model feedback
|
|
|
|
forgecode/
|
|
repo/ - antinomyhq/forgecode source
|
|
feedback/
|
|
localllm/ - Local model feedback
|
|
frontier/ - Frontier model feedback
|
|
```
|
|
|
|
### Research Focus Areas
|
|
- Tool handling and capabilities
|
|
- Skills system effectiveness
|
|
- Prompt engineering strategies
|
|
- Context management
|
|
- Error recovery and resilience
|
|
|
|
### Future Work
|
|
Eventually extract best practices and implement improvements specifically optimized for smaller/local models.
|
|
|
|
### Reference Materials
|
|
Files from `../entropy/Research/md/` contain prompt research and strategies:
|
|
- `Research.md` - General research methodology
|
|
- `Research-prompt.md` - Prompt engineering research and strategies
|
|
- `Research-orchestration.md` - Orchestration patterns and strategies
|
|
|
|
These files contain prompt research and strategies to reference during analysis.
|