bug: Tokenizer test hardcoded to Qwen3.5-4B path which may not exist #56

New issue

Open

opened 2026-05-20 23:58:14 +02:00 by sleepy · 0 comments

sleepy commented

2026-05-20 23:58:14 +02:00

Owner

src/models/qwen3_5/tokenizer.zig line 259 hardcodes the tokenizer path to /Users/sleepy/.sleepy-llm/models/Qwen3.5-4B/tokenizer.json. On machines without the 4B model cached, this causes test failures.

Reproduction

zig build test

Result: Tokenizer Qwen3.5 encode Hello test fails silently.

Expected

Tests should either:

Skip if model not found, OR
Use a mock/test tokenizer.json, OR
Parameterize the path via build option/env var

Fix

Add a comptime or runtime check for the model path, or bundle a minimal test tokenizer.

`src/models/qwen3_5/tokenizer.zig` line 259 hardcodes the tokenizer path to `/Users/sleepy/.sleepy-llm/models/Qwen3.5-4B/tokenizer.json`. On machines without the 4B model cached, this causes test failures. ## Reproduction ```bash zig build test ``` Result: `Tokenizer Qwen3.5 encode Hello` test fails silently. ## Expected Tests should either: - Skip if model not found, OR - Use a mock/test tokenizer.json, OR - Parameterize the path via build option/env var ## Fix Add a comptime or runtime check for the model path, or bundle a minimal test tokenizer.

sleepy added the

bug

label

2026-05-20 23:58:39 +02:00