HIGH: prefix_cache.py uses meta_state before initialization in block loop #43

New issue

Closed

opened 2026-05-09 18:01:34 +02:00 by sleepy · 0 comments

sleepy commented

2026-05-09 18:01:34 +02:00

Owner

Bug

In reconstruct_cache(), the code tries to use meta_state to preserve quantization params BEFORE meta_state is assigned. This causes NameError for layer 0 and stale values for subsequent layers.

Impact

Silent cache misses (NameError caught → return None)
Wrong group_size/bits passed to concatenate_states()

Fix

Move meta_state assignment before the block collection loop, or use first_block_meta_states directly.

File

omlx/cache/prefix_cache.py:1575-1591

## Bug In `reconstruct_cache()`, the code tries to use `meta_state` to preserve quantization params BEFORE `meta_state` is assigned. This causes NameError for layer 0 and stale values for subsequent layers. ## Impact - Silent cache misses (NameError caught → return None) - Wrong group_size/bits passed to concatenate_states() ## Fix Move meta_state assignment before the block collection loop, or use first_block_meta_states directly. ## File omlx/cache/prefix_cache.py:1575-1591

sleepy referenced this issue from a commit

2026-05-09 18:23:42 +02:00

fix(cache): assign meta_state before block loop in reconstruct_cache (#43)

No labels

bug

feature

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

sleepy/omlx#43

No description provided.

Rows
Columns