MEDIUM: BatchQuantizedKVCache missing size() and empty() methods #44

New issue

Closed

opened 2026-05-09 18:01:49 +02:00 by sleepy · 0 comments

sleepy commented

2026-05-09 18:01:49 +02:00

Owner

Bug

BatchQuantizedKVCache does not implement size() and empty() methods that mlx_lm BatchGenerator may call.

Impact

AttributeError during batch operations if mlx_lm calls these methods.

Fix

Add size() and empty() methods.

File

omlx/cache/batch_quantized_cache.py

## Bug BatchQuantizedKVCache does not implement size() and empty() methods that mlx_lm BatchGenerator may call. ## Impact AttributeError during batch operations if mlx_lm calls these methods. ## Fix Add size() and empty() methods. ## File omlx/cache/batch_quantized_cache.py

sleepy referenced this issue from a commit

2026-05-09 18:26:56 +02:00

fix(cache): add size() and empty() to BatchQuantizedKVCache (#44)

No labels

bug

feature

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

sleepy/omlx#44

No description provided.

Rows
Columns