Issues - sleepy/ternary - Sleepy Git (Forgejo)

sleepy/ternary

Search syntax

`assignee:<username>`	Filter by assigned user
`author:<username>`	Filter by the author
`is:all`	All
`is:closed`	Closed
`is:open`	Open
`mentions:<username>`	Filter by mentioned user
`modified:[>\|<]<date>`	Filter by last modified date
`review:<username>`	Filter by user who reviewed
`sort:<by>:[asc\|desc]`	Sort by: `created\|comments\|updated\|deadline`

New issue

_null_context in train.py is fragile and confusing

#18 opened 2026-05-09 19:22:08 +02:00 by sleepy

train_300m.py smoke test has input_ids/labels length mismatch

#17 opened 2026-05-09 19:22:08 +02:00 by sleepy

EngramModule has double residual — one in Engram, one in TransformerBlock

#16 opened 2026-05-09 19:22:08 +02:00 by sleepy

EngramConv.__init__ has hardcoded dilation=3, not passed from EngramModule

#15 opened 2026-05-09 19:22:08 +02:00 by sleepy

mHCConnector.alpha is unused in its own forward() — applied externally in model.py

#14 opened 2026-05-09 07:30:22 +02:00 by sleepy

PackedDataset.__iter__ silently drops items when buffer < batch_size * 4

#13 opened 2026-05-09 07:30:22 +02:00 by sleepy

generate() doesn't use KV cache — re-runs full forward pass every token

#12 opened 2026-05-09 07:30:22 +02:00 by sleepy

sweep.py has duplicate STAGE2 and STAGE3 definitions — second silently overwrites first

#11 opened 2026-05-09 07:30:22 +02:00 by sleepy

NGramHasher.lookup leaves early positions as index 0 — wrong embedding injected

#10 opened 2026-05-08 23:49:37 +02:00 by sleepy

compress_token_ids uses Python hash() — non-deterministic across runs

#9 opened 2026-05-08 23:49:37 +02:00 by sleepy

NGramHasher.lookup uses Python loops over B×T — massive bottleneck

#8 opened 2026-05-08 23:46:06 +02:00 by sleepy

is_full_attention flag is a no-op in TransformerBlock

#7 opened 2026-05-08 23:45:31 +02:00 by sleepy

Fix 6 training bugs causing impossible loss values (~22.3 vs max ~10.4)

#1 opened 2026-05-01 14:01:26 +02:00 by sleepy 5 / 7

Retrain Stage 1 sweep with all bug fixes applied

#2 opened 2026-05-01 14:01:26 +02:00 by sleepy 0 / 2

Run smoke test to validate all fixes before full training

#3 opened 2026-05-01 14:01:26 +02:00 by sleepy 0 / 1

Add offline mode for HuggingFace tokenizer and dataset loading

#4 opened 2026-05-01 14:01:26 +02:00 by sleepy 2 / 3

Fix sweep.py JSON serialization crash (StageConfig not serializable)

#5 opened 2026-05-01 14:01:26 +02:00 by sleepy 0 / 3