sleepy
45c3aad453
- Add Claude Opus 4.7, Kimi K2.6, GLM-5.1 to existing GLM-5, Qwen3-6, MiniMax-M2.7 - Add 5 new challenges: flash attention fwd/bwd, beam search, DFlash, ternary training - Rewrite README with TL;DR rankings, grade matrix, and DeepSeek V4 Pro attribution - Add analysis/ folder with cross-model comparisons and per-challenge deep dives - Add deploy_challenges.sh script - Expand .gitignore to exclude Python envs, ML weights, and build artifacts
695 B
695 B
Implement a numerically stable backward pass for layer normalization from scratch in NumPy.
Constraints:
- Input: x of shape (B, T, D)
- Parameters: gamma, beta of shape (D,)
- Forward: y = gamma * (x - mean) / sqrt(var + eps) + beta
Requirements:
- Derive and implement gradients w.r.t. x, gamma, beta manually (no autodiff).
- Avoid redundant recomputation — reuse intermediates where possible.
- Ensure numerical stability (discuss where instability can occur).
- Provide a gradient check using finite differences.
- Analyze time and memory complexity.
- Explain how you would fuse this into a single kernel for GPU execution.
Do not use PyTorch, TensorFlow, JAX, or autograd.