sleepy
8e72eef09c
- Rename gamma to glm5 and model to minimax-m2.7 - Add model_comparison/ directory with head-to-head analyses - Sanitize all session.jsonl files: remove absolute paths and usernames - Remove __pycache__ artifacts - Add .gitignore
694 B
694 B
Implement a numerically stable backward pass for layer normalization from scratch in NumPy.
Constraints:
- Input: x of shape (B, T, D)
- Parameters: gamma, beta of shape (D,)
- Forward: y = gamma * (x - mean) / sqrt(var + eps) + beta
Requirements:
- Derive and implement gradients w.r.t. x, gamma, beta manually (no autodiff).
- Avoid redundant recomputation — reuse intermediates where possible.
- Ensure numerical stability (discuss where instability can occur).
- Provide a gradient check using finite differences.
- Analyze time and memory complexity.
- Explain how you would fuse this into a single kernel for GPU execution.
Do not use PyTorch, TensorFlow, JAX, or autograd.