Interactive lab · frozen neural-network checkpoint · companion to matrix cocycle

SGD iterates from a real θ⋆

Load a saved network at the edge of stability. Sweep step size and batch size, continue plain SGD from θ⋆, and view the occupancy of \(\delta_t=\theta_t-\theta_\star\) in the frozen full-batch top-2 Hessian plane \((u_1,u_2)\). Same camera as the synthetic matrix-cocycle lab, but the landscape is the true loss.

$$\delta_{t+1}=\delta_t-\eta\Big(\nabla L_{B_t}(\theta_\star+\delta_t)\Big),\qquad z_{i,t}=\langle\delta_t,u_i\rangle,\qquad \widehat\chi_{\mathrm{lin}}=\mathbb E\big[\log|1-\eta\,u_1^\top H_{B}\,u_1|\big],\qquad \widehat\chi_{\mathrm{mat}}=\lim\tfrac1T\sum_t\log\|(I-\eta H_{B_t})v_t\|$$
Precomputed packs (offline GPU). \(\widehat\chi_{\mathrm{lin}}\) = Rayleigh along \(u_1\); \(\widehat\chi_{\mathrm{mat}}\) = full matrix-product LE (all directions). Alignment = mean \(|\langle u_B,u_1\rangle|\) and sequential \(|\langle u_B(t),u_B(t+1)\rangle|\).

Checkpoint pack

Hard-refresh (Ctrl/Cmd+Shift+R) if chips look stale. Bimodal SGD pack chip: ★ SGD B=192 @25k (2nd chip) — not B=192 @145k. Or open ?pack=vit_20260808_0332_04_SGD_lr0.0035_b192_ckpt_step025000. Append ?all=1 for every mid-ckpt.
loading…

Step size

—
below 2 past 2 ηλ₁=2
Classical edge: \(\eta=2/\lambda_1\). Widget reports \(\eta\lambda_1\).

Batch size

—
Full batch ≈ lowest noise (clearest lobes). Smaller \(B\) adds gradient noise.

Presets

Jump to regimes: pre-edge, edge, post-edge bimodal, washed-out small batch.
—

Joint law p(z₁, z₂) — frozen top-2 Hessian frame, log-density

marginal p(z₁)
fixed u₁,u₂
not rotating
z₁ →
↑ z₂
p(z₂)
Default chips = Chen & Bruna B.3.2 paper packs (TF2 / MLP3–4 ReLU) under eos_bimodality_nn_outputs/paper_b32_sgd/. CIFAR ViT overnight packs via packs/manifest.json (?all=1). Synthetic twin: matrix cocycle.