Theory Digest — Mar 31, 2026

1. Motivation: Random matrix theory (RMT) and topological data analysis (TDA) are powerful frameworks for extracting universal structure from complex data. While eigenvalue distributions of large random matrices exhibit universality regardless of entry distributions, connections between RMT and TDA through persistent homology remain largely unexplored.

2. Mathematical setup: Let $M$ be a real symmetric $n \times n$ matrix with distinct eigenvalues $\lambda_1 < \lambda_2 < \cdots < \lambda_n$ and orthonormal eigenvectors $e_1, \ldots, e_n$. Consider the quadratic form

   \[f(x) = x^T M x = \sum_{i=1}^n \lambda_i x_i^2\]

   where $x_i = x \cdot e_i$, restricted to the unit sphere $S^{n-1} = {x \in \mathbb{R}^n :

   x

   = 1}$.

   Assumptions:

3. $M$ is real symmetric with distinct eigenvalues (holds almost surely for GOE, GUE, Wishart)

4. For universality results: matrices drawn from classical ensembles (GOE, GUE, Wishart)

   The sublevel set filtration is

   \[f^{-1}(-\infty, c] = \{x \in S^{n-1} : x^T M x \leq c\}\]

5. Toy example: When $n=3$ with eigenvalues $\lambda_1 = -1, \lambda_2 = 0, \lambda_3 = 2$, the function $f(x) = -x_1^2 + 2x_3^2$ on $S^2$ has critical points at $\pm e_1, \pm e_2, \pm e_3$ with critical values $-1, 0, 2$. The sublevel sets transition $\emptyset \to S^0 \to S^1$ as $c$ increases through the eigenvalues, creating persistence bars of lengths $s_1 = 1$ and $s_2 = 2$.

6. Formal objective: Determine the persistence diagram structure of the sublevel set filtration and prove

   \[\text{PE}_{\text{GOE}} = \log\left(\frac{8n}{\pi}\right) - 1\]

The method proceeds through Morse theory applied to quadratic forms on spheres:

1. Critical point analysis: The critical points of $f

   _{S^{n-1}}$ are determined by the Lagrange condition $\nabla f = 2Mx = 2\lambda x$, giving exactly $2n$ critical points $\pm e_i$ with critical values $\lambda_i$.

2. Morse index computation: The Hessian of $f

   _{S^{n-1}}$ at critical point $e_i$ restricted to the tangent space has eigenvalues $2(\lambda_j - \lambda_i)$ for $j \neq i$, yielding Morse index $i-1$.

3. Sublevel set topology: Between consecutive critical values $\lambda_k < c < \lambda_{k+1}$, the sublevel set

   \[f^{-1}(-\infty, c] \simeq S^{k-1}\]

   by deformation retraction onto ${(x_1, \ldots, x_k, 0, \ldots, 0) : \sum_{i=1}^k x_i^2 = 1}$.

4. Persistence diagram construction: As $c$ increases through $\lambda_k$, the topology changes from $S^{k-2}$ to $S^{k-1}$, creating one finite bar $[\lambda_k, \lambda_{k+1})$ in dimension $H_{k-1}$ with length $s_k = \lambda_{k+1} - \lambda_k$.

5. Statistics computation: Define persistence entropy as

   \[\text{PE} = -\sum_{k=1}^{n-1} \frac{s_k}{\text{TP}} \log\frac{s_k}{\text{TP}}\]

   where $\text{TP} = \sum_{k=1}^{n-1} s_k = \lambda_n - \lambda_1$.

   Toy example application: For $n=3$ with eigenvalues $(-1, 0, 2)$, the persistence diagram has bars $[{-1}, 0)$ in $H_0$ and $[0, 2)$ in $H_1$ with lengths $(1, 2)$. The persistence entropy is $\text{PE} = -\frac{1}{3}\log\frac{1}{3} - \frac{2}{3}\log\frac{2}{3} \approx 0.637$.

Step 1 — Prior work:

1. Bobrowski-Skraba (2024) proved universality for persistence diagrams of geometric filtrations over random point processes
2. Polterovich et al. (2020) studied topological persistence of Laplacian eigenfunctions on surfaces

3. Standard RMT universality (Wigner 1955, Tao-Vu 2011) for eigenvalue distributions

   Step 2 — Delta: This paper establishes the exact connection between random matrix eigenvalues and persistence diagrams via Morse theory of quadratic forms. The key contributions are:

4. proving bar lengths equal eigenvalue spacings exactly
5. deriving closed-form persistence entropy for GOE

6. demonstrating PE as a complementary spectral diagnostic.

   Step 3 — Theory-specific assessment:

   • The main theorem (Morse structure) is somewhat predictable given that quadratic forms on spheres are well-studied, but the precise persistence diagram identification appears new
   • The proof technique is largely routine application of standard Morse theory, though the connection to RMT universality is novel
   • No lower bounds are established; the persistence entropy bound is an exact asymptotic formula rather than a concentration inequality

   The practical applications (GOE vs GUE discrimination, Rosenzweig-Porter detection) show genuine utility but are incremental improvements over existing spectral diagnostics.

   Verdict: INCREMENTAL — solid theoretical connection between established frameworks with modest practical improvements, but the core Morse theory application is relatively straightforward.

The proof strategy combines classical Morse theory with random matrix asymptotics:

1. Morse function verification: Show $f(x) = x^T M x$ on $S^{n-1}$ has exactly $2n$ non-degenerate critical points $\pm e_i$ with critical values $\lambda_i$ and Morse indices.

2. Sublevel set homotopy analysis: For $\lambda_k < c < \lambda_{k+1}$, use the key inequality

   \[\sum_{j>k} \lambda_j x_j^2 \geq \lambda_{k+1} \sum_{j>k} x_j^2\]

   to show the sublevel set deformation retracts to $S^{k-1}$ via the map $(x_1, \ldots, x_k, x_{k+1}, \ldots, x_n) \mapsto (x_1, \ldots, x_k, 0, \ldots, 0)$.

3. Persistence bar identification: The homology transitions $H_{k-2}(S^{k-2}) \to H_{k-1}(S^{k-1})$ create exactly one bar per eigenvalue gap with length

   \[s_k = \lambda_{k+1} - \lambda_k\]

4. Asymptotic persistence entropy: For large $n$ with eigenvalue density $\rho(\lambda)$, use the approximation $s_k \approx 1/(n\rho(\gamma_k))$ where $\gamma_k$ is the $k/(n+1)$-quantile, leading to

   \[\text{PE} \approx \log(n \cdot \text{TP}) + \frac{1}{\text{TP}} \int_{\lambda_-}^{\lambda_+} \log \rho(\lambda) d\lambda\]

5. GOE calculation: For the Wigner semicircle density

   \[\rho_{\text{SC}}(\lambda) = \frac{\sqrt{4-\lambda^2}}{2\pi}, \quad \lambda \in [-2,2]\]

   evaluate the integral using $\lambda = 2\sin\theta$ substitution and the identity $\int_0^{\pi/2} \cos\theta \log\cos\theta \, d\theta = \log 2 - 1$ to obtain

   \[\text{PE}_{\text{GOE}} = \log\left(\frac{8n}{\pi}\right) - 1\]

Primary validation through numerical simulation:

1. GOE universality verification: 200 independent matrices at $n = 50, 100, 200$ show coefficient of variation $\text{CV}(\text{PE}) \sim n^{-0.65}$, confirming concentration.

2. Analytical formula verification: Persistence entropy matches $\log(8n/\pi) - 1$ to within 2.5% at $n = 200$ with monotonically decreasing bias $\sim n^{-0.17}$.

3. Ensemble discrimination: PE achieves AUC 0.978 vs 0.952 for $\langle r \rangle$ in GOE/GUE classification at $n = 100$ (non-overlapping 95% CIs).

4. Rosenzweig-Porter model: PE detects global spectral changes at SNR > 3 by $\lambda = 0.7$ while $\langle r \rangle$ remains at GOE values for all $\lambda \leq 5$.

5. Cross-ensemble verification: Unfolded spacing distributions match Wigner surmise $p_1(s), p_2(s)$ for GOE, GUE respectively via Kolmogorov-Smirnov tests.

   Computational infrastructure: Apple M4 Pro with NumPy/SciPy libraries.

Limitations:

1. TECHNICAL: Finite-size corrections decay slowly as $n^{-0.17}$ due to square-root singularity in semicircle density at spectral edges
2. NATURAL: Requires distinct eigenvalues (satisfied almost surely for classical ensembles)
3. RESTRICTIVE: Information-theoretic ceiling—persistence diagram is bijection of eigenvalue sequence, limiting discriminating power beyond existing spectral methods

4. TECHNICAL: No concentration inequalities established, only asymptotic exact formulas

   Open problems:

5. Derive rigorous concentration bounds for persistence entropy using eigenvalue rigidity theory (Erdős-Yau framework)
6. Extend to non-Hermitian random matrices where Morse theory becomes more complex due to complex eigenvalues

Motivation: Neural networks with more parameters than training examples often generalize well, contradicting classical statistical theory. Predicting when generalization occurs without held-out test data remains challenging, and detecting label corruption in training sets is crucial for real-world ML deployment.

Mathematical setup: Consider a neural network $f_\theta: \mathbb{R}^d \to \mathbb{R}^k$ with weight matrices $W_l \in \mathbb{R}^{m_l \times n_l}$. For each weight matrix, define the Gram matrix:

Let ${\lambda_i^{(l)}}_{i=1}^{n_l}$ be the eigenvalues of $S_l$. Define the tail index $\alpha_l$ via the Hill estimator on the top 10% eigenvalues. For a dataset with label noise fraction $\eta \in [0,1]$, a fraction $\eta$ of labels are replaced uniformly at random.

Assumptions:

1. Weight matrices are initialized with i.i.d. entries of variance $\sigma^2$
2. Networks have sufficient capacity to interpolate training data

3. Training converges to zero loss on corrupted labels

   Toy example: For MLP with architecture [784, 512, 256, 128, 10] on MNIST, the bottleneck layer has weight matrix $W \in \mathbb{R}^{512 \times 256}$. At $\eta = 0$ (clean), we observe $\alpha \approx 2.1$ (heavy tail). At $\eta = 1$ (fully random labels), we observe $\alpha \approx 3.5$ (lighter tail).

   Formal objective: Given trained weight matrices ${W_l}$, predict test accuracy:

   \[\text{test accuracy} = f(\alpha_{\text{bottleneck}}, \text{other spectral measures})\]

The method extracts spectral properties from trained weight matrices and uses them as diagnostics.

Core Algorithm:

1. Train network to convergence on potentially corrupted dataset
2. For each weight matrix $W_l$, compute Gram matrix $S_l = \frac{1}{m_l}W_l^\top W_l$
3. Extract eigenvalues ${\lambda_i^{(l)}}$ and compute tail index $\alpha_l$ via Hill estimator on top 10%
4. Identify bottleneck layer: highest compression ratio with $\min(\dim) \geq 50$

5. Use linear model: $\text{test accuracy} = a \cdot \alpha_{\text{bottleneck}} + b$

   Additional spectral observables:

   • Effective rank: $\exp(H(p))/n$ where $p_i = \lambda_i/\sum_j \lambda_j$
   • Outlier fraction: eigenvalues above Marchenko-Pastur upper edge $\lambda_+ = \sigma^2(1 + \sqrt{\gamma})^2$

   Toy example application: For the MLP example, the bottleneck layer is net.2 (512→256). At clean labels, $\alpha = 2.1$ predicts 98% test accuracy. At 50% label noise, $\alpha = 2.8$ predicts 49% test accuracy via the fitted linear relationship.

Prior work:

• Martin & Mahoney (2021): Established connection between heavy-tailed weight spectra and network quality; proposed tail index α as quality metric across published models
• Meng & Yao (2023): Studied spectral phases vs classification difficulty; proposed early stopping criteria
• Jiang et al. (2020): Comprehensive evaluation showing no single generalization measure dominates all perturbation types

Delta: This work provides:

1. controlled experiments with fine-grained noise gradient (21 levels) and proper LOO cross-validation achieving quantitative prediction (R² = 0.984)
2. identification of bottleneck layer as optimal measurement point vs averaging across layers
3. honest negative result under hyperparameter variation (R² < 0.25)

4. validation on real human annotation noise (CIFAR-10N).

   Theory-specific assessment:

   • Main result is not theoretically surprising: memorizing random labels requires spreading weights across more spectral directions than learning structured patterns
   • Proof techniques are standard (Hill estimator, BBP transition from random matrix theory)
   • No formal bounds proven; theoretical framework is heuristic
   • The linear relationship α ∝ η is plausible but not rigorously established

   Verdict: INCREMENTAL — Solid experimental validation of expected spectral behavior under label noise, with useful practical application to noise detection, but the core theoretical insight follows naturally from random matrix theory principles.

The paper uses empirical validation rather than formal proofs, but connects observations to established random matrix theory:

Key theoretical components:

1. BBP Transition: For rank-$r$ perturbation to random matrix, outlier eigenvalues appear above Marchenko-Pastur edge $\lambda_+ = \sigma^2(1 + \sqrt{\gamma})^2$ iff perturbation strength $\theta > \sigma^2\sqrt{\gamma}$

2. Outlier count argument: - Generalization: $O(k)$ outliers for $k$-class problem - Memorization: $O(\min(N,n))$ outliers for $N$ random examples through width-$n$ layer

3. Monotonicity conjecture:

   \[\alpha_\eta \text{ monotonically increasing in noise fraction } \eta\]

   **Informal proof:** At $\eta = 0$, network learns rank-$k$ boundary producing few outliers (heavy tail). At $\eta = 1$, must memorize $N$ independent random labels producing $O(N)$ outliers (light tail). Intermediate $\eta$ interpolates between these regimes.
   

4. Hill estimator: For eigenvalues $\lambda_{(1)} \geq \ldots \geq \lambda_{(n)}$, tail index:

   \[\hat{\alpha} = \left(\frac{1}{m}\sum_{i=1}^m \log \frac{\lambda_{(i)}}{\lambda_{(m+1)}}\right)^{-1}\]

   where $m$ is number of eigenvalues above chosen quantile.

   Primary validation: Leave-one-out cross-validation across 21 noise levels with linear regression.

Datasets: MNIST, CIFAR-10, CIFAR-10N (human re-annotations)

Architectures: MLP [784,512,256,128,10], CNN (4 conv + 2 FC), ResNet-18

Key experiments:

1. EXP-010 (Label noise gradient): 21 noise levels η ∈ [0,1], 3 seeds each (63 total runs)

2. EXP-011 (Hyperparameter variation): 180 configs varying width, depth, lr, weight decay (540 total runs)

   Key results:

   • Label noise: Tail α achieves LOO R² = 0.984 vs best baseline R² = 0.149
   • Hyperparameter variation: All measures weak (R² < 0.25), simple L₂ norm slightly better (0.219) than tail α (0.167)
   • Real-world validation: CIFAR-10N noise detection with 3% error at 9% aggregate noise, 9% error at 40% worst-annotator noise

   Baselines: Frobenius norms, spectral norms, global L₂ norm, effective rank, level spacing ratio

   Computational: PyTorch on Apple M1, all code available on GitHub

Limitations:

1. RESTRICTIVE: Limited to small-scale networks (MLP, small CNN, ResNet-18) - scaling to large language models and transformers unknown

2. TECHNICAL: Hill estimator requires ≥50 eigenvalues for stability - limits application to narrow layers like fine-tuned output layers

3. TECHNICAL: Hill estimator shows 25% variability across threshold quantiles (q ∈ [0.70, 0.95]) - relative ordering robust but absolute values uncertain

4. RESTRICTIVE: Fails under hyperparameter variation (R² < 0.25) - not a universal generalization predictor

5. TECHNICAL: Real noise detection underestimates at high corruption (9% error at 40% noise) due to class-dependent structure vs uniform calibration

6. NATURAL: No formal generalization bounds - theoretical framework is heuristic

   Open problems:

7. Scaling question: Do spectral signatures of data quality persist in modern large-scale architectures (transformers, foundation models)?

8. Theoretical gap: Prove formal bounds connecting spectral properties to generalization under label noise, particularly the conjectured monotonicity of α in noise fraction η

1. Motivation: Self-supervised pre-training has become a cornerstone of modern machine learning, but existing theoretical bounds leave open whether they accurately capture the complex interaction between pre-training and fine-tuning. Sharp asymptotic theory is needed to understand when pre-training provably outperforms training from scratch.

2. Mathematical setup: Let μpre and μdown be probability measures on input spaces Z and X. We have pre-training dataset D(m)pre = {zj}^m{j=1} where zj ~ μpre i.i.d., and downstream dataset D(n)down = {(xi,yi)}^n{i=1} where (xi,yi) ~ (X,Y) i.i.d. and X ~ μdown. The regression model is:

   \[Y = f⋆(X) + ε\] \[E[ε | X] = 0\] \[σ² := E[ε² | X] < ∞\]

   We parameterize representations via ψ(x,w) ∈ ℝᵖ where w ∈ ℝᵠ⁰. The hypothesis class is Hw = {fθ,w

   θ ∈ ℝᵖ} with fθ,w(x) := ⟨θ, ψ(x,w)⟩. We assume f⋆ is well-specified: f⋆ ∈ F := ∪w∈ℝᵠ⁰ Hw.

   Assumptions:

   1. Pre-training loss ℓpre(w;z) is twice continuously differentiable
   2. Group symmetry: ℓpre(g·w;z) = ℓpre(w;z) for compact Lie group G
   3. Orthogonal equivariance: ψ(x,g·w) = ρ(g)ψ(x,w) for homomorphism ρ: G → O(p)
   4. Population minimizer Ω⋆ is unique in descriptor space

3. Toy example: Consider linear representation ψ(x,M) = Mx with M ∈ ℝᵖˣᵈ under spectral loss. With Σpre = Id and Σ⁺_pre = diag(1,1/2,…,1/d), the population representation targets the first k coordinates. When d=2, k=1, this reduces to learning the first principal component.

4. Formal objective: Characterize the asymptotic distribution of the scaled excess test risk:

   \[Em,n := n[R(D(m)_pre, X1:n) - σ²] →^d Eα\]

   along limits (m,n) → (∞,∞) with m/n → α ∈ (0,∞).

The method uses two-stage M-estimation with Riemannian geometry to handle symmetries:

1. Pre-training stage: Solve ŵm ∈ argmin_w L̂pre(w; D(m)_pre) where L̂pre(w; D(m)_pre) = (1/m)∑ᵐⱼ₌₁ ℓpre(w; zj)

2. Descriptor mapping: Since ŵm is only identifiable up to group symmetry, work with descriptor Ω̂m = R(ŵm) ∈ M where R: ℝᵠ⁰ → ℝᵠ is constant along orbits and M is a Riemannian manifold

3. Downstream stage: Given Ω̂m, solve θ̂m,n ∈ argmin_θ L̂down(θ; Ω̂m, D(n)_down) where:

   \[L̂down(θ; Ω̂m, D(n)_down) := (1/n)∑ⁿᵢ₌₁[yi - ⟨θ, ϕ(xi, Ω̂m)⟩]²\]

4. Risk decomposition: The conditional test risk decomposes as:

   \[E[(Ynew - f̂m,n(Xnew))² | D(m)_pre, X1:n] = σ² + Rep(Ω̂m) + Leakagen(Ω̂m) + Varn(Ω̂m)\]

   Applied to toy example: With spectral loss and d=2, k=1, the method learns Ω̂m ≈ first eigenspace of empirical cross-covariance, then fits linear predictor ⟨θ̂, Ω̂m x⟩ on downstream data. The excess risk scales as σ²·1 + (1/α)·interaction term.

Prior work:

1. Cabannes et al. (2023): “The SSL interplay: Augmentations, inductive bias, and generalization” - studied VICReg-style pre-training with RKHS regression, bounded downstream risk by scaled pre-training loss
2. Ge et al. (2023): “On the provable advantage of unsupervised pretraining” - provided end-to-end bounds via MLE pre-training + ERM fine-tuning with κ-informative condition

3. Saunshi et al. (2019, 2022): Early contrastive learning theory relating pre-training loss to downstream performance

   Delta: This paper introduces Riemannian M-estimation to handle group symmetries in pre-training parameters, develops joint-sample asymptotics (m,n) → ∞, and provides instance-specific characterization via orthogonal equivariance condition.

   Theory-specific assessment:

   • Main theorem provides sharp asymptotic characterization, but is this surprising? The result follows standard M-estimation theory once symmetries are handled via Riemannian geometry - somewhat predictable extension.
   • Proof technique: The Riemannian CLT (Theorem 4.1) appears novel for this setting, requiring non-trivial adaptation of standard M-estimation tools to quotient spaces. The orthogonal equivariance condition is a clean structural insight.
   • Bounds vs. lower bounds: Paper shows improvements over Cabannes et al. (factor k improvement) and Ge et al. (polynomial improvement in problem constants), but no matching lower bounds established.

   The asymptotic analysis is mathematically sophisticated but the main insights (two-term decomposition, m/n scaling) are somewhat expected from classical theory.

   Verdict: INCREMENTAL — solid technical contribution handling symmetries in two-stage estimation, but the core insights follow predictably from extending standard M-estimation theory.

The proof uses Riemannian M-estimation combined with delta-method expansions:

1. Riemannian CLT for pre-training: Key insight is to work in tangent space coordinates. Define vm := logΩ⋆(Ω̂m) ∈ TΩ⋆M and show:

   \[√m vm →^d N(0, H⋆^{-1}Σ⋆H⋆^{-1})\]

   where H⋆ = HessLpre(Ω⋆) and Σ⋆ = E[gradℓpre(Ω⋆;Z)gradℓpre(Ω⋆;Z)ᵀ]. This requires uniform laws on the manifold M.

2. Delta-method for risk decomposition: The key inequality is the exact conditional risk expansion:

   \[E[(Ynew - f̂Ω,n(Xnew))² | D(m)_pre, X1:n] = σ² + Rep(Ω) + Leakagen(Ω) + Varn(Ω)\]

3. Joint-limit analysis: Under scaling n/m → 1/α, show: - nVarn(Ω̂m) →^P σ²deff(Ω⋆) - nLeakagen(Ω̂m) →^P 0 - mRep(Ω̂m) →^d ‖L(Z)‖²_{L²(μdown)} where L(v) = -DΠΩ⋆[v]f⋆

4. Fréchet differentiation: Critical step is showing ΠΩ is Fréchet differentiable as map Ω ↦ ΠΩ on L²(μdown). Uses operator-theoretic analysis of projection operators.

   The technical challenge is interchanging limits and expectations while controlling the random covariance matrices Σn(Ω̂m). The conditioning approach avoids heavy-tail issues from ill-conditioned empirical covariances.

Purely theoretical. The paper provides three case studies (spectral pre-training, factor models, Gaussian mixtures) with explicit asymptotic characterizations. For the Gaussian mixture example, Monte Carlo simulation is used to evaluate the limiting expression ‖L(Z)‖²_{L²(μdown)} since it lacks closed form.

Empirical validation would require:

1. finite-sample experiments showing convergence to asymptotic limits as (m,n) → ∞
2. verification of the phase transition point α > α₀ where pre-training outperforms fine-tuning only
3. comparison of predicted vs. actual improvement factors over prior bounds.

Limitations:

1. Downstream restricted to linear regression - RESTRICTIVE (significantly limits applicability to modern deep learning)

2. Conditional risk analysis avoiding fully averaged risk - TECHNICAL (needed for proof technique but less interpretable)

3. Requires orthogonal equivariance condition ψ(x,g·w) = ρ(g)ψ(x,w) - RESTRICTIVE (rules out many practical pre-training objectives)

4. Joint-sample asymptotics (m,n) → ∞ - NATURAL (standard in asymptotic theory)

5. Group action must be smooth and free on regular set - TECHNICAL (standard geometric requirement)

6. Well-specified downstream model f⋆ ∈ F - RESTRICTIVE (unrealistic in practice)

   Open problems:

7. Extend to non-linear downstream models (neural networks, kernel methods) while preserving orbit-invariance analysis
8. Develop non-asymptotic bounds matching the asymptotic rates derived here