Theory Digest — Apr 14, 2026

1. Motivation: Modern deep neural networks are frequently overparameterized, meaning the number of parameters $p$ greatly exceeds the number of training samples $n$, yet they generalize remarkably well despite classical statistical learning theory predicting severe overfitting. This contradiction has become a central theoretical question in machine learning.

2. Mathematical setup: Consider a neural network $f(x; \theta)$ with parameters $\theta \in \mathbb{R}^p$ trained on dataset ${(x_i, y_i)}_{i=1}^n$. The empirical risk is:

   \[\mathcal{R}_n(\theta) = \frac{1}{n}\sum_{i=1}^n \ell(f(x_i; \theta), y_i)\]

   Classical theory predicts generalization gap bounded by VC dimension:

   \[|\mathcal{R}(\theta) - \mathcal{R}_n(\theta)| \leq O\left(\sqrt{\frac{d + \log(1/\delta)}{n}}\right)\]

   where $d$ is the VC dimension. The assumptions are:

   1. $p \gg n$ (overparameterization)
   2. Models achieve zero training error: $\mathcal{R}_n(\theta) = 0$
   3. SGD optimization: $\theta_{t+1} = \theta_t - \eta \nabla_\theta \mathcal{R}_B(\theta_t)$ for mini-batch $B$

3. Toy example: When $d = 2$ and we have a 2-layer MLP with width $w = 1000$ on MNIST subset with $n = 100$ samples, we have $p \approx 800,000$ parameters. Classical theory predicts massive overfitting, yet empirically the network generalizes with test accuracy $> 90\%$.

4. Formal objective: Explain why the generalization gap remains small:

   \[\mathbb{E}[\mathcal{R}(\theta) - \mathcal{R}_n(\theta)] = o(1)\]

   despite $p \gg n$.

The paper evaluates five explanatory mechanisms through controlled experiments:

1. Implicit Regularization via SGD: Train identical architectures with different batch sizes $b \in {32, 64, 128, 256, 512, 1024, 2048}$, measuring test accuracy. Learning rates scaled as $\eta = \eta_0 \cdot b/128$.

2. Loss Landscape Geometry: Measure flatness via: - Hessian top eigenvalue: $\lambda_{\max} = \max \text{eig}(\nabla^2 \mathcal{R}_n(\theta))$
   - Perturbation sensitivity: $\Delta\mathcal{R} = \mathcal{R}_n(\theta + \epsilon) - \mathcal{R}_n(\theta)$ where $\epsilon \sim \mathcal{N}(0, \sigma^2 I)$

3. Double Descent: Train models with parameter counts from $10^3$ to $10^7$, plotting test error vs. model size.

4. NTK Regime: Measure relative parameter movement:

   \[\Delta\theta_{\text{rel}} = \frac{\|\theta_T - \theta_0\|_2}{\|\theta_0\|_2}\]

5. Lottery Ticket Hypothesis: Iterative magnitude pruning retaining ${70\%, 50\%, 30\%, 20\%, 10\%, 5\%}$ of parameters, retraining from original initialization.

   Application to toy example: For a 2-layer MLP with $w = 32$ on MNIST subset:

   • SGD with batch size 32 achieves 95.4% test accuracy
   • Hessian eigenvalue $\lambda_{\max} = 0.19$
   • Parameter movement $\Delta\theta_{\text{rel}} = 0.94$
   • 70% pruning retains 85.3% test accuracy

Prior work:

1. Zhang et al. (2017): “Understanding deep learning requires rethinking generalization” - showed DNNs can memorize random labels yet generalize on real data
2. Keskar et al. (2017): “On large-batch training for deep learning” - established connection between batch size and generalization via sharp/flat minima

3. Nakkiran et al. (2020): “Deep double descent” - documented test error decreasing beyond interpolation threshold

   Delta: This paper provides the first unified empirical framework evaluating all five major explanatory mechanisms under identical experimental conditions. The quantitative linking of batch size (11.8× Hessian eigenvalue difference), flatness, and generalization (1.61pp accuracy gap) within the same models is novel.

   Theory-specific assessment:

   • The main findings are predictable from prior work - each mechanism was studied individually before
   • The proof techniques are routine - standard empirical evaluation without new mathematical insights
   • No formal bounds are provided; the paper is purely empirical observation
   • The 2.25pp test accuracy difference across batch sizes, while statistically significant, represents an incremental quantification rather than a surprising discovery

   Verdict: INCREMENTAL — Solid empirical integration of known mechanisms but no fundamental theoretical advance or unexpected results.

This is primarily an empirical study with no formal proofs. The main technical contributions are experimental methodologies:

1. Hessian eigenvalue estimation via power iteration:

   \[v_{k+1} = \frac{H v_k}{\|H v_k\|_2}\]

   converging to the top eigenvector after 30 iterations.

2. Perturbation analysis using Gaussian noise:

   \[\theta' = \theta + \epsilon, \quad \epsilon \sim \mathcal{N}(0, \sigma^2 I)\]

   Applied at 7 magnitudes: $\sigma \in {0.0005, 0.001, 0.002, 0.005, 0.01, 0.02, 0.05}$.

3. Statistical analysis via bootstrap resampling:

   \[\text{CI}_{95\%} = [\text{percentile}_{2.5\%}, \text{percentile}_{97.5\%}]\]

   computed over 1,000 bootstrap samples.

4. NTK regime approximation through parameter tracking:

   \[\Delta\theta_{\text{rel}} = \frac{\|\theta_T - \theta_0\|_2}{\|\theta_0\|_2}\]

   Expected to scale as $O(1/\sqrt{w})$ for width $w$.

   The key technical insight is establishing paired measurements linking SGD noise → flat minima → good generalization within the same experimental framework, but this uses standard ML evaluation techniques rather than novel mathematical analysis.

Datasets: CIFAR-10 (50k train), MNIST (60k train, 10k subset for double descent)

Architectures:

• MLPs: 4 layers, width 8-4096, ReLU activation
• CNNs: 6 conv layers + 2 FC, channels 4-96, BatchNorm

Key Results:

• Double descent: Test accuracy improved from 95.4% → 96.9% (MNIST MLP) and 27.6% → 72.0% (CIFAR CNN) beyond interpolation
• Implicit regularization: 2.25pp test accuracy gap between batch sizes 32 vs 2048
• Flatness correlation: 11.8× Hessian eigenvalue difference corresponding to 1.61pp accuracy gap
• NTK convergence: Parameter movement decreased 11.3× from width 32 → 4096
• Lottery tickets: 90% pruning → only 1.15pp accuracy drop; random reinitialization → 2.80pp worse

Baselines: Full-batch GD (failed: 32.7% vs 85.5% for SGD), Adam (comparable to small-batch SGD), random pruning reinitialization

Limitations: Only CIFAR-10/MNIST, max 37M parameters (small by modern standards), CNNs only (no Transformers)

Limitations:

1. TECHNICAL: Limited to CIFAR-10/MNIST benchmarks - widely used but may not capture dynamics of large-scale training

2. RESTRICTIVE: Maximum 37M parameters tested - several orders of magnitude smaller than contemporary language models (billions/trillions of parameters)

3. TECHNICAL: Hessian eigenvalue estimates use power iteration on training subsets - local approximations in extremely high-dimensional spaces

4. NATURAL: Focus on MLPs/CNNs with BatchNorm - standard architectures but excludes Transformers with attention mechanisms

5. RESTRICTIVE: Full-batch GD experiments limited to 5k samples for computational feasibility - may not reflect true full-batch behavior

6. TECHNICAL: NTK approximation incomplete even at width 4096 - true infinite-width limit not reached

   Open problems:

7. Develop unified mathematical framework connecting implicit regularization, flat minima, and double descent beyond empirical correlations
8. Extend analysis to Transformer architectures and billion-parameter language models to test mechanism generality

1. Motivation: Free probability studies noncommutative random variables that model large random matrices, replacing classical independence with free independence. Existing free information geometry lacks a satisfying unified theory due to fundamental obstacles: geodesic concavity of entropy, gradient flow properties, and the relationship between different entropy versions remain open problems.

2. Mathematical setup: Let $(M_t)_{t \geq 0}$ be a noncommutative filtration of tracial von Neumann algebras with $M_s \subseteq M_t$ for $s \leq t$. Consider a free Brownian motion $z_t = (z_{t,j})_{j \in \mathbb{N}}$ compatible with the filtration, where each $z_{t,j} \in M_t$ and $z_{s,t} = z_t - z_s$ is free from $M_s$ for $s \leq t$.

   Define chronological formulas $\mathcal{F}_{\text{chron},m}$ recursively by:

   1. Starting with real parts of traces of polynomials in variables and Brownian increments
   2. Applying continuous functions to existing formulas
   3. Taking suprema/infima over operator-norm balls in chronological order

   The chronological type $\text{tp}^M_{\text{chron}}(x)$ of $x \in M_0^m$ maps each formula $\phi \in \mathcal{F}_{\text{chron},m}$ to $\phi^M(x) \in \mathbb{R}$.

   For matrix approximations, consider classical filtration $(G_t)_{t \geq 0}$ with Brownian motions $Z_t^{(n)}$ on $M_n$. Define the random matrix ultraproduct:

   \[M = \prod_{n \to \mathcal{U}} L^\infty(\Omega, G, P) \otimes M_n\] \[M_t = \prod_{n \to \mathcal{U}} L^\infty(\Omega, G_t, P) \otimes M_n\]

   Take quotient by maximal ideal to obtain II$_1$ factor $Q = M/I_V$ with filtration $(Q_t)_{t \geq 0}$.

3. Toy example: When $m = 1$ and $x$ is a single self-adjoint element, a simple chronological formula is $\phi(x) = \text{tr}(x^2 + z_{0,1}^2)$ where $z_{0,1}$ is the first Brownian increment. This reduces the complexity to understanding how traces of polynomials in $x$ and Gaussian increments behave under matrix approximations.

4. Formal objective: Define the chronological free entropy:

   \[\chi_{\text{chron}}^{\mathcal{U}}(\mu) = \limsup_{n \to \mathcal{U}} \frac{1}{n^2} \log \text{vol}(\{X \in M_n^m : |\Lambda_\phi^{(n)}(X) - \phi^Q(\pi(x))| \leq \delta \text{ for all } \phi \in S\})\]

The method constructs a new version of free entropy $\chi_{\text{chron}}^{\mathcal{U}}$ for chronological types using several key steps:

1. Expand test functions: Replace trace polynomials with chronological formulas that are closed under: - Continuous function application - Partial suprema/infima over operator-norm balls - Free heat semigroup application (in chronological order)

2. Matrix approximation via random filtrations: For each chronological formula $\phi$, define pointwise functions $\Lambda_\phi^{(n)}: M_n^m \to \mathbb{R}$ by taking classical expectations:

   \[\Lambda_\phi^{(n)}(X) = E[\Lambda_\psi^{(n)}(X, Z_{0,t,1}^{(n)}, \ldots, Z_{0,t,m}^{(n)})]\]

   where $Z_t^{(n)}$ is matrix Brownian motion on $M_n$ compatible with classical filtration $(G_t)_{t \geq 0}$.

3. Quotient construction: Build noncommutative filtration $(Q_t)_{t \geq 0}$ from random matrix ultraproduct:

   \[Q_t = \pi(M_t), \quad M_t = \prod_{n \to \mathcal{U}} L^\infty(\Omega, G_t, P) \otimes M_n\]

   where $\pi: M \to Q = M/I_V$ is quotient by maximal ideal determined by pure state $V$.

4. Entropy definition: Define microstate spaces using matrix approximations and chronological formulas:

   \[\Gamma_R^{(n)}(\mu; S, \delta) = \{X \in (D_R^{M_n})^m : |\Lambda_\phi^{(n)}(X) - \phi^Q(x)| \leq \delta \text{ for all } \phi \in S\}\]

   Then:

   \[\chi_{\text{chron}}^{\mathcal{U}}(\mu) = \sup_{S,\delta,R} \limsup_{n \to \mathcal{U}} \frac{1}{n^2} \log \text{vol}(\Gamma_R^{(n)}(\mu; S, \delta))\]

   Applied to toy example: For $\phi(x) = \text{tr}(x^2 + z_{0,1}^2)$ with single self-adjoint $x$, we get:

   \[\Lambda_\phi^{(n)}(X) = E[\text{tr}_n(X^2 + (Z_{0,1}^{(n)})^2)] = \text{tr}_n(X^2) + E[\text{tr}_n((Z_{0,1}^{(n)})^2)] = \text{tr}_n(X^2) + 1\]

   The microstate space becomes standard microstate approximations for $x$ with an additive constant.

Prior work: The closest prior papers are:

1. Voiculescu (1994, 1998): Introduced free entropies $\chi$ and $\chi^*$ but equality remains unknown; no geodesic concavity or gradient flow properties established
2. Biane-Voiculescu (2001): Defined free Wasserstein distance but connection to entropy gradient flows unproven

3. Jekel (previous papers 2021-2024): Developed free entropy for types in language of von Neumann algebras, but lacked filtration structure needed for gradient flow

   Delta: This paper adds several fundamental advances:

   • First construction of free entropy with proven geodesic concavity along Wasserstein geodesics
   • First proof that heat evolution satisfies evolution variational inequality (metric gradient flow property)
   • Chain rule for conditional entropy (previously only inequalities available)
   • Chronological formulas framework enabling closure under heat semigroup and partial suprema/infima

   Theory-specific assessment:

   • Main theorem surprise: The geodesic concavity result (Theorem A) and evolution variational inequality (Theorem B) are genuinely surprising. Free information geometry has been stuck on these fundamental properties for 25+ years.
   • Proof technique: Requires genuinely new techniques combining:
4. Monge-Kantorovich duality for chronological types
5. infinitesimal change of variables for entropy with limited regularity
6. matrix model approximations for optimal couplings. The chronological ordering constraint is a novel technical innovation.
   • Bound tightness: No lower bounds are established. The entropy bounds depend on ultrafilter choice, so tightness relative to absolute free entropy versions remains unclear.

   The technical obstacles overcome are substantial: noncommutative probability spaces don’t admit quantifier elimination (unlike classical case), suprema over operator balls don’t preserve trace polynomial structure, and free heat semigroup application requires sophisticated filtration analysis.

   Verdict: SIGNIFICANT — Resolves long-standing fundamental problems in free information geometry with novel proof techniques, though full connection to classical free entropy versions remains incomplete.

The main proof strategy combines three sophisticated technical innovations:

1. Chronological Monge-Kantorovich duality (Theorem 7.7): For optimal coupling $(x_0, x_1)$ of chronological types $\mu_0, \mu_1$, constructs convex functions $\varphi, \psi$ via Legendre transforms such that:

   \[\varphi(x) + \psi(y) \geq \langle x, y \rangle\]

   with equality for the coupling. Key inequality uses inf-convolution:

   \[\varphi_t(x) = \inf_{z} [\frac{1-t}{t} \varphi_0(z) + \frac{1}{t} \varphi_1(\frac{x-z}{t})]\]

   The technical challenge is that suprema over operator-norm balls don’t preserve trace polynomial structure, resolved using chronological ordering.

2. Infinitesimal change of variables (Theorem 6.25): For chronologically definable predicate $\varphi$ and free Brownian motion $z_t$, establishes:

   \[\frac{d}{dt}\Big|_{t=0} \chi_{\text{chron}}^{\mathcal{U}}(x + z_t) = \int \Delta \varphi(x) \, d\nu(x)\]

   where $\Delta$ is the free Laplacian defined via heat semigroup:

   \[\frac{1}{2}\Delta \varphi(x) = \lim_{t \to 0^+} \frac{\varphi(x + z_t) - \varphi(x)}{t}\]

   The proof uses:

   • Discrete-time approximation of matrix Brownian motion with error bounds uniform in $n$

   • Concentration of measure for Gaussian matrices: $P(

     f(Z^{(n)}) - Ef(Z^{(n)})

     \geq \delta) \leq 2e^{-n^2\delta^2/2L^2}$

   • Ultralimit techniques to pass from matrix approximations to noncommutative setting

3. Matrix model extension theorem (Theorem 7.9): Given optimal coupling $(x_0, x_1)$ in chronological types, constructs matrix approximations $X_0^{(n)}, X_1^{(n)}$ such that:

   • $(X_0^{(n)}, X_1^{(n)})$ form classical optimal coupling
   • Normalized classical entropy approximates chronological entropy:
   \[\lim_{n \to \mathcal{U}} \frac{1}{n^2} h_{\text{classical}}(X_0^{(n)}) = \chi_{\text{chron}}^{\mathcal{U}}(\mu_0)\]

   This uses stochastic control representation:

   \[\tilde{\chi}_{\text{chron}}^{\mathcal{U}}(x) = \sup_\phi [P^{\mathcal{U}}(\phi) - \phi^Q(x)]\]

   where:

   \[P^{\mathcal{U}}(\phi)(y) = \inf_\alpha [\phi^Q(\pi(z_1) + \int_0^1 \alpha_t dt, y) + \frac{1}{2} \int_0^1 \|\alpha_t\|_2^2 dt]\]

   The geodesic concavity follows by combining these three ingredients: duality provides the convex functions, change of variables computes derivatives along geodesics, and matrix extensions handle boundary behavior at geodesic endpoints.

Purely theoretical. The paper establishes foundational properties for a new entropy functional without computational experiments.

Empirical validation would require:

1. Computing $\chi_{\text{chron}}^{\mathcal{U}}$ for explicit examples (semicircular elements, free Gibbs laws) and comparing with existing free entropy versions $\chi, \chi^*$
2. Numerical verification of geodesic concavity along specific Wasserstein geodesics
3. Large deviations experiments for random matrix ensembles to test the rate function predictions (Corollary F)
4. Comparison of matrix approximation convergence rates for chronological vs. standard microstate spaces

Limitations:

1. RESTRICTIVE: Entropy is only defined for chronological types that admit matrix approximations, requiring elementary equivalence to the specific ultraproduct $Q$. This significantly narrows applicability compared to classical free probability.

2. TECHNICAL: Ultrafilter dependence means $\chi_{\text{chron}}^{\mathcal{U}}$ may vary with choice of $\mathcal{U}$, unlike classical entropy which should be canonical. No uniqueness or independence results established.

3. TECHNICAL: The relationship to Voiculescu’s original free entropies $\chi$ and $\chi^*$ remains unclear. The paper doesn’t establish when $\chi_{\text{chron}}^{\mathcal{U}} = \chi$ or $\chi^*$, which would be needed for broader impact.

4. TECHNICAL: Chronological ordering constraint limits the class of test functions. It’s unclear whether this restriction is essential or could be relaxed while preserving the main results.

5. RESTRICTIVE: Results only apply to filtrations generated by free Brownian motion, not general noncommutative filtrations or stochastic processes.

6. TECHNICAL: No explicit computation methods provided. The entropy remains defined implicitly through suprema over infinite-dimensional function classes.

   Open problems:

7. Ultrafilter independence: Prove that $\chi_{\text{chron}}^{\mathcal{U}}$ is independent of ultrafilter choice, establishing canonical entropy values.

8. Connection to classical free entropy: Establish conditions under which $\chi_{\text{chron}}^{\mathcal{U}} = \chi$ or $\chi^*$, particularly for semicircular and free Gibbs laws where explicit computations are feasible.

1. Motivation: Polyak-Ruppert averaged SGD achieves optimal O(1/n) convergence but requires estimating the asymptotic covariance V for confidence intervals and statistical inference. Computing V involves the Hessian H, which is expensive or unavailable in many applications like deep learning.

2. Mathematical setup: Let F(x) = E[f(x,ζ)] be the population risk with minimizer x∗. SGD iterates follow:

   \[x_{t+1} = x_t - \eta_t \nabla f(x_t, \zeta_t)\]

   with polynomial step sizes ηt = η0 t^{-α} for α ∈ (1/2,1). The Polyak-Ruppert average x̄n = n^{-1} ∑_{t=1}^n x_t satisfies:

   \[\sqrt{n}(\bar{x}_n - x^*) \overset{d}{\to} N(0, V)\]

   where V = H^{-1}SH^{-1} with H = ∇²F(x∗) and S = E[∇f(x∗,ζ)∇f(x∗,ζ)^⊤].

   Assumptions:

   1. F is μ-strongly convex: ∇²F(x) ⪰ μI
   2. Hessian is L-Lipschitz: ‖∇²F(x) - ∇²F(y)‖_op ≤ L‖x-y‖
   3. Sub-Gaussian gradient noise with parameter σ
   4. Bounded fourth moments: E[‖ξt‖⁴] ≤ M4
   5. Step-size condition: α ∈ (1/2,1) and η0 > 1/(2μ)

3. Toy example: For d = 2 quadratic F(x) = ½‖x‖², we have H = I and S = I, so V = I. With α = 0.6, batch-means blocks grow as am = ⌊Cm^β⌋ with roughly β = 2.5 for the improved estimator.

4. Formal objective: Estimate V from trajectory {x1,…,xn} to minimize:

   \[E[‖\hat{\Sigma}_n(\rho) - V‖_{op}]\]

The online batch-means estimator partitions {x1,…,xn} into growing blocks of sizes am = ⌊Cm^β⌋ with endpoints tm = ∑_{j=1}^m aj. For each block m, compute the centered block sum:

\[Y_m = \frac{1}{\sqrt{a_m}} \sum_{k=t_{m-1}+1}^{t_m} (x_k - \bar{x}_n)\]

The estimator with burn-in fraction ρ ∈ (0,1] is:

\[\hat{\Sigma}_n(\rho) = \frac{1}{K_n} \sum_{m=b_n-K_n+1}^{b_n} Y_m Y_m^⊤\]

where Kn = ⌊ρbn⌋ and bn = Θ(n^{1/(β+1)}) is the number of complete blocks.

Key innovation: Set β = (1+2α)/(2(1-α)) instead of the original β = 2/(1-α), which creates more blocks (bn = Θ(n^{1/3}) vs Θ(n^{1/4}) at α → 1/2⁺) and reduces variance.

Applying to the toy example: With d = 2, α = 0.6, the optimal β† ≈ 2.33 gives roughly n^{0.3} blocks. Each block m has size roughly m^{2.33}, and the last 50% of blocks (ρ = 0.5) are averaged to form the final estimate.

Prior work:

1. Zhu, Chen and Wu (2023): Original online batch-means estimator achieving O(n^{-(1-α)/4}) = O(n^{-1/8}) rate
2. Roy and Balasubramanian (2023): Extended to Markovian noise, same O(n^{-1/8}) rate

3. Chen et al. (2020): Plug-in estimator using Hessian information achieving O(n^{-1/2}) but requiring second-order oracle access

   Delta: This paper provides:

4. Rigorous per-block bias analysis showing stationarity bias is O(τm/am) where τm/am is mixing-time to block-size ratio
5. Optimal block-growth parameter β† = (1+2α)/(2(1-α)) improving rate from O(n^{-(1-α)/4}) to O(n^{-(1-α)/3})

6. Minimax rate Θ(n^{-(1-α)/2}) via Le Cam lower bound and trajectory-regression upper bound

   Theory-specific assessment:

   • Main theorem is somewhat predictable: improving batch-means rates via better bias-variance tradeoffs is natural
   • Proof uses standard techniques (matrix Bernstein, linearization) but the per-block bias analysis is more careful than prior work
   • Bounds are reasonably tight: batch-means achieves O(n^{-1/6}) while minimax optimal is O(n^{-1/4}), leaving a polynomial gap
   • The Le Cam construction is standard two-point method on quadratic objectives

   Verdict: INCREMENTAL — solid improvement over Zhu, Chen and Wu (2023) but uses expected techniques and leaves gap to minimax rate.

1. Three-way error decomposition: Decompose estimation error as:

   \[\hat{\Sigma}_n(\rho) - V = T_1 + T_2 + T_3\]

   where T1 = variance term, T2 = stationarity bias, T3 = nonlinearity bias.

2. Linearization via Taylor expansion: Replace SGD recursion with linearized version:

   \[\delta_{t+1} = (I - \eta_t H)\delta_t - \eta_t \xi_t - \eta_t r_t\]

   where δt = xt - x* and remainder rt satisfies E[‖rt‖²] = O(t^{-2α}).

3. Variance bound via matrix Bernstein: Use mixing properties under condition β > α/(1-α) to show blocks are approximately independent. Apply matrix Bernstein inequality:

   \[E[‖T_1‖_{op}] = O(K_n^{-1/2}) = O(n^{-1/(2(β+1))})\]

4. Key per-block bias analysis: Show that block m has stationarity bias:

   \[‖V^{(m)} - V‖_{op} = O(\tau_m/a_m) = O(m^{-\gamma})\]

   where γ = β(1-α) - α and τm is the mixing time. This gives:

   \[‖T_2‖_{op} = O(n^{-\gamma/(β+1)})\]

5. Rate optimization: Balance variance and bias terms by setting:

   \[\frac{1}{2(β+1)} = \frac{\gamma}{β+1} \Rightarrow γ = \frac{1}{2}\]

   This yields optimal β† = (1+2α)/(2(1-α)) and rate O(n^{-(1-α)/3}).

6. Le Cam lower bound construction: Use two quadratic objectives F0(x) = ½‖x‖², F1(x) = ½x^⊤(I+δA)x with same noise S = I. The KL divergence bound:

   \[D_{KL}(P_0^{(n)} \| P_1^{(n)}) \leq \kappa\delta² n^{1-α}\]

   Combined with separation ‖V0 - V1‖op ≥ δ gives lower bound Ω(n^{-(1-α)/2}).

Purely theoretical. The paper mentions numerical experiments in Section 8 but focuses on theoretical analysis. Empirical validation would involve:

1. Testing the improved O(n^{-1/6}) rate vs original O(n^{-1/8}) rate on quadratic and non-quadratic objectives
2. Comparing batch-means vs trajectory-regression estimators
3. Validating the burn-in parameter ρ selection and block-growth parameter β tuning
4. High-dimensional performance where d grows with n

Limitations:

1. TECHNICAL: Mixing condition β > α/(1-α) requires blocks to contain many mixing times - needed for proof technique but potentially removable
2. TECHNICAL: Fixed dimension d assumption - constants depend on d through log d factors from matrix Bernstein inequality
3. NATURAL: Strong convexity and Lipschitz Hessian assumptions - standard in SGD theory

4. RESTRICTIVE: Batch-means achieves O(n^{-(1-α)/3}) vs minimax O(n^{-(1-α)/2}) - fundamental gap or artifact of method?

   Open problems:

5. Close the gap between batch-means rate O(n^{-1/6}) and minimax rate O(n^{-1/4}) - can batch-means methods achieve minimax optimality?
6. Extend to high-dimensional regime where d = d(n) grows with n while preserving polynomial convergence rates.