Theory Digest — Mar 29, 2026

1. Motivation: Foundation model adaptation to new domains with limited supervision suffers from latent distribution mismatch, unstable optimization dynamics, and miscalibrated uncertainty propagation. This problem matters in deploying large-scale pretrained models across heterogeneous real-world environments where target data is scarce.

2. Mathematical setup: Let $D_s = {(x_s^i, y_s^i)}_{i=1}^{n_s}$ denote the labeled source dataset and $D_t = {x_t^j}_{j=1}^{n_t}$ denote the sparsely labeled target dataset. A pretrained foundation encoder $f_\theta : X \to \mathbb{R}^d$ maps inputs to latent representations $z = f_\theta(x)$. The latent representations are modeled as stochastic variables:

   \[z_s \sim p_s(z)\] \[z_t \sim p_t(z)\]

   where $p_s$ and $p_t$ represent source and target latent density manifolds respectively.

   Assumptions:

   1. The transport operator $T_\phi$ satisfies Lipschitz continuity with constant $L_\phi$
   2. The loss function is sub-Gaussian
   3. Bounded transport curvature and finite PAC-Bayesian divergence

3. Toy example: When $d=2$ and source covariance $\Sigma_s = I_2$, the source distribution is $p_s(z) = \mathcal{N}(0, I_2)$ while the target exhibits geometric deformation $p_t(z) = \mathcal{N}(\mu_t, \Sigma_t)$ with $\mu_t \neq 0$ and $\Sigma_t \neq I_2$. This captures the core difficulty of aligning mismatched latent geometries under uncertainty.

4. Formal objective: The probabilistic transport functional to minimize is:

   \[L_{transport} = \mathbb{E}_{z_s \sim p_s}\left[\int_{\mathbb{R}^d} c(z_s, z_t) q_\phi(z_t | z_s) dz_t\right] + \lambda \text{KL}(q_\phi(z_t) \| p_t(z_t))\]

The method introduces a Bayesian transport operator $T_\phi$ parameterized by $\phi$ such that:

\[q_\phi(z_t | z_s) = T_\phi(p_s(z))\]

where $q_\phi$ denotes the transported posterior distribution in the target latent space.

The uncertainty propagation dynamics are modeled through a stochastic differential mapping:

\[dz_t = \mu_\phi(z_s) dt + \Sigma_\phi^{1/2}(z_s) dW_t\]

where $W_t$ is a standard Wiener process.

The unified optimization objective integrates:

\[L = L_{task} + \alpha L_{transport} + \beta \text{KL}(\rho \| \pi)\]

where $\alpha$ and $\beta$ regulate geometric alignment and posterior complexity respectively.

Algorithm steps:

1. Initialize pretrained parameters $\theta_0$ and prior $\pi$
2. Sample minibatch latent embeddings $z_s \sim p_s$

3. Estimate stochastic transport posterior $q_\phi(z_t

   z_s)$

4. Update encoder parameters by minimizing $L$
5. Regularize posterior complexity via PAC-Bayesian penalty

6. Iterate until convergence of transport divergence

   Applied to toy example: For $d=2$ Gaussian case, the transport operator learns to map $\mathcal{N}(0, I_2) \to \mathcal{N}(\mu_t, \Sigma_t)$ while maintaining calibrated uncertainty through the covariance propagation $\Sigma_\phi(z_s)$.

Step 1 — Prior work:

• “PAC-Bayes and domain adaptation” (Germain et al., 2020): Established PAC-Bayesian bounds combining source risk with distribution divergence terms
• “Domain-Indexing Variational Bayes” (Xu et al., 2023): Introduced continuous latent domain descriptors through variational inference
• “Bayesian Domain Adaptation with Gaussian Mixture Domain-Indexing” (Ling et al., 2024): Extended to mixture-based priors for richer inter-domain modeling

Step 2 — Delta: This paper combines Wasserstein-type probabilistic transport with PAC-Bayesian regularization. The specific additions are:

1. stochastic differential equation formulation of latent transport
2. unified objective integrating transport and PAC-Bayesian terms

3. convergence analysis of the gradient flow dynamics.

   Step 3 — Theory-specific assessment:

   • The main theorem combining transport costs with PAC-Bayesian bounds is a predictable extension of existing results
   • The proof technique assembles known lemmas from optimal transport and PAC-Bayes theory without fundamental innovation
   • No lower bounds are provided or compared against
   • The convergence analysis relies on standard convexity arguments for KL functionals

   The experimental validation shows moderate improvements but lacks comparison to recent strong baselines in foundation model adaptation.

   Verdict: INCREMENTAL — solid combination of existing optimal transport and PAC-Bayesian techniques but lacks surprising theoretical insights or breakthrough empirical results.

The main proof strategy combines optimal transport duality with PAC-Bayesian change-of-measure inequalities:

1. Transportation-cost inequality: The change-of-measure bound relates source and target risks through Wasserstein distance:

   \[\mathbb{E}_{z_t \sim p_t}[\ell(z_t)] - \mathbb{E}_{z_s \sim p_s}[\ell(z_s)] \leq W_2(p_s, p_t)\]

2. PAC-Bayesian concentration: Applying Bernstein-type concentration to the posterior yields:

   \[\mathbb{P}\left[R_t(\rho) \leq \hat{R}_s(\rho) + \sqrt{\frac{\text{KL}(\rho \| \pi) + \log(2\sqrt{n_s}/\delta)}{2n_s}}\right] \geq 1-\delta\]

3. Kantorovich dual formulation: The Wasserstein distance is upper-bounded using:

   \[W_2(p_s^\theta, p_t^\theta) \leq \sup_{\|f\|_L \leq 1} \left|\int f(z) p_s^\theta(z) dz - \int f(z) p_t^\theta(z) dz\right|\]

4. Convergence analysis: The gradient flow dynamics are analyzed using the Lyapunov functional:

   \[V(t) = L(\rho_t) - L(\rho^*)\]

   which decreases monotonically due to convexity of the KL functional in distribution space.

   The key technical insight is that probabilistic transport acts as a curvature stabilizer, but this follows from standard smoothness properties of Wasserstein potentials.

Datasets: The paper uses synthetic distribution shift scenarios and controlled benchmark datasets but lacks details on specific real-world datasets.

Baselines: Standard fine-tuning, DANN (adversarial domain adaptation), and generic “Bayesian DA” methods.

Key numbers:

• 53% reduction in geometry mismatch vs adversarial transfer
• 55% reduction in target risk vs standard fine-tuning
• 21% improvement in uncertainty calibration vs Bayesian baselines
• 63% reduction in transport energy vs deterministic methods

The experimental section lacks comparison to recent strong foundation model adaptation methods and relies heavily on synthetic scenarios. The evaluation metrics (geometry discrepancy, transport energy) are method-specific rather than standard domain adaptation benchmarks.

Limitations:

1. Computational overhead from stochastic transport estimation and covariance propagation scales as O(md² + Km²) - TECHNICAL (could potentially be reduced with low-rank approximations)
2. Experimental validation focuses on controlled synthetic scenarios rather than real-world deployment - RESTRICTIVE (significantly limits practical applicability assessment)
3. Assumes bounded transport curvature and sub-Gaussian loss functions - NATURAL (widely satisfied in practice)
4. Requires access to source domain during adaptation - NATURAL (standard assumption in domain adaptation)

5. Memory complexity O(d²) for posterior covariance storage - TECHNICAL (addressable via low-rank parameterization)

   Open problems:

6. Extending to multimodal foundation models where different modalities may require different transport operators
7. Developing adaptive PAC-Bayesian priors that automatically adjust to domain shift severity

1. Motivation: Bayesian structural equation models (SEMs) rely on computationally expensive MCMC methods. This matters because SEMs are widely used in psychology, social sciences, and latent variable modeling where fast, accurate inference would enable broader adoption of Bayesian approaches.

2. Mathematical setup: Let $y_s \in \mathbb{R}^p$ be observed data for subject $s = 1, \ldots, n$. The SEM is defined as a hierarchical latent Gaussian model:

   \[y | \eta, \vartheta_1 \sim \prod_{s=1}^n \phi(y_s; \nu + \Lambda\eta_s, \Theta)\] \[\eta | \vartheta_2 \sim \prod_{s=1}^n \phi((I-B)\eta_s; \alpha, \Psi)\] \[\vartheta := \{\vartheta_1, \vartheta_2\} \sim \pi(\vartheta)\]

   where $\phi(\cdot; \mu, \Sigma)$ is multivariate normal density, $\eta_s \in \mathbb{R}^q$ are latent variables, $\Lambda$ is $p \times q$ factor loading matrix, $\Theta$ is $p \times p$ error covariance, $B$ is $q \times q$ regression coefficient matrix (hollow), and $\Psi$ is $q \times q$ structural disturbance covariance.

   Assumptions:

   1. $B$ can be permuted to strictly triangular form (acyclic graph)
   2. Standard regularity conditions for likelihood identification
   3. Measurement errors independent of $\eta_s$
   4. Gaussian distributions throughout

3. Toy example: When $q=2$, $p=4$, $n=100$ with $\Lambda$ having two loadings per factor and $B_{2,1} \neq 0$, the marginal likelihood becomes a multivariate Gaussian with mean $\mu(\vartheta) = \nu + \Lambda(I-B)^{-1}\alpha$ and covariance involving matrix inversions that become computationally expensive for MCMC.

4. Formal objective: Target the posterior distribution

   \[\pi(\vartheta | y) \propto \int \pi(y | \eta, \vartheta) \pi(\eta | \vartheta) \pi(\vartheta) d\eta\]

The method analytically integrates out latent variables $\eta$ to work with the marginal likelihood directly, then uses a 4-stage approximation:

Stage 1 - Joint Laplace Approximation: Find posterior mode $\vartheta^* = \arg\max_\vartheta \log \pi(y, \vartheta)$ and compute Hessian:

\[H := -\nabla^2_\vartheta \log \pi(y, \vartheta)|_{\vartheta=\vartheta^*}\]

Stage 2 - Skew-Normal Marginal Profiling: For each parameter $\vartheta_j$, construct conditional mean path $C_j$ and evaluate log-posterior on grid:

\[G_j = \{\vartheta^* + t v_j | t \in [-4,4]\}\]

where $v_j = \Omega_{\cdot j}/\sqrt{\Omega_{jj}}$ with $\Omega = H^{-1}$. Apply volume correction:

\[h_{kj} := \log \pi(\vartheta(t_k) | y) + t_k \gamma'_j(0)\]

Fit skew-normal $SN(\xi_j, \omega_j, \alpha_j)$ to these evaluations by weighted least squares.

Stage 3 - Variational Bayes Correction: Find location shift $\hat{\delta}$ by minimizing KLD:

\[\hat{\delta} = \arg\max_\delta \mathbb{E}_{\vartheta \sim N_m(\vartheta^*+\delta,\Omega)}[\log \pi(\vartheta | y)]\]

Stage 4 - Gaussian Copula Sampling: Generate joint samples preserving skew-normal marginals:

1. Sample $z^{(b)} \sim N_m(0, R^*)$ with NORTA-adjusted correlation
2. Transform: $u^{(b)}_j = \Phi(z^{(b)}_j)$

3. Invert: $\vartheta^{(b)}_j = Q^{(j)}_{SN}(u^{(b)}_j)$

   Toy example application: For the simple case, the method would find the mode of the 6-dimensional posterior (2 loadings per factor + variance parameters), profile each dimension on a grid of ~40 points, fit skew-normal to each profile, then generate correlated samples that preserve the fitted marginal shapes.

Prior work:

1. “Integrated nested Laplace approximations for Bayesian analysis of structured additive regression models” (Rue et al., 2009) - original INLA framework for latent Gaussian models
2. “Bayesian structural equation modeling with blavaan” (Merkle et al., 2021) - showed marginal likelihood approach improves over data augmentation in Stan

3. Various Bayesian SEM papers using MCMC (Muthén & Asparouhov, 2012; Merkle & Rosseel, 2018)

   Delta: This paper adapts INLA specifically to SEMs by analytically integrating out latent variables rather than treating them as a GMRF. Key additions:

4. efficient volume correction via gradient-based updates
5. skew-normal fitting to dense grids instead of derivative matching
6. VB mean correction in parameter space rather than latent field space

7. Gaussian copula reconstruction with NORTA adjustment.

   Theory-specific assessment:

   • Main result is engineering/computational rather than theoretical breakthrough
   • Proof techniques are standard (Laplace approximation, variational bounds, skew-normal properties)
   • No new theoretical bounds or complexity analysis
   • Accuracy depends on standard Laplace approximation assumptions (unimodal, well-separated mode)

   Verdict: INCREMENTAL — solid engineering contribution that makes existing INLA ideas more efficient for SEMs, but no fundamental theoretical advances.

The method relies on standard approximation techniques rather than novel proofs:

1. Laplace Approximation: Uses second-order Taylor expansion around posterior mode:

   \[\log \pi(\vartheta | y) \approx \log \pi(\vartheta^* | y) - \frac{1}{2}(\vartheta - \vartheta^*)^T H (\vartheta - \vartheta^*)\]

2. Conditional Mean Path Lemma (Lemma 3.1): For multivariate normal $\vartheta \sim N_m(\vartheta^*, \Omega)$:

   \[\pi(\vartheta_j = x) \propto \pi(\vartheta_j = x, \vartheta_{-j} = \mathbb{E}[\vartheta_{-j} | \vartheta_j = x])\]

3. Volume Correction (Lemma 3.2): Key technical result showing volume slope admits form:

   \[\gamma'_j(0) = -\frac{1}{2} \text{tr}(P^\perp_j J')\]

   where $P^\perp_j = I_m - w_j w_j^T$ projects orthogonal to scan direction and $J’ = \frac{d}{dt}J(t)

   _{t=0}$ with $J(t) = L^T H(t) L$.

4. Gradient Representation (Corollary 3.1): Shows volume slope computable via finite differences:

   \[\gamma'_j(0) = -\frac{1}{2}\sum_{k=1}^m \frac{d}{dt}[L_{·k}^T \frac{d}{dL_{·k}} g]|_{t=0} + \frac{1}{2}\frac{d}{dt}[v_j^T \frac{d}{dv_j} g]|_{t=0}\]

   requiring only $m+2$ gradient evaluations per parameter.

5. Skew-Normal Properties: Uses standard moment formulas and Owen’s T-function for CDF evaluation.

6. Variational Bound: Minimizes KLD $D_{KL}(q_\delta | \pi(\cdot

   y))$ where $q_\delta = N_m(\vartheta^* + \delta, \Omega)$ via expected log-posterior maximization.

Datasets: Political Democracy SEM benchmark (Bollen, 1989) with n=75, p=11 observed variables, q=3 latent factors, m=42 parameters.

Simulation study: Generated data at sample sizes n ∈ {75, 150, 300, 600, 1200, 2400} with B=250 replications each.

Key numbers:

• Speed: INLA method 1.68 seconds vs MCMC 27.9 seconds per chain (16x speedup)

• Accuracy: Median standardized mean discrepancy

  ΔMean

  /SD_MCMC < 0.06 across all parameter classes, maximum 0.22

• Marginal agreement: Jensen-Shannon similarity >98.5% for all 42 parameters vs MCMC
• Coverage: 95% credible intervals achieve 90-96% coverage, 50% intervals achieve 44-54% coverage across simulation conditions
• Convergence: MCMC achieved R̂=1.00 and minimum n_eff=9,646

Baselines: High-quality MCMC (3 chains, 10,000 post-warmup samples each) using Stan via blavaan package.

Simulation-based calibration: Under informative priors shows excellent calibration, under diffuse priors shows miscalibration due to selection bias from failed optimization attempts (500/5418 success rate).

Limitations:

1. Unimodal assumption - RESTRICTIVE: Requires well-separated interior mode with positive definite Hessian, fails for multimodal or boundary-concentrated posteriors

2. Skew-normal flexibility - TECHNICAL: Skew-normal family bounded to skewness ∈ (-0.9953, 0.9953), may inadequately capture heavy-tailed or boundary-concentrated distributions

3. First-order volume correction - TECHNICAL: Uses linear approximation γ_j(t) ≈ γ_j(0) + t γ’_j(0), higher-order terms could improve accuracy

4. Gaussian copula dependence - NATURAL: Preserves only linear correlations from Hessian, loses nonlinear dependence structure that MCMC would capture

5. Normal-theory SEMs only - NATURAL: Method specific to continuous Gaussian SEMs, doesn’t extend to categorical, multilevel, or non-normal cases

6. Prior sensitivity - RESTRICTIVE: Requires reasonable priors to avoid numerical failures, diffuse priors lead to ~91% optimization failure rate

   Open problems:

7. Extension to non-Gaussian SEMs (ordinal, count, survival outcomes) where analytical integration of latent variables is not available

8. Development of adaptive prior elicitation schemes that automatically avoid parameter space regions leading to numerical instability while maintaining weak informativeness

1. Motivation: Real-world networks often exhibit geometric structure where edges depend on similarity between latent features of vertices, rather than simple density patterns. This work addresses detecting such local geometric structure in random graphs, which has applications in identifying genuine users among bots, coordinated accounts, or collusive firms.

2. Mathematical setup:

   Consider the hypothesis testing problem:

   \[P := G(n, p, d, k) \quad \text{vs.} \quad Q := G(n, p)\]

   Under model $P$:

   1. Each vertex $i \in [n]$ joins community $S$ independently with probability $k/n$
   2. Each community vertex $i \in S$ receives latent vector $U_i \stackrel{i.i.d.}{\sim} U(S^{d-1})$
   3. For $i, j \in [n], i \neq j$: if $i,j \in S$, edge $ij$ exists iff $\langle U_i, U_j \rangle \geq \tau(p,d)$; otherwise edge exists with probability $p$

   The threshold $\tau(p,d)$ satisfies:

   \[P(\langle U_i, U_j \rangle \geq \tau(p,d)) = p\]

   Key assumptions:

   1. $k = k(n) \to \infty$ as $n \to \infty$
   2. $d$ sufficiently large
   3. $p \leq 1/2$

3. Toy example: When $d = 2, k = 7, p = 0.18$, the community vertices have latent vectors on $S^1$ (unit circle). Edges within the community form based on angular proximity, creating cycle-rich structure that differs from Erdős-Rényi at the dependency level while maintaining identical marginal edge probabilities.

4. Formal objective: Determine the detection threshold - the largest dimension $d$ (as function of $n, p, k$) for which weak/strong detection remains possible:

   \[d^* = \sup\{d : \text{detection possible}\}\]

The authors propose three test statistics based on signed triangle counts.

Global test:

\[f_{\text{tri}}(G) := \sum_{i<j<\ell \in [n]} (G_{ij} - p)(G_{j\ell} - p)(G_{i\ell} - p)\]

Scan test:

\[f_{\text{scan}}(G) := \max_{A \subseteq [n], |A| = k_-} \sum_{i<j<\ell \in A} (G_{ij} - p)(G_{j\ell} - p)(G_{i\ell} - p)\]

where $k_- := \lfloor 0.9k \rfloor$.

Constrained scan test:

\[\tilde{f}_{\text{scan}}(G) := \max_{A \in C(G), |A| = k_-} \sum_{i<j<\ell \in A} (G_{ij} - p)(G_{j\ell} - p)(G_{i\ell} - p)\]

The constraint set $C(G)$ requires:

\[\sum_{i<j \in A} W_{ij}^2 \leq \sigma^2\] \[\max_{i<j \in A} |W_{ij}| \leq B\]

where $W_{ij} = \sum_{\ell < i, \ell \in A} (G_{\ell i} - p)(G_{\ell j} - p)$ and $\sigma^2 = k^3 p^2 + O(k^4 p^4 \log^2(1/p)/d)$, $B = O(kp^2 \log k)$.

Application to toy example: For the $d=2, k=7, p=0.18$ case, the planted community would have elevated signed triangle count due to geometric homophily, while background Erdős-Rényi triangles have expected signed count zero due to centering.

Step 1 — Prior work:

• BDER16: Detection threshold $d = \Theta(n^3)$ for full model $G(n,p,d)$ vs $G(n,p)$ with fixed $p$
• LMSY22: State-of-the-art bounds $d = \tilde{O}(n^3 p^3)$ (upper) and $d = \tilde{\Omega}(n^3 p^2)$ (lower) for general $p$

Step 2 — Delta: This paper introduces locality by considering hidden community of size $k \ll n$, extending from full model ($k=n$) to planted subgraph setting. New contributions:

• Three-tier testing framework (global/scan/constrained scan)
• Truncated second moment method via Wishart-GOE comparison
• Tensorization approach adapted for community structure
• Low-degree polynomial lower bounds

Step 3 — Theory-specific assessment:

• Main theorem is predictable extension from $k=n$ to general $k$, but requires significant technical innovation
• Proof techniques combine standard methods (signed triangle statistics) with novel matrix comparisons and constraint-based scanning
• Bounds achieve $d = \tilde{\Theta}(k^2 \vee k^6/n^3)$ for fixed $p$, extending LMSY22’s $d = \tilde{\Theta}(n^3 p^2)$ to planted setting
• No known lower bounds for comparison in planted setting

Verdict: INCREMENTAL — solid extension of high-dimensional random geometric graph detection to planted community setting with several technical advances, but follows expected trajectory from prior work.

1. Upper bounds via signed triangle statistics:

   Key insight: geometric homophily creates triangle excess in community. For global test:

   \[\mathbb{E}_{G \sim P}[f_{\text{tri}}(G)] = \binom{n}{3} \left(\frac{k}{n}\right)^3 \mathbb{E}_{G \sim G(n,p,d)}[(G_{12}-p)(G_{23}-p)(G_{13}-p)]\]

   By BDER16/LMSY22:

   \[\mathbb{E}_{G \sim G(n,p,d)}[(G_{12}-p)(G_{23}-p)(G_{13}-p)] \geq \frac{p^3 \log^{3/2}(1/p)}{C\sqrt{d}}\]

   Variance control requires bounding signed 4-cycle contributions under $P$.

2. Scan test concentration: Uses strong concentration (Lemma A.1) for polynomials of subgaussian variables to handle union bound over $\binom{n}{k_-}$ subsets.

3. Constrained scan filtering: Constraints eliminate dense clique-like patches that produce large signed wedge counts without geometric structure. Uses Freedman’s inequality with martingale differences:

   \[M_r - M_{r-1} = (G_{ij} - p) \sum_{\ell < i, \ell \in [k_-]} (G_{\ell i} - p)(G_{\ell j} - p)\]

4. Lower bound via truncated second moment:

   Views $P, Q$ as pushforwards of Wishart vs GOE. Key technical step - comparison between Wishart and spherical Wishart distributions:

   \[d_{TV}(\text{Wishart}_k, \text{Spherical-Wishart}_k) = o(1) \text{ when } d \gg k^2\]

   Second moment calculation truncates extreme eigenvalues, comparing overlap of two i.i.d. communities.

5. Tensorization for $p$-dependence: Sequential vertex revelation with augmented latent variables $(U_i, V_i)$ encoding both features and community membership. Recursive inequalities for martingale second moments improve condition from $d = \tilde{\Omega}(k^3 p^3)$ to $d = \tilde{\Omega}(k^2 p^2)$.

6. Low-degree lower bounds: Bounds squared Fourier coefficients $\sum_H

   \Phi_P(H)

   ^2$ over subgraphs $H$ of polylog size. Uses forest contributions vanish (Corollary 4.2) and moment bounds depending only on $v(H), e(H)$.

Purely theoretical. Empirical validation would involve:

1. Synthetic experiments comparing detection performance of proposed test statistics across different parameter regimes $(n, p, d, k)$

2. Real network analysis on datasets with known geometric structure (social networks, biological networks) to validate model assumptions

3. Computational experiments measuring runtime of scan vs constrained scan tests to quantify efficiency gains

4. Comparison with spectral methods and other community detection algorithms in geometric setting

Limitations:

1. Gap between upper/lower bounds for $0 < \alpha < 1$ (vanishing $p$ regime) - TECHNICAL (likely tightened with better analysis)

2. Assumption $p \leq 1/2$ - NATURAL (standard in random graph literature, extends to $p \in (1/2,1)$ by symmetry)

3. Community size $k \to \infty$ - TECHNICAL (needed for concentration arguments)

4. Spherical latent space $S^{d-1}$ - RESTRICTIVE (real networks may have different geometric structures)

5. Binary community membership - RESTRICTIVE (overlapping or soft communities more realistic)

6. Low-degree evidence relies on heuristics - TECHNICAL (rigorous computational hardness remains open)

   Open problems:

7. Close the polynomial gap in $p$ between upper bound $d = \tilde{O}(n^3 p^3)$ and lower bound $d = \tilde{\Omega}(n^3 p^2)$ for $k = n$ case

8. Extend to multiple communities or overlapping community structures while preserving geometric signal detection