1. Motivation (2–3 sentences): Electronic health records contain multimodal clinical data (structured measurements and clinical notes) that are observed irregularly over time. The observation process itself depends on the patient’s latent health state - sicker patients receive denser monitoring - making missingness informative. While methods exist for handling missing data, how to extract information from the observation patterns themselves remains underexplored.

2. Mathematical setup: Consider a finite-horizon partially observable Markov decision process (POMDP) $M = (S, A, O, P, \Omega, R, \gamma, H)$ where $S$ is latent state space, $A$ is action space, $O$ is observation space. At decision step $h$, the agent observes:

   \[o_h = (y_h^s, m_h^s, y_h^t, m_h^t) \in O\]

   For structured data, $y_h^s \in \mathbb{R}^{|T_h| \times D}$ denotes measurements at times $T_h$ and $m_h^s \in {0,1}^{|T_h| \times D}$ indicates missingness. For text, $y_h^t$ is raw text and $m_h^t$ indicates text availability. Key assumptions:

   1. True patient state $s_h \in S$ is not directly observed
   2. Observations are sparse and irregular
   3. Observation process depends on latent patient condition (MNAR)
   4. Different modalities have distinct recording processes

3. Toy example: When $D=2$ with heart rate and blood pressure, a low-acuity patient might have measurements every 8 hours with random missingness, while a high-acuity patient receives hourly monitoring. The observation pattern $(m_{h,HR}, m_{h,BP})$ itself signals patient severity.

4. Formal objective: Learn encoder $g_\theta$ and policy $\pi$ that maximize expected return:

   \[J(\pi) = \mathbb{E}_{\tau \sim \pi}\left[\sum_{h=0}^{H-1} \gamma^h r_h\right]\]

   where $s_h = g_\theta(I_h)$ and $I_h = {x, o_1, \ldots, o_h}$.

The method combines multimodal MNAR-aware encoding with Bayesian filtering for dynamic patient representation learning.

1. MNAR-Aware Structured Encoding: Extend GRU-D with explicit decay factors and MNAR features:

   \[\xi_{\phi,u} = \exp(-\max(0, W_{\phi,\xi} \cdot \text{mean}(\delta_u) + b_{\phi,\xi}))\] \[\hat{y}_{u}^{s,d} = m_u^{s,d} \cdot y_u^{s,d} + (1-m_u^{s,d}) \cdot [\xi_{y,u}^d \cdot y_{u'}^{s,d} + (1-\xi_{y,u}^d) \cdot \mu^d]\]

   where $\delta_u$ is time-since-last-observation and $\mu^d$ is empirical mean.

2. Text Fusion with Documentation Process: Create documentation factor:

   \[\eta_h^t = \text{MLP}(m_h^t, \delta_h^t, \kappa_h^t)\] \[F_h^{\text{doc}} = \text{GRU}(F_{h-1}^{\text{doc}}, \eta_h^t)\]

   Then fuse via gating:

   \[\phi_h = \text{LayerNorm}(\bar{\phi}_h^s + g_h \odot (\hat{\phi}_h^t - \bar{\phi}_h^s))\]

3. Action-Conditioned Latent Dynamics: Use VAE with action-conditioning:

   \[z_{h+1} \sim p_\theta(z_{h+1} | z_h, \phi_h, a_h)\]

4. Policy Learning: Apply Implicit Q-Learning (IQL) on posterior states $s_h = \phi_h + \text{proj}(z_h)$.

   Toy example application: For a 2-variable case with heart rate missing for 6 hours, the method would decay the stale heart rate value, incorporate the gap duration as an MNAR feature, and update the belief state based on this observation pattern.

Step 1 — Prior work:

1. “Recurrent Neural Networks for Multivariate Time Series with Missing Values” (Che et al., 2018) - introduced GRU-D for handling irregular sampling in clinical time series
2. “The Artificial Intelligence Clinician learns optimal treatment strategies for sepsis in intensive care” (Komorowski et al., 2018) - established offline RL framework for sepsis treatment

3. “Causal representation learning from multimodal clinical records under non-random modality missingness” (Liang et al., 2025) - modeled informative missingness in multimodal EHR but without temporal dynamics.

   Step 2 — Delta: This paper adds three key components:

4. temporal modeling of MNAR patterns across modalities over time
5. explicit documentation-process modeling for clinical text

6. action-conditioned latent dynamics that preserve gradient flow for policy learning. The theoretical contribution (Theorem 1) shows why action-conditioning is necessary for credit assignment.

   Step 3 — Theory-specific assessment:

   • The main theoretical result (Theorem 1) is somewhat predictable - it formalizes the intuitive fact that without action-conditioning, gradients can’t flow from future rewards to current actions
   • The proof technique is routine, using basic chain rule calculations
   • No lower bounds are established for the RL performance gains
   • The MNAR modeling extensions are incremental improvements over GRU-D

   Verdict: INCREMENTAL — solid engineering contribution combining existing techniques with modest theoretical insight, but lacks fundamental algorithmic innovation.

The main theoretical result (Theorem 1) uses a straightforward gradient flow argument:

1. Key setup: Under action-independent dynamics where $z_{h+1} = f(z_h, \phi_h, \omega_h)$ with $\omega_h \perp a_h$, and deterministic observation encoding $\partial \phi_{h’}/\partial a_h = 0$ for $h’ > h$.

2. Core inequality: For policy gradient of future rewards:

   \[\frac{\partial}{\partial \pi} \mathbb{E}\left[\sum_{h'=h+1}^{H-1} \gamma^{h'} r_{h'} \mid s_h, a_h\right] = \sum_{h'=h+1}^{H-1} \gamma^{h'} \frac{\partial r_{h'}}{\partial s_{h'}} \frac{\partial s_{h'}}{\partial a_h}\]

3. Chain rule expansion: Since $s_{h’} = g_\theta(\phi_{h’}, z_{h’})$:

   \[\frac{\partial s_{h'}}{\partial a_h} = \frac{\partial g_\theta}{\partial \phi_{h'}} \frac{\partial \phi_{h'}}{\partial a_h} + \frac{\partial g_\theta}{\partial z_{h'}} \frac{\partial z_{h'}}{\partial a_h}\]

4. Vanishing gradients: The first term is zero by assumption. The second term is also zero because under action-independent dynamics, $z_{h’}$ doesn’t depend on $a_h$ for $h’ > h$.

5. Terminal result: With sparse terminal rewards, this makes the policy gradient zero for all non-terminal steps, preventing learning of long-term action consequences.

   The proof is elementary but serves to motivate the action-conditioned VAE architecture. The MNAR modeling techniques are primarily empirical extensions without theoretical guarantees.

Main datasets: MIMIC-III (15,415 ICU stays, 13.2% mortality), MIMIC-IV (32,837 stays, 11.8% mortality), eICU (24,562 stays, 12.1% mortality). All use 72-hour sepsis cohorts with 4-hour decision intervals.

Key baselines: AI Clinician, BCQ, CQL, DDPG with Clinician Supervision, SBCQ, MedDreamer, plus clinician behavior.

Main results: OPL-MT-MNAR achieves FQE 0.679 vs clinician 0.528 on MIMIC-III, 0.634 vs 0.521 on MIMIC-IV, and 0.604 vs 0.534 on eICU. For outcome prediction, achieves AUROC 0.886 for post-72-hour mortality vs 0.867 for MedDreamer.

Ablations show text provides substantial value (structured-only: 0.574 → nursing notes: 0.624 → full: 0.679). Controlled study shows MNAR+DocProcess provides dominant +33.9% improvement over strong baseline.

Gains are largest in high-acuity patients where MNAR patterns are most informative. Bootstrap confidence intervals: [0.673, 0.686] for OPL-MT-MNAR vs [0.520, 0.536] for clinicians.

1. Policy evaluation relies on off-policy estimation from observational data rather than prospective validation - RESTRICTIVE (limits clinical deployment confidence despite bootstrap intervals)

2. Treatment discretization into 9 actions and 4-hour intervals - TECHNICAL (simplifies learning but leaves finer-grained control unexplored)

3. Unobserved confounding from verbal communication and bedside assessments not in EHR - NATURAL (standard limitation in EHR-based studies)

4. Limited to three U.S. ICU datasets - TECHNICAL (broader healthcare system validation needed for generalization)

5. Joint multitask learning may suffer negative transfer as tasks scale - TECHNICAL (more auxiliary objectives could hurt specialized performance)

6. Action-conditioned latent dynamics require careful posterior-collapse monitoring - TECHNICAL (VAE training instability)

   Open problems:

7. How to extend MNAR modeling to real-time settings where observation patterns evolve dynamically during treatment
8. Developing theoretical guarantees on when and how much MNAR observation patterns improve policy learning compared to ignoring missingness patterns

1. Motivation: High-frequency limit order books (LOBs) exhibit complex volatility patterns that standard models fail to capture accurately. The volatility signature plot—showing realized volatility increasing with sampling frequency—reveals underlying microstructure dynamics driven by order flow clustering and state-dependent feedback.

2. Mathematical setup: The Extended State-Dependent Hawkes Process models event arrivals on a probability space where {Tn} are event times, {En} ∈ E are event types (14 categories including marketable limit orders), and {Xn} ∈ X are LOB states (spread values). The intensity is:

   \[\tilde{\lambda}_{ex}(t) = \phi_e(X(t), x)\left(\nu_e + \sum_{e',x'} \int_{[0,t)} k_{e'e}(t-s, x')d\tilde{N}_{e'x'}(s)\right)\]

   The transition weights satisfy $\phi_e(x,x’) \in [0,1]$ with binary constraint $\Phi_{e,x} := \sum_{x’} \phi_e(x,x’) \in {0,1}$.

   Assumptions:

   1. Exponential kernels: $k_{e’e}(t,x) = \alpha_{e’xe}\exp(-\beta_{e’xe}t)$
   2. Binary admissibility: events are either allowed ($\Phi_{e,x} = 1$) or forbidden ($\Phi_{e,x} = 0$) in each state
   3. State transitions are Markovian and deterministic given order type

3. Toy example: When $x=1$ (minimum spread), price-improving orders (ALS, ALB) are physically impossible, so $\phi_{ALS}(1,2+) = 0$. When $x=2+$ (wide spread), marketable limit orders can close the spread with $\phi_{MLO}(2+,1) \approx 0.98$.

4. Formal objective: Estimate parameters $(\phi, \nu, \theta)$ to maximize the log-likelihood:

   \[\ln L(\phi, \nu, \theta) = \sum_n \ln \lambda_{E_n}^{\dagger}(T_n) - \int_0^T \sum_e \lambda_e^{\dagger}(t)dt\]

The method involves a two-step estimation procedure leveraging separability:

1. Apply KKT conditions to estimate transition probabilities:

   \[\hat{\phi}_e(x,x') = \begin{cases} 0 & \text{if no transitions observed} \\ \frac{\text{count}(x \to x' \mid e)}{\text{count}(x,e)} & \text{otherwise} \end{cases}\]

2. Estimate Hawkes parameters using recursive updates for exponential kernels:

   \[R_{e'xe}(n) = R_{e'xe}(n-1)\exp(-\beta_{e'xe}(T_n - T_{n-1})) + \mathbf{1}_{E_{n-1}=e',X_{n-1}=x}\alpha_{e'xe}\]

3. Define local branching ratio for instability detection:

   \[n_{ex} = \sum_{e'} \frac{\alpha_{e'xe}}{\beta_{e'xe}}\]

4. Update mid-price with state-dependent impact: $M(T_n) = M(T_{n-1}) + \Delta M(e,x)$

   Applied to toy example: With $x=1$ and arrival of ALS event, the KKT estimator correctly sets $\hat{\phi}_{ALS}(1,2+) = 0$ since price improvement is impossible. The recursive formula updates only admissible intensities, preventing spurious parameter drift.

Prior work:

• Morariu-Patrichi & Pakkanen (2022): Established rigorous foundation for state-dependent Hawkes with row-sum constraint Φ_e,x = 1
• Bacry et al. (2015): Standard Hawkes process framework for market microstructure endogeneity
• Filimonov & Sornette (2012): Introduced branching ratio concept for market criticality analysis

Delta: This paper relaxes the rigid row-sum constraint to allow Φ_e,x ∈ {0,1}, permitting “state disappearances” when orders are physically impossible. Uses KKT conditions to prove likelihood separability remains intact despite relaxed constraints.

Theory assessment:

• Main result is predictable extension of prior work - allowing zero probabilities is natural for physical constraints
• Proof technique is routine application of KKT conditions to constrained MLE
• No comparison to known lower bounds provided; claims of “super-criticality” (ρ > 1) while maintaining stability are not rigorously proven theoretically

The empirical validation showing unconstrained models “explode” while this model remains stable suggests some value, but the theoretical novelty is limited to relaxing an artificial constraint.

Verdict: INCREMENTAL — solid application of standard optimization theory to remove an unnatural constraint in existing state-dependent Hawkes framework.

The main proof strategy uses Karush-Kuhn-Tucker conditions for constrained optimization:

1. Construct the Lagrangian for the constrained MLE problem with physical boundary constraints

2. Apply KKT stationarity conditions to derive the optimal estimator

   Key inequality driving separability (Theorem 3.1):

   \[\int_0^T \sum_e \lambda_e^{\dagger}(t)dt = \int_0^T \sum_e \Phi_{e,X(t)} \lambda_e(t)dt\]

   Since $\Phi_{e,X(t)} \in {0,1}$ is deterministic (depends only on LOB geometry), the compensator integral separates from the transition probability estimation.

   Key equation for KKT-derived estimator (Theorem 3.2):

   \[\hat{\phi}_e(x,x') = \frac{\sum_{n=1}^N \mathbf{1}_{\{x_{n-1}=x, e_n=e, x_n=x'\}}}{\sum_{n=1}^N \mathbf{1}_{\{x_{n-1}=x, e_n=e\}}}\]

   Technical insight: The binary constraint $\Phi_{e,x} \in {0,1}$ acts as a “gate” that completely turns off intensity during inadmissible periods, preventing the accumulation of spurious residual mass that would bias parameter estimates in unconstrained models. The proof leverages the deterministic nature of these physical constraints to maintain likelihood separability.

Dataset: Three months of MUFG (8306) tick data from Nikkei NEEDS (Oct-Dec 2020), 100-microsecond resolution, morning sessions 09:15-11:15.

Key baselines: Standard SD-Hawkes, Constant Hawkes, Poisson models.

Key results:

• Volatility signature plot: Only ExsdHawkes reproduces the empirical upward slope without explosive divergence
• Branching ratios: Equilibrium state ρ(1) ≈ 0.19, disequilibrium ρ(2+) ≈ 2.67 (super-critical)
• Residual analysis: ExsdHawkes maintains i.i.d. Exp(1) residuals across all state-event pairs
• Transition probabilities: Correctly identifies zero probability for physically impossible events (ALS/ALB at minimum spread)

SD-Hawkes shows explosive volatility growth and biased residuals due to intensity leakage through inadmissible states. The empirical evidence strongly supports the necessity of physical constraints for model stability.

Limitations:

1. Binary state discretization (x ∈ {1, 2+}) is overly simplistic - RESTRICTIVE (ignores fine-grained spread variations in practice)

2. Exponential kernel assumption for computational tractability - TECHNICAL (likely removable with more sophisticated numerical methods)

3. Focus on single asset (MUFG) over limited time period - TECHNICAL (straightforward to extend but computational cost increases)

4. State transitions assumed deterministic given order type - NATURAL (consistent with LOB mechanics)

5. Morning session only to avoid microstructure breaks - TECHNICAL (full-day modeling requires handling market closure effects)

   Open problems:

6. Extend to multi-asset portfolio setting where cross-asset state dependencies create higher-dimensional constraint surfaces
7. Theoretical proof of global stability despite local super-criticality - current argument relies on empirical observation of mean reversion

Motivation: In machine learning, stochastic gradient descent (SGD) converges to noisy approximations of true optima, making uncertainty quantification critical. Estimating the asymptotic covariance matrix enables confidence intervals and statistical inference, but requires methods compatible with SGD’s computational efficiency.

Mathematical setup: Let

\[(X_i)_{i=1}^n\]

be SGD iterates generated by:

\[x_i = x_{i-1} - \eta_i \nabla f(x_{i-1}, \xi_i)\]

where $\xi_i$ are i.i.d. samples from distribution $\Pi$, step sizes $\eta_i = \eta i^{-\alpha}$ with $\eta > 0$ and $\alpha \in (0.5, 1)$.

The averaged SGD satisfies:

\[\sqrt{n}(\bar{x}_n - x^*) \Rightarrow N(0, \Sigma)\]

where

\[\Sigma = \nabla^2 F(x^*)^{-1} E[\nabla f(x^*, \xi) \nabla f(x^*, \xi)^T] \nabla^2 F(x^*)^{-1}\]

Assumptions:

1. $F(x)$ is continuously differentiable and strongly convex with parameter $\mu > 0$
2. $f(x, \xi)$ is continuously differentiable with uniformly integrable gradients
3. Gradient noise satisfies $E_\xi |\nabla f(x^*, \xi)|_2^q < \infty$ for some $q \geq 8$

4. $\nabla f(x, \xi)$ is stochastically Lipschitz continuous

   Toy example: When $d=1$ and $f(x, \xi) = (x - \xi)^2/2$ with $\xi \sim N(x^*, 1)$, this reduces to estimating $\Sigma = 1$ from the SGD sequence $x_i = (1 - \eta_i)x_{i-1} + \eta_i \xi_i$.

   Formal objective: Estimate the limiting covariance matrix:

   \[\hat{\Sigma}_n \to \Sigma \text{ as } n \to \infty\]

The method constructs a bias-reduced estimator using batch-wise corrections. Define batches $B_i = {i - \ell_i + 1, \ldots, i}$ with sizes $\ell_i = O(i^\alpha \log i)$.

The de-biased estimator is:

\[\hat{\Sigma}_n = \frac{1}{n} \sum_{i=1}^n \left[ (x_i - \bar{x}_n)\left(\sum_{k=i-\ell_i+1}^i x_k - \ell_i \bar{x}_n\right)^T + \left(\sum_{k=i-\ell_i+1}^i x_k - \ell_i \bar{x}_n\right)(x_i - \bar{x}_n)^T - (x_i - \bar{x}_n)(x_i - \bar{x}_n)^T \right]\]

The key insight comes from the identity:

\[\sigma_n = \frac{1}{n} \sum_{i=1}^n E\left[2(X_i - E(X_i))\sum_{k=1}^i (X_k - E(X_k)) - (X_i - E(X_i))^2\right]\]

Block-based batching scheme: Define sequence $(a_m)_{m \geq 1}$ with $a_1 = 1$ and:

\[a_m - a_{m-1} + 1 = \lfloor a_m^\alpha \log(a_m) \rfloor\]

Each batch spans two consecutive blocks for online updates.

Application to toy example: For $d=1$ with $f(x,\xi) = (x-\xi)^2/2$, the oracle estimator becomes:

\[\hat{\sigma}_n = \frac{1}{n} \sum_{i=1}^n \left[2x_i \left(\sum_{k=i-\ell_i+1}^i x_k\right) - x_i^2\right]\]

This directly estimates $\text{Var}(\sqrt{n}\bar{x}_n) = 1$ with convergence rate $n^{(\alpha-1)/2}\sqrt{\log n}$.

Step 1 — Prior work:

1. “Statistical inference for model parameters in stochastic gradient descent” (Chen et al. 2020) - introduced plug-in estimator requiring Hessian information, achieving rate $O(n^{-\alpha/2})$
2. “Online covariance matrix estimation in stochastic gradient descent” (Zhu et al. 2023) - developed Hessian-free batch-means estimator with rate $O(n^{(\alpha-1)/4})$

3. “A single-pass algorithm for spectrum estimation with fast convergence” (Xiao & Wu 2011) - bias-reduction for stationary time series, achieving different theoretical guarantees

   Step 2 — Delta: This paper adds a bias-reduction technique to online batch-means estimation, improving convergence rate from $n^{(\alpha-1)/4}$ to $n^{(\alpha-1)/2}\sqrt{\log n}$ while remaining Hessian-free and fully online.

   Step 3 — Theory-specific assessment:

   • Main theorem is predictable given prior bias-reduction work, but applying it to non-stationary SGD iterates requires non-trivial technical development
   • Proof technique combines known martingale analysis with novel batch-size selection $\ell_i = O(i^\alpha \log i)$ to balance bias-variance tradeoff
   • Bound is shown to be tight in simple mean estimation case via Theorem 3
   • No known lower bounds exist for this specific problem

   The improvement is genuine but incremental - it’s a natural extension of bias-reduction ideas to SGD covariance estimation.

   Verdict: INCREMENTAL — solid improvement of convergence rate using established bias-reduction principles, but represents expected technical progress rather than conceptual breakthrough.

The proof uses multi-stage approximation combined with martingale analysis:

1. Linear approximation: Replace nonlinear SGD recursion with linear system:

   \[U_i = (1 - \eta_i A)U_{i-1} + \eta_i \epsilon_i\]

   where $A = \nabla^2 F(x^*)$ and $\epsilon_i$ are martingale differences.

2. i.i.d. approximation: Further approximate with i.i.d. noise sequence:

   \[\tilde{U}_i = (1 - \eta_i A)\tilde{U}_{i-1} + \eta_i \epsilon_i^*\]

   where $\epsilon_i^* = -\nabla f(x^*, \xi_i)$ are i.i.d.

3. Key decomposition:

   \[\|\hat{\Sigma}_n - \Sigma\| \leq \|\hat{\Sigma}_n - \hat{\Sigma}_{n,\delta}\| + \|\hat{\Sigma}_{n,\delta} - \hat{\Sigma}_{n,U}\| + \|\hat{\Sigma}_{n,U} - \hat{\Sigma}_{n,\tilde{U}}\| + \|\hat{\Sigma}_{n,\tilde{U}} - \Sigma\|\]

4. Batch size analysis: Critical choice $\ell_i = C i^\alpha \log i$ ensures:

   \[|E(\hat{\sigma}_n - \sigma_n)| \lesssim \frac{1}{n} \sum_{i=1}^n \exp(-\eta i^{-\alpha} \ell_i)\]

   becomes negligible when $C > \eta^{-1}$.

5. Moment bounds using Rosenthal inequality:

   \[\left\|\sum_{i=1}^n \xi_i\right\|_p \leq C_p \max\left(\left(\sum_{i=1}^n E|\xi_i|^2\right)^{1/2}, \left(\sum_{i=1}^n E|\xi_i|^p\right)^{1/p}\right)\]

6. Exponential weight analysis via Lemma 8:

   \[|Y_j^{(i)}| \asymp \exp\left(\frac{\lambda \eta}{1-\alpha}(j^{1-\alpha} - i^{1-\alpha})\right)\]

   handles correlation structure in SGD iterates.

Datasets: Synthetic data with three models:

1. Linear regression: $b_i \sim N(a_i^T x^*, 1)$ with squared loss
2. Logistic regression: $b_i \in {1,-1}$ with logistic loss

3. Expectile regression: $\tau$-expectile loss with $b_i \sim N(a_i^T x^*, 1)$

   Baselines: Online batch-means estimator (Zhu et al. 2023)

   Key numbers: For $d=20$, $n=200,000$:

   • Linear: De-biased error 3.82±0.33 vs Online BM 5.66±0.74
   • Logistic: De-biased error 33.92±1.28 vs Online BM 41.98±3.51
   • Expectile: De-biased error 4.02±0.29 vs Online BM 6.18±0.77

   Consistent 20-30% improvement in estimation accuracy across all settings. Learning rate $\eta_i = 0.5 i^{-0.505}$ used throughout. Results averaged over 500 independent runs.

Limitations:

1. Strong convexity assumption - TECHNICAL (commonly used in SGD theory, likely removable via standard techniques)
2. Stochastic Lipschitz condition with finite moments - TECHNICAL (standard regularity condition)
3. Polynomial step size decay $\eta_i = \eta i^{-\alpha}$ - NATURAL (widely used in practice)
4. Batch size requires logarithmic factor - TECHNICAL (artifact of proof technique, possibly improvable)

5. Analysis limited to operator norm - TECHNICAL (extends to other matrix norms but constants unknown)

   Open problems:

6. Extension to non-convex optimization where SGD exhibits more complex dynamics and limiting distributions may differ
7. Dimension-dependent convergence rates and optimal constants for high-dimensional problems where $d$ grows with $n$