Differentiable latent structure discovery for interpretable forecasting in clinical time series

Abstract

Abstract Background Timely, uncertainty-aware forecasting from irregular electronic health records (EHR) can support critical-care decisions, yet most approaches either impute to a grid or sacrifice interpretability. We introduce StructGP, a continuous-time multi-task Gaussian process that couples process convolutions with differentiable structure learning to uncover a sparse, ordered directed acyclic graph (DAG) of inter-variable dependencies while preserving principled uncertainty. We further propose LP-StructGP, which augments StructGP with latent pathways —shared, temporally shifted trajectories inferred via subject-specific coupling filters and a softmax gating mechanism—to capture cross-patient progression patterns. Both models are trained under sparsity and acyclicity constraints (augmented Lagrangian, Adam) using scalable low-rank updates: StructGP via exact marginal likelihood maximization and LP-StructGP via an online conditional marginal-likelihood objective. Results In controlled, correctly specified simulations, graph recovery improved with cohort size, with the median Structural Hamming Distance reaching zero at the largest cohort size, while pathway assignments showed high Adjusted Rand Index. Additional experiments showed reduced graph-recovery performance under MIMIC-IV-derived observation schedules. Our analysis establishes that the ordered StructGP graph is identifiable from the population marginal likelihood under the stated model, observation-design, and noise assumptions. For LP-StructGP, the corresponding result identifies the shared ordered inter-task graph conditionally on fixed subject-level pathway filters. On a MIMIC-IV septic shock cohort ( n = 1,008; norepinephrine, creatinine, mean blood pressure), StructGP improves short-horizon (6 h) forecasting over independent-task baselines (average RMSE 0.68 [95% CI: 0.63–0.74] vs. 0.88 [0.83–0.94]) and, with 15 additional inputs, markedly outperforms unstructured kernels (0.63 [0.58–0.69] vs. 3.02 [2.85–3.18]) with superior calibration (coverage 0.96 vs. 0.84). For long horizons (up to 6 days), LP-StructGP further reduces error for creatinine (RMSE 0.95 [0.88–1.03] vs. 1.17 [1.08–1.25]) and improves overall coverage (0.93 [0.93–0.94] vs. 0.91 [0.91–0.92]). On the PhysioNet Challenge (12k patients, 41 variables), StructGP attains competitive accuracy (MAE $$3.72{\times}10^{-2}$$ ) relative to a strong published comparator graph neural model. Conclusion These results show that structured process convolutions with latent pathways deliver interpretable, scalable, and well-calibrated forecasting for irregular clinical time series.