Every Locale Has Its Rhythm: Endow Time-Series Transformers with Spatial Insights for Long-term Spatio-Temporal Forecasting

Abstract

Transformers have demonstrated promise in time-series forecasting, attributed to their superior capability of capturing temporal dependencies. Nevertheless, prevailing transformer models predominantly concentrate on the temporal dependencies within single-/multi-variate time series. This focus results in insufficient characterization of spatial correlations among time series. To bridge this gap, this paper introduces G raph T i me-Ser i es T ransformer (GïT), a novel approach aimed at enhancing long-term spatio-temporal forecasting. GïT judiciously integrates the principles of Transformer and Graph Neural Network (GNN). It designs a novel Vertex-wise Decoupled Patching scheme, where each univariate time series in the temporal graphs is segmented into subseries-level patches, which serve as input tokens to the Transformer. These patches are subsequently input into a Transformer encoder to generate representations of patches that capture temporal correlations. The Transformer representations of univariate time series are subsequently processed by Cross-vertex Multivariate Representation , where representations of univariate time series are re-associated to vertices in the temporal graphs and enhanced with Laplacian position encoding. The enhanced representations are further processed by a graph convolutional network to capture the spatial correlation between time series. GïT is evaluated over 6 spatio-temporal forecasting datasets spanning a variety of implementation domains. Experimental results demonstrate that GïT outperforms existing SOTA in terms of forecasting accuracy, achieving a maximum performance improvement of 15.9%.