Journal of Computer Technology and Software

Vol. 3 No. 7 (2024)
Articles

Representation Learning with Multi-Task Self-Supervision for Structurally Diverse Spatiotemporal Time Series Forecasting

pdf
  • Cong Nie
DOI: https://doi.org/10.5281/zenodo.18666198

Published 2024-10-30

How to Cite

Nie, C. (2024). Representation Learning with Multi-Task Self-Supervision for Structurally Diverse Spatiotemporal Time Series Forecasting. Journal of Computer Technology and Software, 3(7). https://doi.org/10.5281/zenodo.18666198
Creative Commons License

This work is licensed under a Creative Commons Attribution 4.0 International License.

Abstract

This study presents a self-supervised spatiotemporal joint forecasting method designed to address structural diversity, relational inconsistency, and the coupling of short-term and long-term dynamics in heterogeneous spatiotemporal time series. The method introduces a multi-task self-supervised mechanism that includes spatial encoding reconstruction, temporal sequence masking, and relational structure constraints. These components allow the model to learn latent dependencies across relations, nodes, and temporal scales without labeled data. The framework first employs a spatial encoder to capture multi-type relational features in heterogeneous structures, then uses a temporal encoder to model complex dynamic variations. Meanwhile, the self-supervised tasks provide additional structural constraints that enhance the model's ability to recognize structural differences and temporal patterns during training. To validate the effectiveness of the approach, this study designs multidimensional sensitivity experiments that analyze the effects of spatial encoding dimension, time window length, and the number of heterogeneous relation types. The results show that the method maintains stable modeling performance under different structural conditions and improves the efficiency of capturing key dependencies in heterogeneous spatiotemporal systems. Further analysis indicates that the self-supervised framework reduces reliance on labels while enhancing generalization through tasks such as structural reconstruction and temporal recovery. This gives the model stronger robustness and applicability in complex system modeling. Overall, the proposed method provides a unified and scalable forecasting framework for spatiotemporal data with multiple relations, multiple scales, and heterogeneous structures, and contributes to advancing modeling techniques for structurally complex systems.

pdf

References

  1. Bao Y, Huang J, Shen Q, et al. Spatial-temporal complex graph convolution network for traffic flow prediction[J]. Engineering Applications of Artificial Intelligence, 2023, 121: 106044.
  2. Wen H, Lin Y, Xia Y, et al. Diffstg: Probabilistic spatio-temporal graph forecasting with denoising diffusion models[C]//Proceedings of the 31st ACM international conference on advances in geographic information systems. 2023: 1-12.
  3. Ji J, Wang J, Huang C, et al. Spatio-temporal self-supervised learning for traffic flow prediction[C]//Proceedings of the AAAI conference on artificial intelligence. 2023, 37(4): 4356-4364.
  4. Yang Y, Guan Z, Wang Z, et al. Self-supervised heterogeneous graph pre-training based on structural clustering[J]. Advances in Neural Information Processing Systems, 2022, 35: 16962-16974.
  5. Zhang W, Wang H, Zhang F. Spatio-temporal Fourier enhanced heterogeneous graph learning for traffic forecasting[J]. Expert Systems with Applications, 2024, 241: 122766.
  6. P. Veličković, G. Cucurull, A. Casanova, A. Romero, P. Liò, and Y. Bengio, “Graph attention networks,” arXiv:1710.10903, 2017.
  7. G. Zerveas, S. Jayaraman, D. Patel, A. Bhamidipaty, and C. Eickhoff, “A transformer-based framework for multivariate time series representation learning,” in Proc. 27th ACM SIGKDD Conf. Knowledge Discovery & Data Mining (KDD), 2021, pp. 2114-2124.
  8. J. Y. Franceschi, A. Dieuleveut, and M. Jaggi, “Unsupervised scalable representation learning for multivariate time series,” in Advances in Neural Information Processing Systems (NeurIPS), vol. 32, 2019.
  9. S. Y. Shih, F. K. Sun, and H. Y. Lee, “Temporal pattern attention for multivariate time series forecasting,” Machine Learning, vol. 108, no. 8, pp. 1421-1441, 2019.
  10. D. Carpov, N. Chapados, and Y. Bengio, “N-BEATS: Neural basis expansion analysis for interpretable time series forecasting,” arXiv:1905.10437, 2019.
  11. M. Wang, “Multi-level attention and sequence modeling for dynamic user interest representation in real-time advertising recommendation,”2023.
  12. R. Fang, “Transaction network graph neural networks for automated and robust financial fraud detection in corporate auditing,” 2024.
  13. Y. Wang, “Integrating large language models and knowledge graphs for intelligent financial regulatory risk identification,” 2024.
  14. A. Xie, “Adaptive privacy-aware federated language modeling for collaborative electronic medical record analysis,” 2024.
  15. Y. Xing, “Enhancing advertising recommendation performance via integrated causal inference and exposure bias correction,” 2023.
  16. Wu Z, Pan S, Long G, et al. Graph wavenet for deep spatial-temporal graph modeling[J]. arXiv preprint arXiv:1906.00121, 2019.
  17. Zheng C, Fan X, Pan S, et al. Spatio-temporal joint graph convolutional networks for traffic forecasting[J]. IEEE Transactions on Knowledge and Data Engineering, 2023, 36(1): 372-385.
  18. Hu N, Zhang D, Xie K, et al. Graph learning-based spatial-temporal graph convolutional neural networks for traffic forecasting[J]. Connection Science, 2022, 34(1): 429-448.
  19. Xu Y, Cai X, Wang E, et al. Dynamic traffic correlations based spatio-temporal graph convolutional network for urban traffic prediction[J]. Information Sciences, 2023, 621: 580-595.