Spatiotemporal Vision Transformer for Short Time Weather Forecasting

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

5 Scopus citations


Weather forecasting is a critical research area that may lead to serious consequences, if not done accurately. In order to manage the impact of adverse weather effects short time forecasting is employed, since it can be highly accurate to predict a short time, e.g., hours, into the future. In this work, we tackle the challenge of short time weather forecasting using a novel approach based on a modified UNet based model. Specifically, all convolution-based building blocks were replaced by 3D shifted window transformers in both encoder and decoder branches. Shifted window transformers greatly reduce computational complexity requirements, a major constraint of self-attention, without sacrificing performance. To support the pre-training of the transformer-based model a carefully crafted augmentation scheme was proposed. The model was tested on the IEEE Big Data Weather4cast Competition data, which requires the prediction of 8 hours ahead frames (4 per hour) from an hourly weather product sequence. We show the importance of including other weather products in encouraging spatial generalization, while this may not be optimal for temporal generalization. The model demonstrates highly competitive performance on both the validation and test datasets. The code is available online at

Original languageBritish English
Title of host publicationProceedings - 2021 IEEE International Conference on Big Data, Big Data 2021
EditorsYixin Chen, Heiko Ludwig, Yicheng Tu, Usama Fayyad, Xingquan Zhu, Xiaohua Tony Hu, Suren Byna, Xiong Liu, Jianping Zhang, Shirui Pan, Vagelis Papalexakis, Jianwu Wang, Alfredo Cuzzocrea, Carlos Ordonez
PublisherInstitute of Electrical and Electronics Engineers Inc.
Number of pages6
ISBN (Electronic)9781665439022
StatePublished - 2021
Event2021 IEEE International Conference on Big Data, Big Data 2021 - Virtual, Online, United States
Duration: 15 Dec 202118 Dec 2021

Publication series

NameProceedings - 2021 IEEE International Conference on Big Data, Big Data 2021


Conference2021 IEEE International Conference on Big Data, Big Data 2021
Country/TerritoryUnited States
CityVirtual, Online


  • Now-casting
  • SwinUNet3D Architecture
  • Video Swin-Transformer
  • Weather forecasting


Dive into the research topics of 'Spatiotemporal Vision Transformer for Short Time Weather Forecasting'. Together they form a unique fingerprint.

Cite this