Spatiotemporal Swin-Transformer Network for Short Time Weather Forecasting

Earth Observatory is a growing research area that is using AI for short time forecasting, a Now-casting scenario. In this work, we tackle the challenge of weather forecasting using the video transformer network. In recent times, many variants of the vision transformer were explored, with major constraints being the computational complexity of Attention and the data hungry training. We explore the use of Video Swin-Transformer together with a carefully crafted augmentation scheme to tackle the data hungry transformer network. In addition, we use a gradual spatial reduction on the encoder side and cross-attention on the decoder. The proposed network is tested on the Weather4Cast2021 weather forecasting challenge data, which requires the prediction of 8 hours ahead future frames (4 per hour) from an hour weather product sequence. The model results in a highly competitive performance on both the validation and test datasets. The code is available online at https://github.com/bojesomo/Weather4cast2021-SwinEncoderDecoder.