The purpose of this video is to dissect and learn about the Attention Is All You Need transformer model by using bare-bones PyTorch classes to forecast time series data.
Code Repo:
https://github.com/BrandenKeck/pytorch_fun
Very helpful:
https://github.com/oliverguhr/transformer-time-series-prediction/blob/master/transformer-singlestep.py
https://github.com/ctxj/Time-Series-Transformer-Pytorch
https://github.com/huggingface/transformers/blob/v4.40.1/src/transformers/models/time_series_transformer/modeling_time_series_transformer.py#L1179
Attention Is All You Need:
https://arxiv.org/pdf/1706.03762.pdf